Editors and publishers are always curious to learn how their journal is performing compared to others in the field. They are also keen to discover whether the content they are publishing is attracting citations. In this piece, we would like to share with you a number of visualization techniques that can help to generate insights into journal performance.
Term mapping
How can you determine what the ‘hot’ topics are in a specific journal, group of journals or subject area? Or, more specifically, which topics have shown active growth and strong impact in research output (published articles) in recent years? To answer this question, we developed a new visualization tool in collaboration with the CWTS research group, which specializes in bibliometrics at the University of Leiden. The tool has access to all journals and conference proceedings indexed in Scopus. Drawing on this information, it can generate maps revealing the relationships between terms used in titles and abstracts of articles published in one or more selected journals. It does this with the help of a computer program called VOSviewer (1).
How is a term map created?
There are a number of steps involved in producing a term map.
- First we need to determine which journal or journals should be included. If a group of journals or a subject area is the focus of the analysis, a keyword search in Scopus can help with this.
- Once the journal(s) have been chosen, the tool performs an analysis of the words and phrases found in the titles and abstracts of articles over a specified period of time (e.g. within the last two, five or 10 years). The publication and citation windows can have separate values, so that it’s also possible to determine how well content published in a specific year has been cited in the years since publication.
- After a map is generated, it can be checked for uninformative terms, such as publisher or society names, and generic terms such as “literature”, “presentation”, or “feature”. These can be removed and, if needed, a new version of the map can be created.
Clusters of co-occurring terms
The map shown in Figure 1 is known as a co-occurrence cluster map. Every term that occurs at least five times in the titles and abstracts of articles in the selected journals is represented by an individual node on the map. The larger the node, the more articles contain the term and the smaller the space between the terms, the more often they tend to co-occur. However, it is important to note that this is a 2D representation of a multi-dimensional network, so the proximity of terms cannot perfectly reflect the relationship in all cases. Finally, the terms are colored into clusters of terms that tend to co-occur.
- Green (middle and top left) related to statistics and experiments;
- red cluster (right-hand side) related to nursing education;
- blue cluster (bottom left) related to surgery; and
- yellow cluster (left) related to clinical trials and literature reviews.
- The map in Figure 1 was generated for a group of six selected Nursing journals, and is based on roughly 1,900 articles and reviews published in these journals from 2009 to 2012. The map clearly contains four main clusters of co-occurring terms:
Field expertise can help appropriately check and name the clusters, as well as predict which clusters are likely to contain the most highly-cited content, and why.
Figure 1 - Journal term co-occurrence cluster similarity map for a group of six selected Nursing journals from 2009-2012. Source: Scopus.
Highly-cited terms
The next step in determining hot topics in the field is to check which terms are relatively well cited in comparison to the rest of the content published in the journal(s). This can be done by changing the coloring in the cluster map to show the average citation impact of the articles containing that term, relative to the average citation impact (1.00) of all articles included in the map (Figure 2). As older publications have had more time to be cited, the citations are normalized by year of publication to make a fair comparison possible. In Figure 2, terms with above average citation impact are colored red, terms with average citation impact are green and terms with below average citation impact are shown in blue.
Figure 2 - Journal term co-occurrence citation impact map for a group of six selected Nursing journals from 2009-2012. Source: Scopus
We can clearly see that the relatively highly-cited terms tend to occur to the left of the map. These are terms found mainly in the yellow and green clusters in Figure 1, related to experiments (green) and clinical trials (yellow). Highly-cited terms in these areas include:
- Nurse staffing, self-esteem and statistical terms (green cluster, top left).
- Depressive symptoms, meta-analysis, pregnancy and controlled trials (yellow cluster, left-hand side).
Hot topics?
Finally, a Scopus keyword search can be performed for the terms in the map with the highest relative citation impact, to determine if these were isolated occurrences. The outcome of this keyword search, restricted to the Nursing field, confirmed that there were at least four areas in this analysis which had a compound annual growth rate (CAGR) of more than 5 percent, which indicates that there was an above average increase in the number of papers published in these areas over the past five years, as the average CAGR is 3–5 percent (see Table 1).
Table 1 – In Figure 2, relatively highly-cited terms were identified. In this table we list the number of papers to feature those terms alongside their compound annual growth rates. Source: Scopus
The Scopus keyword search confirmed that the topics suggested by the map were indeed topics that have been attracting attention in the field. Although this specific map at field level is somewhat generic, it does provide a general idea of where to look for hot topics in more detail.
One editor’s experiences
Dr. Paul H. Gobster is a Research Social Scientist for the US Department of Agriculture (USDA) Forest Service. He has just stepped down after four years as co-Editor-in-Chief of Elsevier’s Landscape and Urban Planning, remaining on the journal’s Board as Associate Editor. He and his colleagues used term maps to aid the development of an editorial for the journal’s 40th anniversary (2).
Dr. Gobster said: “We identified important concepts and themes represented in its published content and developed a time-series of four maps to qualitatively describe changes across each successive decade.
“The term maps were relatively easy to interpret and produced visualizations that were suitable for presentation to readers within our editorial. I believe the term maps have additional value for journal administrative and strategic planning functions — the clustering can help to clarify thematic content for manuscript classification and assignment of submissions to Associate Editors, and the clusters and specific terms (their presence, positions, and any changes over time) may help to identify enduring and emerging subthemes of work.”
The benefits of journal mapping
While term maps are used to highlight the topics published within a journal or discipline, journal mapping can be used to examine a journal’s position and scope and its interactions with other journals in the field. As with term maps, Scopus can provide the source data, ensuring the analysis draws on all indexed journals.
These journal maps are formed using citation links. A citation from a paper published in one journal to a paper published in another establishes that their respective contents are relevant to each other, and suggests a level of similarity between the two. In any given time period, a journal tends to contain citations to many other journals, and those it cites the most should be the journals with which it is most closely related. For instance, if Journal A provides many citations to Journal B and only a few to Journal C, this is a sign that it has a stronger connection to Journal B. If over time the balance shifts so that it begins to provide more citations to Journal C, this indicates that the scope of the journals or structure of the field is changing and it is becoming progressively more related to Journal C. When the citation links are built up over many more journals than in this simplified example, a map is a convenient way to display the links and see how journals interact to form larger groups.
See Figure 3 for an example of a journal map based upon the same six Nursing journals used in the term maps examples above.
Figure 3 - Journal map based on a group of six selected Nursing journals from 2009-2012.
Each journal on the map is displayed as a node (circle), with size determined by the average citations to that journal's papers in the time period. You can see in Figure 3 that the general Medicine journals included in the map have far higher average citation impact than the other journals. The selected journals are in blue and all fall into the region of core Nursing journals, while other journals are in grey and included because of their citation links to these seed journals. Citation relationships are shown as edges (lines) of varying thickness. These citation relationships are normalized by the number of citations received by the cited journal and by the number of citations given by the citing journal. The thicker the line, the higher the proportion of citations represented.
In this example map, key areas of different health science specialties have been labelled based on the journal groups. This allows you to see the links between broader specialties as well as individual journals. These groupings will tend to be fairly stable, but comparing maps based on different time periods allows you to identify newly-emerging journals in a given area or the changing research relationships that lead one topic area to become more relevant to another over time.
The citation environment in which a journal sits is unique and dynamic, and analysis of this can be used as an objective means for determining the competitive position of an established journal in a research field.
Using the maps to support your work
Both term mapping and journal mapping can help to benchmark the journal against competitors and provide useful insights for editorial board meetings. While a few strategic reasons for using these analytical tools have been suggested in the text above, their real advantage lies in how adaptable they are to different research questions. If you would like to know more about how these tools can help you, or other analytical tools to provide insight into the position of your journal, please contact your publisher.
References
(1) Van Eck, N.J., & Waltman, L. (2010) “Software survey: VOSviewer, a computer program for bibliometric mapping”, Scientometrics, Vol 84, No. 2, pp. 523–538.
(2) Gobster, P.H. (2014) “(Text) Mining the LANDscape: Themes and trends over 40 years of Landscape and Urban Planning”, Landscape and Urban Planning, Vol 126, pp. 21–30.
Author biographies
Dr. Daphne van Weijen
Dr. Daphne van Weijen, PhD, joined Elsevier’s Research & Academic Relations department in 2012. Part of her role as a Publishing Information Manager is to advise publishers and editors on ways to improve the success and quality of their journals, using bibliographic data and visualization techniques. Daphne has a background in Sociolinguistics and Language Teaching, and completed her PhD research on First and Second Language Writing Processes at Utrecht University in 2009.
Matthew Richardson
Matthew Richardson works within Elsevier’s Research & Academic Relations department as a Publishing Information Manager. He studies scientific research through the lenses of publication and citation trends, with a focus on the information flows that construct and change research disciplines. His particular focus is on visualizing networks of scientific publications and fields, to enable a deeper understanding of how journals and disciplines relate to one another. Matthew completed a Master’s Degree in Writing from the University of Warwick, UK, before joining Elsevier’s Oxford office in 2010.