FAQs: Measuring Journal Impact

I’m putting together my tenure file and need quantitative information on the impact and relative disciplinary importance of a few journals in which I’ve published. What’s my best option for proceeding?

***

I get this question a lot, both directly from SOC faculty and when I’m working at the reference desk during the summer. Like most library questions, it has an “short” answer and a “long” answer, depending on how much the person asking really wants to know (or has time to know).

This is mind, let’s start with…

THE 2-MINUTE VERSION OF THE LONG ANSWER:

There’s actually a good deal of controversy over the validity of “impact factor” as an accurate measure of the importance of scholarly work.

The standard formula for impact, which has been around since the 1950s, is based on the number of academic citations that a journal receives in one year, divided by to the total number of items published by the journal in the prior two years. However, modern critics of impact factor have been quick to point out that this calculation does not account for many alternative metrics of influence, such as references to articles on websites, through social media, and in the popular press. Moreover, they argue, traditional impact factor as a purely quantitative measure fails to capture more qualitative and informal forms of scholarly recognition which are otherwise very prominent in everyday academic life.

This answer thus leads us to slew of additional “long” questions. For instance: what is the best measure of impact and influence given the current scholarly communication landscape? How do on-going changes in educational technologies and social media affect the future of academic bibliometrics?

Further resources:

***

The long answer to this question is clearly important, and deserves further discussion by academic librarians and faculty alike. However, practically speaking, it’s usually more important to provide patrons at the desk with near-term solutions to the question of scholarly impact. For this reason, we move on to….

THE 3-MINUTE VERSION OF THE SHORT ANSWER: (I know it’s longer than the “long” answer — but in this case, it’s worth the extra minute.)

When it comes to impact factor research here at AU,  the two best resources for getting started are usually Web of Science’s Cited Reference Search, and Thomas Reuters’ Journal Citation Reports.

Web of Science

Web of Science Homepage (AU Authentication Required)

Web of Science is a massive citation index that includes over 12,000 journals and 148,000 proceedings, often thought of as the “default” database for science research. Offered as part of Thomas Reuters’ larger “Web of Knowledge” platform, it is also home to an essential multidisciplinary tool for citation analysis called “Cited Reference Search,” which users can access using a link just above the Web of Science logo (see red arrow in image above).

Click the image to watch a demo of Cited Reference Search, or here.

Cited Reference Search is an excellent tool for impact research, in that it takes advantage of journal citations from three key indices: the Science Citation Index, Social Sciences Citation Index, and Arts & Humanities Citation Index. In this way, users can search for references to specific authors or publications in range of disciplinary journals. It’s fast and relatively easy — a great choice if you want to know more about how many times one of your articles has been cited by other academic publications.

Journal Citation Reports

Journal Citation Reports is the other key tool for impact research — and, as an offering of the same parent company as Web of Science, it can even be accessed through the same database link. For example, to get there from Web of Science, click on the “Additional Resources” tab, to the right of the Web of Science tab, and select “Journal Citation Reports” from the resulting list of analytical tools.

Click the image to watch a demo of Journal Citation Reports, or here.

According to the Thomas Reuthers website, the value of JCR is its ability to offer a “systematic, objective means to critically evaluate the world’s leading journals, with quantifiable, statistical information based on citation data.” In plain speech, this means that JCR offers individual journal statistics by year in all of the following key impact categories:

  • Journal Impact Factor: A number calculated by dividing the # of “Cites to recent items” (i.e. cites of items published by a journal within the last 2 years) by the # of “Recent items” (i.e. the total number of items published by a journal within the last two years).
  • Five-Year Journal Impact Factor: A number calculated by dividing the # of cites to journal items published in the last five years (e.g. for 2010, items published between 2005 and 2009) by the # of items published by the journal in the last five years.
  • Journal Self-Cites: in a given year, identifies how many times authors published in a specific journal cited articles published in the same journal (not necessarily self-author cites). This section also calculates what the journal’s impact factor would the year would be if Web of Science did not take self-cites into account.

In addition to these core statistics, JCR includes interesting stats on “Journal Cited Half-Life,” “Journal Citing Half-Life,” and “Journal Source Data” (e.g. the breakdown of a journal’s cite-able items in articles, reviews, and more). These additional statistics are sometimes helpful to librarians and faculty in the early stages of article submission, as they collectively indicate practical information about the longevity and currency of  information published in each journal.

***

In the end, the key thing to know about researching impact is that statistics only tell part of the picture. For this reason, traditional bibliometrics will inevitably need to be augmented by qualitative information — or even other, less traditional types of quantitative information.

The other thing to remember is that when it comes to research, the Library is here to help, at the very least with its resources, and very possibly with the attention of its librarians and staff. The same goes with impact factor research — and SOC’ers, I look forward to helping you however and whenever I can.

Happy Fall semester, everyone.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s