We would be lying if we said that our journal's impending first impact factor is not of interest to us....However, for a number that is so widely used and abused, it is surprising how few people understand how a journal's impact factor is calculated, and, more importantly, just how limited it is a means of assessing the true impact of an individual publication in that journal....Because a journal's impact factor is derived from citations to all articles in a journal, this number cannot tell us anything about the quality of any specific research article in that journal, nor of the quality of the work of any specific author....Moreover, a journal's impact factor says nothing at all about how well read and discussed the journal is outside the core scientific community or whether it influences health policy. For a journal such as PLoS Medicine, which strives to make our open-access content reach the widest possible audience --such as patients, health policy makers, non-governmental organizations, and school teachers-- impact factor is a poor measure of overall impact....
After one in-person meeting, a telephone conversation, and a flurry of e-mail exchanges, we came to realize that Thomson Scientific has no explicit process for deciding which articles other than original research articles it deems as citable. We conclude that science is currently rated by a process that is itself unscientific, subjective, and secretive.
Even more importantly, it is time to reconsider the whole process of accurately assessing an individual paper's worth not only to scientists, but also to the wider community of readers. First, although any measure of impact will remain flawed in some way, when assessing the impact of individual articles or of the papers of individuals or groups of scientists, it surely makes more sense to measure the citations specifically to those individual articles (or to papers by individuals or groups of scientists) rather than using a journal's impact factor as a proxy measure. However, it is not clear whether Thomson Scientific could measure such individual article citations accurately. Second, we urge the company to take its responsibility seriously and increase transparency and accountability. Third, we suggest that the company's staff engage in the ongoing debate among other shareholders of scientific publishing and recognize that, there are --finally-- other ways of measuring impact and visibility of scholarly articles....
The opening up of the literature means that better ways of assessing papers and journals are coming—and we should embrace them.
Posted by
Peter Suber at 6/13/2006 10:29:00 AM.
The open access movement:
Putting peer-reviewed scientific and scholarly literature
on the internet. Making it available free of charge and
free of most copyright and licensing restrictions.
Removing the barriers to serious research.
I recommend the OA tracking project (OATP) as the best way to stay on top of new OA developments. You can read the OATP feed on a blog-like web page or subscribe to it by RSS, email, or Twitter. You can also help build the feed by tagging new developments you encounter.
