Open Access News

Neal S. Young, John P. A. Ioannidis, and Omar Al-Ubaydli, Why Current Publication Practices May Distort Science, PLoS Medicine, October 7, 2008. 

Summary:  The current system of publication in biomedical research provides a distorted view of the reality of scientific data that are generated in the laboratory and clinic. This system can be studied by applying principles from the field of economics. The “winner's curse,” a more general statement of publication bias, suggests that the small proportion of results chosen for publication are unrepresentative of scientists' repeated samplings of the real world. The self-correcting mechanism in science is retarded by the extreme imbalance between the abundance of supply (the output of basic science laboratories and clinical investigations) and the increasingly limited venues for publication (journals with sufficiently high impact). This system would be expected intrinsically to lead to the misallocation of resources. The scarcity of available outlets is artificial, based on the costs of printing in an electronic age and a belief that selectivity is equivalent to quality. Science is subject to great uncertainty: we cannot be confident now which efforts will ultimately yield worthwhile achievements. However, the current system abdicates to a small number of intermediates an authoritative prescience to anticipate a highly unpredictable future. In considering society's expectations and our own goals as scientists, we believe that there is a moral imperative to reconsider how scientific data are judged and disseminated.

From the body of the paper:

...For most published papers, “publication” often just signifies “final registration into oblivion”. Besides print circulation, in theory online journals should be readily visible, especially if open access....

If “the striving for knowledge and the search for truth are still the strongest motives of scientific discovery”, and if “the advance of science depends upon the free competition of thought”, we must ask whether we have created a system for the exchange of scientific ideas that will serve this end.

Update (10/7/08).  Also see Richard Smith's comments on this article.  Excerpt:

...[The authors argue that] the fact that it is so important [to publish in high-prestige journals] is distorting science. And I think that the authors are right....

[The winner's curse] phenomenon operates in science publishing because the elite journals that accept only a fraction of papers submitted to them go for the “best” and are thus likely to be publishing papers that are suffering from the winner's curse — for example, in that they give dramatic results that are a considerable distance from the “true” results. They are exciting outliers....The articles that the high impact journals publish are bound to be atypical....

Most scientists read a few high profile journals — and so are fed a systematically distorted view of the evidence. It's also these journals that are most widely reported in the media and fed to policy makers, so increasing the impact of the distortion....

For me this paper simply adds to the growing evidence and argument that we need radical reform of how we publish science. I foresee rapid publication of studies that include full datasets and the software used to manipulate them without prepublication peer review onto a large open access database that can be searched and mined. Instead of a few studies receiving disproportionate attention we will depend more on the systematic reviews that will be updated rapidly (and perhaps automatically) as new results appear.

Update (10/10/08).  Also see Stevan Harnad's comments.  Excerpt:

There are reasons to be skeptical about the conclusions of this PLoS article. It says that science is compromised by insufficient "high impact" journals to publish in. The truth is that just about everything gets published somewhere among the planet's 25,000 peer reviewed journals, just not all in the top journals, which are, by definition, reserved for the top articles -- and not all articles can be top articles. The triage (peer review) is not perfect, so sometimes an article will appear lower (or higher) in the journal quality hierarchy than it ought to. But now that funders and universities are mandating Open Access, all research, top, middle and low will be accessible to everyone. This will correct any access inequities and it will also help remedy quality misassignment (inasmuch as lower quality journals may have fewer subscribers, and users may be less likely to consult lower quality journals). But it will not change the fact that 80% of citations (and presumably usage) goes to the top 20% of articles, though it may flatten this "skewness of science" (Seglen 1992) somewhat.

Update (10/12/08).  Also see the anonymous story in The Economist about this article.  Excerpt:

...Ioannidis and his colleagues argue that the reputations of the journals are pumped up by an artificial scarcity of the kind that keeps diamonds expensive. And such a scarcity, they suggest, can make it more likely that the leading journals will publish dramatic, but what may ultimately turn out to be incorrect, research....

The researchers are not suggesting fraud, just that the way scientific publishing works makes it more likely that incorrect findings end up in print. They suggest that, as the marginal cost of publishing a lot more material is minimal on the internet, all research that meets a certain quality threshold should be published online....

Update (10/13/08).  Also see John Timmer's comments.

Update (10/13/08). Also see Jake Young's comments:

...This is a ringing endorsement of open access journals -- partially because they can publish more results (including negative ones) and partially because they give greater access to papers about large data sets. Hear, hear for PLoS!