Open Access News

Many journals combine open access and open review, but some are finding that open review is not working as well as they wanted. 

Nikolaus Kriegeskorte is thinking about how to make open review work better.  See his full-length elaboration of the idea, Open post-publication peer review, his abridged version, and his FAQ.  Excerpt from the full-length elaboration:

...Beyond open access, which is generally considered desirable, the essential drawbacks of the current system of scientific publishing are all connected to the particular way that peer review is used to evaluate papers. In particular, the current system suffers from a lack of quality and transparency of the peer review process, a lack of availability of evaluative information about papers to the public, and excessive costs incurred by a system, in which private publishers are the administrators of peer review. These problems can all be addressed by open post-publication peer review.

Open: Any scientist can instantly publish a peer review on any published paper. The scientist will submit the review to a public repository. Reviews can include written text, figures, and numerical quality ratings. The repository will link each paper to all its reviews, such that that readers are automatically presented with the evaluative meta-information. In addition, the repository allows anyone to rank papers according to a personal objective function computed on the basis of the public reviews and their numerical quality ratings. Peer review is open in both directions: (1) Any scientist can freely submit a review on any paper. (2) Anyone can freely access any review.

Post-publication: Reviews are submitted after publication, because the paper needs to be publicly accessible in order for any scientist to be able to review it. Post-publication reviews can add evaluative information to papers published in the current system (which have already been secretly reviewed before publication). For example, a highly controversial paper appearing in Science may motivate a number of supportive and critical post-publication reviews. The overall evaluation from these public reviews will affect the attention given to the paper by potential readers. The actual text of the reviews may help readers understand and judge the details of the paper....

Note...that public post-publication reviews differ in two crucial respects:

(1) They do not decide about publication – as the papers reviewed are already published.

(2) They are public communications to the community at large, not secret communications to editors and authors.

This makes the peer reviews the equivalent of getting up to comment on a talk presented at a conference. Because these reviews do not decide about publication, they are less affected by politics. Because they are communications to the community, their power depends on how compelling their arguments are to the community....

Signed or anonymous: The open peer reviews can be signed or anonymous. In analyzing the review information to rank papers, signed reviews can be given greater weight if there is evidence that they are more reliable.

Digitally authenticated: Reviewers can digitally sign their reviews by public-key cryptography. The idea of digitally signed public reviews has been developed [at GPeerReview]....

Paper selection by arbitrary evaluation functions: The necessary selection of papers for reading can be based on the reviews and their associated numerical judgments. Any reader can define a paper selection function based on content and quality criteria and will automatically be informed about papers best conforming to his or her criteria. The evaluative function could for example, exclude anonymous reviews, exclude certain reviewers, weight evidence for central claims over potential impact of the results etc.

Webportals as entry points to literature: Webportals can define such evaluation functions for subcommunities – for scientists too busy (or too lazy) to define their own. Such webportals would provide a generalized access to the literature that transcends all journals. A webportal can be established cheaply by individuals or larger organizations that share a common set of criteria for paper prioritization.

Comments

• I've often maintained that achieving OA and reforming peer review are independent projects, and that OA is compatible with every kind of peer review, from the most conservative to the most innovative.  I still believe that.  But I want to see open review work.  It's the most promising way to shift peer review from a pre-publication task to a post-publication task, removing both a cause of delay and a financial incentive for publishers to resist OA.  It's also the most promising way to create a useful plurality of evaluations of the primary literature.  As I argued in 2002:

  The beauty of second-order tools using first-order scholarship as data [basically, retroactive peer-review judgments about research articles] is that there can never be too many of them.  If proliferating first-order judgments creates information overload, then proliferating second-order judgments creates competition, and this competition will be beneficial for users and self-limiting.  Second-order judgments are valuable even when they conflict, because different users have different needs, interests, projects, standards, and approaches.  You should have a choice among services competing to help you decide what deserves your time and attention.  Of those services that know what you want, some will be faster, cheaper, or friendlier in providing it.  Of those that are fast, cheap, and friendly, some will know better what you want.  If putting priced paper literature online free of charge accelerates research, then a robust market of sophisticated, competing second-order tools will accelerate it again.

• If I have a new thought on how to make open review work, it's this:  editors should solicit and assign retroactive reviews as they now solicit and assign prospective reviews.  Scholars should be just as willing to write retroactive reviews as prospective reviews, and their work will give editors and readers the help they do not always receive from spontaneous user comments.  Retroactive reviews need not supplant spontaneous user comments and can supplement them.  They can be anonymous or attributed, according to local practice.  By referring to "editors" I don't mean to presuppose the continuing existence of "journals" which yoke together certification and distribution.  Editors and editorial boards could be free-floating, limit themselves to certification, and leave distribution to the network of OA repositories.