Open Access News

News from the open access movement


Friday, January 18, 2008

Overcoming obstacles to a citation index for OA articles

The citation extraction process in CitEc, RePEc blog, January 16, 2008.  Excerpt:

CitEc is an experimental autonomous citation index, that is, it is a software system which is able to automatically extract references out of the full texts of documents and create links between citing references and cited papers.

With its last update, the CitEc database has reached almost three million references and more than one million citations between documents available in RePEc. This is an important threshold but still is far of being a complete set of citations. There are some limits in the references extraction process:

First, the system needs to have open access to a electronic version of the documents full text. Many journals listed in RePEc have restricted access....We try to get on board as many publishers as possible but unfortunately not all of them are willing to collaborate with us at this time....

Second, the URL provided by the RePEc archive maintainer must be correct and must point to the PDF file containing the document full text and not to an intermediate abstract page or similar. Some archives provides this kind of links to force the researchers to pass through their institutional web pages....

The third limit is more technical. In order to extract references, the PDFs files need to be converted into plain ASCII text....There are a wide variety of PDF files created in different ways and not all of them can be converted.

Finally, the systems does a parsing of the references section, which first needs to be isolated, to identify each reference and split it in its parts: title, author, year, etc. The parsing is done using pattern matching techniques which in some cases are not able to identify the full list of existing references.

As the last update as of December 31, 2007, the CitEc numbers are: 527,357 articles and working papers available in RePEc. Of them, 343,441 cannot be processed by the system due to limitations mentioned in the first two points above....

That leaves an amount of 183,916 documents available to be processed by CitEc. Of them, the process was successfully completed in 134,130 papers, that is the 73% of the available documents. The complete list of sources and the number of processed documents for each series or journal is available here....