Officials from the leading Internet search engine are working to remove barriers that prevent their technology from reaching vast troves of information buried in government databases....[W]hile portions of agency Web sites are easily indexed by Google and other common search engines, the engines cannot search other areas, known as the deep Web....
As much as 40 percent of the content on agency Web sites is invisible to Google's crawlers, said [J.L. Needham, a strategic partner development manager at Google]....
Needham said he is meeting with a variety of agencies to discuss how the information housed in their databases can be made available in the search results from engines such as Google, Yahoo or MSN....
A Dec. 16, 2005, memorandum from Clay Johnson, deputy director for management at the Office of Management and Budget, states that all agencies must set up their public information so that it is searchable by Sept. 1, 2006. It states that "increasingly sophisticated Internet search functions" can "greatly assist agencies in this area."
Agencies also were required to provide all public data in an open format that allows the public to aggregate "or otherwise manipulate and analyze the data to meet their needs" by Dec. 31, 2005, according to a separate OMB memorandum signed by Johnson on Dec. 17, 2004....
In addition to the technical challenges presented by the company's request, EPA has to consider whether a commercial company could assert proprietary ownership on federal data or whether providing government data to one company would provide an unequal playing field for other companies, Luttner said.
Needham said Google, for one, does not want to assert ownership over any information obtained from agencies, and agency efforts to improve the ability to search their Web sites would likely be equally beneficial to its competitors....
Posted by
Peter Suber at 10/26/2006 11:00:00 AM.
The open access movement:
Putting peer-reviewed scientific and scholarly literature
on the internet. Making it available free of charge and
free of most copyright and licensing restrictions.
Removing the barriers to serious research.
I recommend the OA tracking project (OATP) as the best way to stay on top of new OA developments. You can read the OATP feed on a blog-like web page or subscribe to it by RSS, email, or Twitter. You can also help build the feed by tagging new developments you encounter.
