News sites seek new controls over search engines

News publishers are poised to unveil new proposals that will give them greater control over the way that search engines index their websites.

Currently, website administrators can create a robots.txt file that tells search engines’ robots to ignore certain pages. Robots are the software tools that search engines such as Google use to index the contents of sites.

But many publishers feel that this does not give them sufficient control and the new proposals seek to add extra stipulations. For example, publishers could tell search engines how long they want pages to be listed, or instruct them to ignore links within pages.

The new system is known as the Automated Content Access Protocol and the various publishers’ bodes behind the proposal believe that it could help to prevent the disputes that have put publishers at loggerheads with search engines, notably in Belgium.

The new commands will be published later today. But since system works on voluntary basis, they will not be recognised until search engines agree to update their robots.

Google said that while it welcomes any initiative that would help it to work more closely with publishers, it would not be making any decision on adopting ACAP until it has been thoroughly evaluated. Not least because it has to work for all sites, not just those producing news content.

“Before you go and take something entirely on board, you need to make sure it works for everyone,” said spokeswoman Jessica Powell.

Gavin O’Reilly, chairman The World Association of Newspapers, which helped to formulate ACAP, said that it has been designed with this in mind, so that it is to be applicable to every type of content published online, including video and audio.

“ACAP is not about good and evil, but about the management and promotion of quality content within the framework of our current and future digital strategies. Ultimately, ACAP will benefit everyone: publishers, search engines and, most of all, consumers, by creating a fair and content-rich online experience for all of us: no standards wars; no proprietary monopolies; open collaboration: win, win for all.”

For more information go to the-acap.org.