Microsoft and Yahoo! back Google website-indexing standard

Three search leaders support XML-based format for search engine crawlers

Simon Aughton
16 Nov 2006

Yahoo! and Microsoft have given their backing to a Google technology designed to provide webmasters with an easy way to notify search engines about their websites.

The two companies, Google's main rivals in the search business, have given their support to Sitemaps 0.9 and will collaborate on future enhancements to the protocol

A Sitemap is an XML file that can be made available on a website and acts as a marker for search engines to crawl certain pages. It lets webmasters list all of their URLs along with optional metadata, such as the last time the page changed, to improve how search engines crawl and index their websites. With all three major search operators supporting the technology, webmasters now need only to submit their content in one format.

As well as providing easier indexing of static Web pages, Sitemaps also supports dynamic sites where content management systems (CMS) are deployed. By placing a Sitemap file on Web servers, webmasters can automatically tell crawlers what pages are present and which have recently changed.

Narayanan Shivakumar, a distinguished entrepreneur with Google said that Sitemaps has already improved the quality of its search results.

'Announcing industry supported Sitemaps is an important milestone for all of us because it will help webmasters and search engines get the most relevant information to users faster,' he said. 'Sitemaps address the challenges of a growing and dynamic Web by letting webmasters and search engines talk to each other, enabling a better Web crawl and better results.'

Danny Sullivan, editor-in-chief of Search Engine Watch said that this kind of open standard is exactly what webmasters have been asking for.

'This is a great development for the whole community and addresses a real need of webmasters in a very convenient fashion,' he said. 'I believe it will lead to greater collaboration in the industry for common standards, including those based around robots.txt, a file that gives Web crawlers direction when they visit a website.'

For more information on Sitemaps go to

Google first announced Google Sitemaps in June 2005.

Read more about: