Start A Blog
Formula 10
Categories
Contact Me
Archives

Ask a Question

Robots File Tells Search Engines Where Sitemap Is Located

Sitemap

Did you know there is a tag you can place in your robots.txt file that will tell search engines where your sitemap file is located. Here is an example. I placed this line in my own robots.txt file. It doesn't matter which line you place it on, just make it its own line.

Sitemap: http://www.buzzybeemarketing.com/sitemap.php

Robot File

All the major search engines will follow this syntax. You can also define the location of your sitemap for Google in Google's Webmaster Tools. It's important to tell other search engine like MSN and Yahoo where your sitemap is. Don't just concentrate on Google. MSN actually offers a higher conversion rate on sales. My guess is it has to do with msn being set as the default browser on new computers.

There are other nifty features you can utilize with robotx.txt You can block certain areas from being spidered by the search engines using this tag:

Disallow: /admin/

It would be a waste of the spider's time to crawl internal pages.

Written By Drew Sire

Tags