Robots.txt File Generator

Robots.txt is a special text file on your website that instruct robots whether or not to crawl and index certain pages and directories within your site.

The generated code must be placed inside a text file called robots.txt that is located at the root level of your website. For example, the robots.txt file for google.com is located at http://www.google.com/robots.txt

Select User Agent:

Directory or Pages to disallow:

For example:
1. images
2. contact/secret.htm

You can add multiple user agents and directories by clicking on this button multiple times.
Share and Enjoy:
  • Digg
  • del.icio.us
  • Facebook
  • Google Bookmarks
  • email
  • Netvibes
  • Twitter
  • Yahoo! Buzz

Hosting

Looking for a hosting provider for your website? Check out hostgator, our hosting provider of choice. Unlimited Domains, Disk Space and Bandwidth for just $7.95 per month.

Special offer: Get started from just 1 cent! Enter wordpress as the coupon code.

Hostgator

  • Unlimited Disk Space and Bandwidth
  • Flexible, Easy to Use Control Panel
  • Unlimited Sub Domains, FTP Accounts, and Email Accounts
  • 99.9% Uptime Guarantee
  • No Contract with a 45 Day Money Back Guarantee