A robots.txt file is a special text file that is located in your Web server’s root directory /robots.txt. You need a robots.txt file only if your site includes content that you don’t want search engines to index. If you want search engines to index everything in your site, you don’t need a robots.txt file. Robots.txt [...]
Posts Tagged ‘disallow’
What is a robots.txt file?
Posted in SEO, tagged disallow, robots on September 12, 2009 | Leave a Comment »
