Learn How It Works!
Learn how to navigate SEOENG

SEOENG® Learning Center...

Robots.txt

This file is located at the root of a Website, and it is named "robots.txt". It indicates directives to Robots, which are automated computer programs used by Search Engines to "crawl" the Internet and store information about each Webpage on the Internet. The directives in this file can tell a Robot to not index certain Webpages, which might be duplicate or useless Webpages to a Search Engine. Some common Robots are GoogleBot™ (Google), Slurp™ (Yahoo), BingBot™ (Microsoft), and SEOENGBot™ (SEOENG LLC.). SEOENGBot™ follows the robot standard.

Robots.txt

Allow SEOENGBot™ to Crawl your Website!

SEOENGBot™ is a very responsible Spider-Bot. It follows all robots.txt directives, and ensures that the Website it is crawling is not harmed by excessive bandwidth requests. To ensure that SEOENGBot™ can analyze your Website, place the following code into your robots.txt file.

# SEOENGBot knows where to go.
User-agent: seoengbot
Disallow:

Ensure that SEOENGBot™ can crawl your Website by adding this to robots.txt

Current Limitation:

New directives as noted here: http://www.ysearchblog.com/archives/000508.html, are not yet supported, but are planned to be integrated into SEOENGBot™ in the very near future. Please see a list of features of our Search Engine to keep updated on when this is added.

Additional Information: