ArielisBot is Arielis' web crawling bot, or spider, for discovery of new and updated information for search indexing.
The best way to identify accesses by ArielisBot is to use the user-agent: ArielisBot.
How to include your content:
ArielisBot's crawl process begins with directory listings, and is augmented to include trending content revealed by links provided by webmasters.
Qualifying sites may be submitted for review here
. After approval, ArielisBot will crawl your site and index your content for search.
How to exclude your content:
ArielisBot obeys The Robots Exclusion Protocol (robots.txt).
To prevent ArielisBot from crawling your entire site:
To prevent ArielisBot from crawling part of your site:
How to remove content from Arielis:
Blocking ArielisBot by robots.txt will prevent new content from being indexed. Should you need previously indexed content removed, Please let us know.