WadooBot

Version: 1.0
Bot Type: Good (Identifies itself, has an official moniker)
Category: Link Data Collection
Obeys Robots.txt: Yes
Obeys Crawl Delay: Yes
User-Agent String: Mozilla/5.0 (compatible; WadooBot/1.0; +https://www.wadoo.net/wadoobot/)

What is WadooBot?

WadooBot is a Web Crawler that powers Wadoo! Search Results. It constantly crawls web to fill our database with new links and check the status of the previously found ones to provide the most comprehensive and up-to-the-minute data to our users.

What is WadooBot doing on your website?

WadooBot is crawling your website making notes of outbound links and adding them to our database. It will periodically re-crawl your website to check the current status of previously found links.

Our crawler does not collect or store any other information about your website. It does not trigger ads on your website (if any) and won’t add numbers to your Google Analytics traffic.

Does it respect robots.txt file?

Yes. Absolutely.

We strictly respect robots.txt, both disallow and allow rules.

How to control WadooBot on your website?

As mentioned above, WadooBot strictly follows the robots.txt file on your website. So you can fully control it on your website if you need.

To change the frequency of WadooBot visiting your site, you can specify the minimum acceptable delay between two consecutive requests from our bot in your robots.txt file:

User-agent: WadooBot
Crawl-Delay: [value]

Where Crawl-Delay value is time in seconds.

If for some reason you want to prevent WadooBot from visiting your site, put the two following lines into the robots.txt file on your server:

User-agent: WadooBot
Disallow: /

Please note that WadooBot may need some time to pick the changes in your robots.txt file. This will be made prior to each next scheduled crawl.

Please also note that if your robots.txt contains errors and WadooBot won’t be able to recognize your commands it will continue crawling your website the way it did before.

Read more about robots.txt at www.robotstxt.org.

If you think that WadooBot is someway misbehaving on your website or if you have any questions about it, please don’t hesitate to contact our Support Team.