Who is
this bot????
Keyword Hero Bot is one of our crawlers, which we use to collect information about the structure of your website. The data gathered allows us to customize our services to your account and to continuously improve our products.
We use a server network, so IP may change often.
The best way to recognize him is thru the user agent
“Mozilla/5.0 (compatible; keyword-hero bot/1.0; +https://keyword-hero.com/bot)”.
Keyword Hero
Bot on your robots.txt
Directories and URLs
If you want to keep our bot from crawling your site, all you have to do is add his user agent to your robots.txt file.
On the right, you find an example if you only want to exclude the bot from crawling specific directories or URLs of your site:
[crayon attributes]
##Excluding Keyword Hero’s Bot from directories or URLs
#################################
User-agent: keyword-hero bot
Disallow: /admin/
Disallow: /scripts/
Disallow: /images/
[/crayon]
The entire domain
Of course, you can also keep him away from your entire domain. On the right, you find an example to exclude the bot from crawling your whole site:
[crayon attributes]
##Excluding Keyword Hero’s Bot from your entire domain
#################################
User-agent: keyword-hero bot
Disallow: /
[/crayon]