Bots

Bots

Get Started

"Bots" section

Bots, robots, spiders or web crawlers are special programs acting as a normal user. They usually scan your website and discover new content or try to index that content.

For example, a "GoogleBot" is responsible for crawling your website to be index it and present in the Google search results. There are other bots such as Yahoo, or Yandex, Majestic bot etc.

This view gives you a good overview of what pages are these bots and spiders actually interested in.

Get Started

Previous Post Next Post