AI crawlers ยท Microsoft

Bingbot

Used for: Bing search.

What it does

Microsoft says it's "our standard crawler and handles most of our crawling needs each day."

What blocking it changes

Microsoft's page says robots.txt can be configured to tell Bing's crawlers how to interact with your site, but doesn't spell out the effects of blocking it.

1%

of the 228 sites we've audited with a readable robots.txt block it site-wide.

How to block it

User-agent: Bingbot
Disallow: /

How to allow it

User-agent: Bingbot
Allow: /

Good to know

Bing Webmaster Tools also lets you control crawl rates by the hour.

Sources

What does your robots.txt say?

See how your site treats Bingbot and the other major crawlers.

Check my site

Other crawlers