What it does
OpenAI says it's "used to surface websites in search results in ChatGPT's search features."
What blocking it changes
OpenAI says sites opted out "will not be shown in ChatGPT search answers," though they "can still appear as navigational links."
7%
of the 228 sites we've audited with a readable robots.txt block it site-wide.
How to block it
User-agent: OAI-SearchBot
Disallow: /
How to allow it
User-agent: OAI-SearchBot
Allow: /
Good to know
OpenAI says robots.txt changes can take about 24 hours to take effect for search.
Sources
What does your robots.txt say?
See how your site treats OAI-SearchBot and the other major crawlers.
Check my site