What we check
We look for a Content-Signal: line in your robots.txt.
of 198 sites we could read pass this.
Why it matters
Content Signals is Cloudflare's mechanism for stating how automated systems should use your content. It has three categories: search, ai-input and ai-train.
These are preferences, not enforcement. The site says "not all automated systems honor robots.txt files, and some may ignore these Content-Signal directives."
search covers search indexes and excerpts, and doesn't include AI-generated summaries. Those fall under ai-input.
How to fix it
- Decide your preference for each category.
- Add the line inside a user-agent group in robots.txt.
User-Agent: *
Content-Signal: ai-train=no, search=yes, ai-input=yes
Allow: /
Fix it yourself with AI
Small fixes shouldn't sit in a queue. Give your AI agent one prompt, and it fixes and deploys for you.
PromptShip is made by the team behind this checker.
Sources
Fixed it?
Run your site through the checker, and re-check after you ship to see what improved.
Check my site