stripe.com

Can AI crawlers find your content and understand your policies?

• ✓
  robots.txt present

  HTTP 200
• !
  sitemap.xml present

  Declared in robots.txt but /sitemap.xml not found
• ✓
  Sitemap declared in robots.txt
• !
  AI bot rules in robots.txt

  No explicit rules for GPTBot / ClaudeBot / Google-Extended
• !
  Content Signals declared

  No Cloudflare Content-Signal directive

Standard well-known endpoints agents probe to interact with your site.

• ✗
  MCP server card (/.well-known/mcp/server-card.json)

  Publish an MCP server card to expose agent tools
• ✗
  API catalog (/.well-known/api-catalog)

  RFC 9727 API catalog helps agents discover your APIs
• ✗
  OAuth protected resource (/.well-known/oauth-protected-resource)

  RFC 9728 lets agents discover auth requirements
• ✗
  OpenID configuration (/.well-known/openid-configuration)

  Helpful if your site offers identity / sign-in for agents

How well your site speaks formats AI agents prefer.

• ✓
  llms.txt present

  HTTP 200
• !
  Markdown content negotiation

  Returned text/html; charset=utf-8 despite Accept: text/markdown
• !
  Link header on /

  No Link header — agents can't follow alt-format relations

HTML signals that Google AI Search + LLM crawlers use to rank you.

• ✓
  Homepage reachable

  HTTP 200
• ✓
  HTTPS
• ✓
  Title tag

  Stripe | Financial Infrastructure to Grow Your Revenue
• ✓
  Meta description

  Stripe is a financial services platform that helps all types of businesses accept payments…
• ✓
  Canonical URL
• ✓
  Open Graph tags

  title, description, image
• ✗
  Schema.org structured data

  Add JSON-LD (Organization, Product, FAQPage…)
• ✓
  Mobile viewport
• ✓
  html lang attribute
• ✗
  Image alt text

  4/34 images have alt

Optional: cryptographic verification for trusted bots.

• ✗
  Web Bot Auth directory

  Publish /.well-known/http-message-signatures-directory to verify trusted bots