Server-level blocking of AI crawler user agents
ua_enforcement
+2.5
AI systems are stopped at the door, so none of your content is available for them to summarize or recommend.
Your security or hosting layer blocks AI crawlers before they can reach your pages.
Requests with AI crawler user agents get blocked at the edge. Review CDN/WAF bot rules if you want AI visibility.
Evidence from your pageGPTBot got HTTP 403; OAI-SearchBot got HTTP 403; ChatGPT-User got HTTP 403; ClaudeBot got HTTP 403; Claude-SearchBot got HTTP 403; Claude-User got HTTP 403; PerplexityBot got HTTP 403; Perplexity-User got HTTP 403; Meta-ExternalAgent got HTTP 403; Bytespider got HTTP 403; CCBot got HTTP 403; Amazonbot got HTTP 403. Servers that verify crawler IPs will block a spoofed user agent from any scanner; the real crawler may pass.
Review your crawler access policy. We map which AI systems you are blocking, deliberately or by accident, and align robots.txt, CDN rules, and directives with the visibility you actually want.
Get an access review