Small, single-purpose checks built on the same engine as the full scanner. Each answers one concrete question about how AI crawlers see your site: what text they can read, which of them you allow, and how to control them. No signup, no catch, and no JavaScript magic. Just what the crawlers actually receive.
Fetch any page the way GPTBot does (no JavaScript) and see the exact text an AI crawler can read, before you assume it can read anything.
Paste a robots.txt or scan a domain, and see which of the 15 named AI crawlers each rule actually allows or blocks, plus any syntax problems.
Every major AI crawler in one table: vendor, purpose, the user-agent string it sends, docs, and the exact robots.txt line to allow or block it.
When someone asks ChatGPT, Claude, or Perplexity about your market, the assistant does not open your site in a browser. It relies on what its crawler fetched: your raw HTML, usually without running any JavaScript. If your content only appears after the page renders, the crawler sees an empty shell, and the AI answer is built without you in it.
This is a different problem from classic search. Google renders JavaScript and has spent years learning your site; most AI crawlers do neither. A page can rank on Google and still be invisible to AI. These tools measure that AI layer directly, one question at a time, so you can see exactly where you stand before you decide what to fix.
Reach for this before you assume any AI can describe your product. It is the fastest way to catch the single most common AI-visibility failure: content that only appears after JavaScript runs.
What you learn: The exact text in your initial HTML, the way a non-rendering crawler receives it, plus the framework you are on and whether the page came back readable, thin, or blank.
Open What GPTBot sees →Reach for this after you edit robots.txt, or whenever you are not certain you are letting in the crawlers you actually want. Google retired its own tester, and none of the survivors were built for AI bots.
What you learn: For all 15 named AI crawlers, whether your file allows or blocks each one at the site root, the exact rule that decides, and any syntax that would trip a real parser.
Open robots.txt tester →Reach for this when you need to allow or block a specific AI bot and want the correct product token, not a guess. It is also the reference to cite when someone asks which crawlers exist.
What you learn: Every major AI crawler in one table: who runs it, whether it is for training or live AI search, the user-agent it sends, official docs, and the copy-paste robots.txt lines to control it.
Open AI crawler directory →Each tool requests your page or robots.txt once, with the crawler's own user agent, and never runs JavaScript. That is the whole point: you see what the bot receives, not what a browser paints.
The extraction, robots parsing, and crawler list come from the same code behind the graded scanner. The tools are focused slices of it, not a separate, lesser check. The methodology covers the scoring side.
We fetch from our own servers, so a site that verifies crawlers by IP address can behave differently for the real bot. Every result says so plainly. Evidence, not guarantees.
AI visibility is whether AI systems like ChatGPT, Claude, and Perplexity can actually read your pages and use them in their answers. It is separate from Google ranking: a page can rank well in classic search and still be invisible to AI, usually because AI crawlers receive different HTML than a browser does.
Most do not. Crawlers like GPTBot and ClaudeBot fetch your raw HTML and, unlike Google, generally do not run JavaScript. If your important content is injected by JavaScript after the page loads, those crawlers get an empty shell, and so does any AI answer about you. That gap is what the What GPTBot sees tool exposes.
No. robots.txt only decides whether a crawler is permitted to fetch the page. Even a crawler you welcome gets nothing useful if the page needs JavaScript to render, has no structured content, or blocks it at the server or CDN layer. Access is the doorway; readability is the room.
Classic SEO tools check Google, which renders JavaScript and has its own rules. These tools check the AI layer specifically: how much of your content survives in the raw HTML that non-rendering AI crawlers receive, and which named AI bots your rules allow. It is the same idea people call generative engine optimization, or GEO.
Yes, free, and no signup. Each tool runs one bounded check and shows you the result. There is nothing to install and no credit card. The paid part is optional: if a tool surfaces a problem, we can implement the fix for you and re-scan to prove the gain.
The 15 named crawlers we track across OpenAI (ChatGPT), Anthropic (Claude), Perplexity, Google, Apple, Meta, ByteDance, Amazon, and Common Crawl. The AI crawler directory lists every one with its user agent and how to control it.
No tool can promise that, and we will not pretend otherwise. What these do is remove the reasons AI cannot read or reach you in the first place. Being readable and reachable is the prerequisite for being cited; it is not a ranking trick.
The full scan grades your page across every named AI crawler, diffs the crawler view against a real browser render, and ranks the fixes that move the score. Free, and under a minute.