Free tools / What GPTBot sees
What does GPTBot see on your page?
AI crawlers fetch your HTML but do not run JavaScript. Paste a URL and see the exact readable text they get.
One fetch.
The whole document.
Five lines of an example page, shown as a browser assembles them and as a non-rendering crawler receives them. Three of the five are written into the document by script after load, so the crawler never sees them.
Same-day courier for independent shops
Flat-rate local delivery across the city, booked in under a minute.
Live parcel tracking on every job, shared with your customer.
Returns pickup from your counter, next working day.
Simple per-parcel pricing, and no monthly fee.
All five lines are on the screen a visitor sees, so the page looks complete to everyone who checks it in a browser.
Same-day courier for independent shops
Flat-rate local delivery across the city, booked in under a minute.
empty div
Live parcel tracking on every job, shared with your customer.
empty div
Returns pickup from your counter, next working day.
empty div
Simple per-parcel pricing, and no monthly fee.
Two lines survive the fetch. The other three are not in the response at all, so nothing an AI says about this page can be built from them.
An example page, written for this explanation rather than measured from a real site. Run the check above to see your own lines.
What it cannot tell you
- How much larger the rendered page is. Prose Parity needs both readings and this check makes only the crawler one, so it can show that the crawler view is thin without putting a percentage on the gap. The methodology covers the graded version.
- How your server answers OpenAI's own addresses. Every fetch here identifies as GPTBot but leaves from our network, so a site that verifies crawlers by IP can answer the real crawler differently.
- What your robots.txt says for the other named AI crawlers. That is a different question, and the robots.txt tester answers it for all 15 tokens.
- Whether the words are useful, distinctive, or worth quoting. The verdict is a word count against one floor, not a judgement of the writing.
Fetch once.
Extract. Show.
Three steps, and the third is the point: the document itself, with a word count read against one floor rather than a score to argue with.
-
Fetch as the crawler
One request with no browser and no JavaScript execution, sent under GPTBot's published user agent, the same string printed further down this page.
-
Extract the main content
The main-content reading of the raw HTML, the same one Prose Parity starts from. Navigation, header, footer and aside are left out as boilerplate, with a fallback to all visible text when no main region exists.
-
Show you the text
The extraction as plain text, with the word count, so you can read exactly what an assistant would be summarising.
Three readings from one request
The extraction, the verdict floor and the robots parser are the same code a full graded scan runs. This is a slice of it, not a lesser check.
The readable text, and how much of it
The main-content extraction from your raw HTML, falling back to all visible text when no main region is found, printed up to 4,000 characters. The word count is taken on the full extraction before that cap, so a truncated box never deflates the number.
The verdict, against one floor
That word count against a single threshold: blank at zero words, thin below 120, readable at 120 and above. Alongside it, whether a title element was found in the raw HTML and the HTTP status your server returned.
The robots.txt answer for GPTBot
Your robots.txt fetched from the same origin and the GPTBot rules evaluated against the path you submitted. When the file cannot be read the answer is left blank rather than guessed, and a recognisable platform fingerprint is reported when one is present.
The exact request string, read from the same registry the fetcher uses:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.4; +https://openai.com/gptbot
If the crawler view came back thin
Thin or blank almost always means the words are written into the document by script rather than served in the response. The fix is specific to how your site is built.
- Find out which stack you are on. The result names a platform fingerprint when it recognises one, and Fix AI visibility by stack is the index of every guide below.
- On Next.js, move the content out of client components. Why AI crawlers can't read your Next.js site covers server rendering, static generation and the useEffect pattern that empties a page.
- On a React single-page app, add a server-rendered shell. Why AI crawlers can't read your React site sets out the options in order of effort.
- On Framer, check what publishes into the served HTML. Why AI crawlers can't read your Framer site covers page settings, headings and the schema that has to be added by hand.
- On Shopify, look at the apps injecting content. Why AI crawlers can't read your Shopify site covers theme sections against app-rendered blocks.
- On Lovable, confirm the export is server rendered. Why AI crawlers can't read your Lovable site covers what the builder emits and what it does not.
- Re-run this check. It costs nothing, and the word count moving from thin to readable is the proof the fix landed.
Three people this is built for
No account, no email, and no grade to argue with. It is the fastest way to end a disagreement about whether content is on the page.
Writers and editors
You want to see whether the copy you published is the copy a machine receives, before anybody starts optimising it.
Engineers
You are debugging a render path and need the crawler's view of the response, without setting up a headless fetch yourself.
Anyone in a meeting
Somebody says the content is there. Open the page, read the extraction, and the question is settled in ten seconds.
Most people have never seen their own page the way a crawler receives it. The browser fills in everything the crawler does not get: fonts, images, and the content your JavaScript fetches after load. Strip that away and what is left is the document an AI assistant is actually working from.
If your pricing, your proof, or your product description is missing from that text, no amount of prompt engineering will put it back.
Common questions
Does this run JavaScript, the way my browser does?
No. The check makes one plain HTTP request for the URL, reads the HTML that comes back, and extracts the readable text from it. There is no browser and no rendering step, which is the position a non-rendering crawler is in. Text that a script writes into the page after load is absent from this view by construction, not by accident.
Is this exactly what the real GPTBot receives?
The user agent is GPTBot's published request string and the fetch runs the same way, without JavaScript. What differs is the source address: our request leaves from our own network, not OpenAI's. A server that verifies crawlers by IP address, or that treats our network differently, can answer the real GPTBot differently from this check. Read the result as evidence about your HTML, not as proof about your firewall.
Does the check look at robots.txt?
Yes. It fetches robots.txt from the same origin and evaluates the GPTBot rules against the path you submitted. GPTBot obeys robots.txt, so a disallow means the real crawler fetches nothing at that URL whatever the HTML holds, and the result says so while still showing the text GPTBot would get if it were allowed. A missing robots.txt counts as allowed. If robots.txt cannot be read at all, the result leaves the answer blank rather than claiming a block it did not see. Rules for the other named AI crawlers are a different question, and the robots.txt tester answers it.
What makes a page blank, thin, or readable?
One threshold, applied to the word count of the extracted text: no words at all is blank, fewer than 120 words is thin, and 120 words or more is readable. It is the same floor the rest of the scanner uses. Readable means there is text for an AI system to work with. It does not mean the page is well structured, allowed to every crawler, or likely to be cited.
Does this measure prose parity?
No. Prose parity compares the crawler fetch with a real browser render of the same page, and this tool makes only the crawler fetch. It can show you that the crawler view is thin; it cannot tell you how much larger the rendered page is, because it never renders one. The graded scan runs both fetches and reports the gap as a percentage.
Does the answer hold for AI crawlers other than GPTBot?
For the reading half, largely yes: any AI crawler that does not run JavaScript receives the same initial HTML, so the readable text here is a fair proxy for what they get. The access half is not shared. Each crawler has its own robots.txt rules and its own user agent, and a server can allow one while blocking another, so a per-crawler answer needs a per-crawler check.
Why does the extraction leave out my navigation?
Because navigation, header, footer and aside are boilerplate, and counting them would let a big menu inflate a thin page. When the page has a recognisable main region, the text here is the same main-content reading the Prose Parity metric starts from. When it does not, the check falls back to all visible text rather than showing nothing, which is the one case where menu text can appear.
The text looks complete. Am I fine?
For this page, on this fetch, yes. The full scan is what tells you whether the rendered version contains materially more than this, and whether the crawlers you care about are allowed to request it in the first place.
Is this the same as the full scan?
No. This fetches once and shows you the text. The full scan also renders the page in a real browser, diffs the two, checks access for all 15 named AI crawlers, and grades the result with ranked fixes. Both are free.
Read your own page as a machine does.
It takes ten seconds and it is usually the moment the problem becomes obvious. The graded scan is the next step once you have seen it.