BlogFindings
Text in hydration JSON scored the same as no text at all, both F at 34.4
Two golden fixtures stored in this repository produce an identical composite of 34.4, an identical grade of F, and identical access, structure and schema sub-scores. One of them ships zero words in its initial HTML. The other ships all 485 of them, inside a JSON payload. The only stored number that separates the two is retrievability, and retrievability is worth nothing in the grade.
This repository has two stored fixtures that make the difference concrete, and the reason to write about them is that the scoring treats them as the same page. Both grade F. Both land on a composite of 34.4. Every sub-score that feeds that composite is identical on both. One of them transmits none of its prose to a crawler that does not run JavaScript. The other transmits every word of it. The number that knows the difference exists, is computed on every scan, is printed in the report, and is weighted at zero. This post says how the two numbers are calculated, why one of them was left out of the grade, and where our own way of finding the text stops working.
In short
- The fixtures synthetic-csr-spa and synthetic-hydration-next, both stored in core/fixtures/ in this repository and read on 19 August 2026, each produce a composite of 34.4 and a grade of F, with access 1, structure 0.462 and schema 0.25 identical on both.
- The two differ in exactly one stored score: prose parity is 0 on both, while retrievability is 0 on the client-rendered fixture and 0.980 on the hydration fixture, whose 485 words ship inside a script element carrying type application/json.
- Retrievability is not a key in SCORE_WEIGHTS in core/src/config.ts, which holds parity at 0.5, access at 0.25, structure at 0.15 and schema at 0.1, so retrievability contributes 0 of the 100 composite points and appears only as a defect.
- Lantad's harvester reads text out of an initial HTML response from two places only: any script element whose type is a JSON media type, and four hard-coded hydration global names in core/src/extract.ts. A payload assigned to any other global name is not harvested.
- On the five real sites captured into fixtures on 15 July 2026, astro.build, gymshark.com, developer.mozilla.org, webflow.com and en.wikipedia.org, retrievability equalled prose parity to every decimal place, so this divergence was constructed rather than observed in the wild.
| Stored field | synthetic-csr-spa | synthetic-hydration-next | Feeds the grade |
|---|---|---|---|
| Composite | 34.4 | 34.4 | Yes |
| Grade | F | F | Yes |
| Prose parity | 0 | 0 | Yes, 50 points |
| Retrievability | 0 | 0.980 | No, 0 points |
| Access | 1 | 1 | Yes, 25 points |
| Structure | 0.462 | 0.462 | Yes, 15 points |
| Schema | 0.25 | 0.25 | Yes, 10 points |
| Raw visible tokens | 0 | 0 | Input |
| Rendered main tokens | 485 | 485 | Input |
| First defect raised | csr_blank | hydration_only | No, reported beside |
Two stored fixtures, one grade, and one number that disagrees
The fixtures are called synthetic-csr-spa and synthetic-hydration-next, and both live in core/fixtures/ alongside the captures of real sites this blog has drawn on before. Synthetic means constructed: neither is a scan of anybody's website, and no figure below describes a real company. They exist so that a code path can be pinned by a test rather than argued about.
The first is a single-page application shell. Its stored initial HTML is two lines: a head with a title, and a body holding an empty div and a script tag pointing at a bundle. There is no prose in it at all. The second stores a title, an empty container div, and a script element carrying the attributes id equal to underscore underscore NEXT underscore DATA underscore underscore and type equal to application/json, inside which sits a JSON object whose fields hold the page title and 485 words of body copy.
Run both through the scoring and the reports come out the same. Composite 34.4. Grade F. Access 1, structure 0.462, schema 0.25, prose parity 0. The rendered view of both is identical too, at 485 main-content tokens beginning with the same words. Every structure check answers the same on both: a title is present, a meta description is not, there is not exactly one h1, there is no canonical, heading coverage passes, no block is oversized, there is no llms.txt. The schema report on both records zero blocks in the raw response and an Organization type present only after JavaScript runs.
One stored number is different. Retrievability reads 0 on the shell and 0.980 on the hydration fixture, and the defect each raises differs in step: the shell raises csr_blank, whose evidence line reads that only 0 percent of the rendered main content exists in the initial HTML, while the hydration fixture raises hydration_only, whose evidence line reads that prose parity is 0 percent but retrievability is 98 percent, so the content exists in hydration JSON rather than in readable HTML. Two pages that are structurally identical to the grader, and materially different to anyone trying to work out what went wrong. That is worth knowing before you read a letter grade as a diagnosis, which is a theme this site has been consistent about in why a grade is withheld rather than guessed.
synthetic-csr-spa
- head: title only
- body: one empty div, one script src
- Words of prose in the response: none
- Prose parity 0, retrievability 0
synthetic-hydration-next
- head: title only
- body: one empty div, one JSON script element
- Words of prose in the response: 485, inside JSON
- Prose parity 0, retrievability 0.980
What retrievability measures that prose parity does not
Both numbers come out of the same function, called once each with a different third argument. The function takes a needle and a haystack of word tokens, cuts each into overlapping runs of 8 words, hashes those runs, and reports what fraction of the needle's runs are also in the haystack. The needle in both calls is the same thing: the main content of the page as it appears after a browser has rendered it. The haystack is where they part.
For prose parity the haystack is the visible text of the initial HTTP response and nothing else. Visible text means what the extractor collects while walking the document: the characters between tags, with script, style, noscript, template, svg and iframe subtrees dropped. A word sitting inside a script element is not visible text by that definition, so it contributes nothing.
For retrievability the haystack is that same visible text plus a third input, a list of strings harvested out of JSON found inside the response. Everything else about the call is the same. So retrievability is never lower than parity, and the gap between them is exactly the amount of the rendered page that exists somewhere in the response as JSON string data and nowhere in the document's readable text. On the hydration fixture that gap is the whole page. On the shell it is nothing, because there is no JSON to harvest either.
Naming the two apart matters because they answer different questions and people run them together. Parity answers whether a crawler that does not execute JavaScript comes away with your prose. Retrievability answers whether the words were on the wire at all. A site can pass the second and fail the first, and that is precisely the state where a manual check with a browser's view-source and a text search reassures you while a crawler leaves with a title and a container. The crawler's view of a URL is the way to see this on a page you own, because it shows the response before anything has run. The distinction also sets the boundary for what an AI crawler is being asked to do: fetching bytes is not the same operation as extracting text, and only one of them is cheap.
Flow: Rendered main content (needle) to Prose parity; Initial response visible text (haystack) to Prose parity; Rendered main content (needle) to Retrievability; Initial response visible text (haystack) to Retrievability; Strings harvested from JSON (extra haystack) to Retrievability.
Where the text actually sits when it sits in hydration JSON
The hydration fixture puts its copy inside a script element whose type attribute is application/json. That is a documented and correct thing to do rather than a trick. MDN's reference for the script element, last modified 9 May 2026, states that when the type attribute is a value that is not a JavaScript MIME type, the embedded content is treated as a data block and will not be processed by the browser, and that all of the other attributes are ignored, including src. Its own worked example is a server-generated block of user data carried exactly this way. The markup is doing what the specification says it should. The text simply is not text as far as anything reading the document tree is concerned.
Our extractor finds it because it looks in two specific places, and it is worth being precise about them, because their edges are the edges of the number. First, when a script element's type contains json, the body is parsed as JSON and every string in it of three characters or more is collected, at any depth, deduplicated. That is the branch the hydration fixture takes. Second, when a script element is ordinary JavaScript, a regular expression looks for an assignment to one of four global names: an initial state global, a Nuxt global, an Apollo state global and a preloaded state global, each optionally prefixed with window. If it matches, the code slices out the balanced JSON that follows and harvests the strings from it, falling back to scraping quoted string literals if the slice will not parse.
Four names is a short list, and it is a hard-coded one rather than a list derived from any published survey of frameworks. A payload assigned to a global that is not one of those four is not harvested at all, which means retrievability would read 0 on it and the report would raise csr_blank rather than hydration_only. That is a limit of this scanner rather than a fact about the web, and it belongs in the same post as the finding. The same shape of limit was written up when JavaScript added no new crawl paths across the captured set: what a tool looks for decides what it can find.
There is a second reason the shape of the payload matters, which is size. A serialised catalogue or a full page state can be large, and a large response has its own consequences upstream of readability: Googlebot reads the first two megabytes of a supported file and stops. A page whose prose is one kilobyte of markup and four hundred kilobytes of state is spending its budget on the copy nobody can read.
- Script element with an ld+json type Parsed as JSON-LD; every string of three characters or more is collected at any depth.
- Script element with any other json type The branch the hydration fixture takes. Parsed as JSON and harvested the same way.
- Ordinary script assigning one of four named globals An initial state, Nuxt, Apollo state or preloaded state global, optionally prefixed with window.
- Ordinary script assigning any other global Not matched, so nothing is harvested and retrievability reads the same as prose parity.
- Text inside a script, style, noscript, template, svg or iframe subtree Dropped from visible text, so it never counts toward prose parity by either route.
Why retrievability is worth 0 of the 100 points
The composite is a weighted sum of four things and retrievability is not one of them. SCORE_WEIGHTS in core/src/config.ts holds parity at 50 points, access at 25, structure at 15 and schema at 10. Those are decisions somebody made rather than measurements anybody took, and this blog has said so every time it has quoted them. Retrievability appears nowhere in that object, so it contributes nothing, and the two fixtures land on the same number for the arithmetic reason that the only input on which they differ is not an input.
The case for leaving it out is the one the whole product rests on. Bytes arriving is not the same as prose being read. If retrievability carried weight, a page could buy back points by shipping its copy in a state blob, which is the practice the tool exists to flag. Scoring it would reward the thing being diagnosed. So it is computed, printed, and used to pick which defect to raise, and it stays out of the grade. That is the same treatment Entity Confidence gets: measured on the same fetch, reported beside the score, weighted at nothing.
Where retrievability does have teeth is in the defect logic, and there it uses two thresholds also stored in config: a parity floor of 0.3 and a retrievability ceiling of 0.8. Parity under the floor while retrievability is over the ceiling is what the code calls the hydration pattern, and it is the only condition that raises hydration_only. Below the floor without that combination raises csr_blank instead. Under 0.9 raises a partial-parity defect. Those cut points are settings, not findings about how much of a page a crawler needs, and quoting them as though they were findings would be the error this site keeps flagging in other people's numbers.
The related cap in the same area of the code is the one on structured data. Schema present only after JavaScript runs is worth at most a quarter of the schema sub-score, which is why both fixtures score 0.25 there despite declaring an Organization: the type appears in the rendered DOM and not in the response. Google publishes the opposite behaviour for itself. Google's page on structured data generated with JavaScript, carrying Last updated 2025-12-10 UTC, states that Google Search can understand and process structured data that is available in the DOM when it renders the page. Both can be true at once, and the gap between them is exactly the question of which fetchers run a browser, which is not uniform across the crawler tokens this scanner evaluates. Reading more into a structured data block than the fetcher can see is a recurring way to overestimate a page.
What two synthetic fixtures do not show, and what to check instead
The honest boundary on all of the above is that the divergence was built, not found. Ten fixtures in this repository produce a parity number, and retrievability differs from parity in exactly one of them, the constructed hydration case. On the five real sites captured on 15 July 2026, being the astro.build homepage, the gymshark.com homepage, the developer.mozilla.org landing page for HTML, the webflow.com homepage and the English Wikipedia article on web crawlers, the two figures are identical to every decimal place: 1 and 1, 0.958 and 0.958, 0.988 and 0.988, 0.980 and 0.980, 0.994 and 0.994. Five pages is not a survey, and the same caveat was attached when three of five captured pages carried JSON-LD. What the identity does establish is that on those five, the JSON haystack added nothing parity had not already found, so nothing here is evidence about how common the pattern is in the wild. The running survey of what real scans show sits on the research page and the crawlability study rather than in a fixture directory.
Nor does any of this describe what a given AI crawler does. Google publishes its own process: Google's JavaScript SEO basics, Last updated 2026-03-04 UTC, describes crawling, rendering and indexing as three separate stages, says a page may stay on the render queue for a few seconds but that it can take longer than that, and says a headless Chromium renders the page once resources allow. That is a documented deferral rather than a refusal. No comparable process document exists for the crawlers that feed AI answers, so whether any of them would recover the 485 words out of that payload is not something this repository knows or claims. The measurable thing is what your server sends, which is why the scoring methodology reports the response rather than an inference about the reader.
For anyone who wants to check their own site rather than read about fixtures, the sequence is short. Fetch the page with no browser and look at what came back. If your copy is not in the markup, find out whether it is anywhere in the response at all, because that tells you whether you are looking at a rendering problem or a data-shape problem, and the fixes differ. Shipping the prose server-side is the fix in both cases, and the per-stack notes are written up for Next.js, for React and for Framer. We also keep a deliberately client-rendered page of our own, the client-rendered canary, whose one fact is assembled by JavaScript after load so that only a JavaScript-executing agent can read it.
None of this makes a page uncitable on its own, and it would be overclaiming to say it does. What it changes is whose job the extraction becomes. A page that ships readable prose has done the work once, for everybody. A page that ships a state blob has moved the work to whoever fetches it, and every fetcher that declines gets a title and an empty container. That is the whole practical content of AI visibility at this layer, and it sits underneath everything else anyone does for generative engine optimisation: a sentence nobody extracted cannot be quoted, however good it is.
| Fixture | Real capture | Prose parity | Retrievability | Differ |
|---|---|---|---|---|
| astro-static | astro.build | 1.000 | 1.000 | No |
| gymshark-shopify | gymshark.com | 0.958 | 0.958 | No |
| mdn-html-docs | developer.mozilla.org | 0.988 | 0.988 | No |
| webflow-platform | webflow.com | 0.980 | 0.980 | No |
| wikipedia-web-crawler | en.wikipedia.org | 0.994 | 0.994 | No |
| synthetic-ssr-perfect | Constructed | 1.000 | 1.000 | No |
| synthetic-noai | Constructed | 1.000 | 1.000 | No |
| synthetic-robots-blocked | Constructed | 1.000 | 1.000 | No |
| synthetic-csr-spa | Constructed | 0.000 | 0.000 | No |
| synthetic-hydration-next | Constructed | 0.000 | 0.980 | Yes |
Lantad
Published .
A common way to check whether a crawler can read a page is to fetch the URL without a browser and search the response for a sentence you know is on the page. If the sentence is there, the check passes. That test is cheap, it is the right instinct, and on a modern front end it can pass on a page that no reader will ever get a word out of, because the sentence is sitting inside a hydration JSON payload rather than in any element that holds text.
Common questions
What is the difference between prose parity and retrievability?
The haystack. Both compare the rendered main content of a page against the initial HTTP response using the same containment function over overlapping runs of 8 words. Prose parity searches only the response's visible text, which excludes anything inside a script element. Retrievability searches that same visible text plus strings harvested out of JSON in the response, so it is never lower than parity and the gap between them is the portion of the page that exists as JSON data and not as readable markup.
Does content in a hydration payload count toward the AI Visibility Score?
No. SCORE_WEIGHTS in core/src/config.ts holds four keys, parity at 0.5, access at 0.25, structure at 0.15 and schema at 0.1, and retrievability is not one of them. A page whose prose ships only inside JSON scores 0 on prose parity and is graded as though the words were not there. Retrievability is still computed and printed, and it decides which defect is raised, but it contributes 0 of the 100 composite points.
Why do a client-rendered shell and a hydrated page get the same grade?
Because the only stored number on which they differ carries no weight. The fixtures synthetic-csr-spa and synthetic-hydration-next in this repository both produce a composite of 34.4 and a grade of F, with access 1, structure 0.462, schema 0.25 and prose parity 0 on both. Retrievability reads 0 on the first and 0.980 on the second, and the reports differ only in that number and in whether the defect raised is csr_blank or hydration_only.
Can Lantad detect any hydration payload?
No. Its extractor harvests JSON strings from two places only: any script element whose type contains json, which covers both JSON-LD and plain JSON data blocks, and an ordinary script assigning one of four hard-coded global names covering an initial state, Nuxt, Apollo state and preloaded state global. A payload assigned to any other global name is not harvested, so retrievability would read the same as prose parity and the report would raise csr_blank instead of hydration_only.
See what AI can read on your site
Run a free scan and get a graded report of exactly what AI crawlers can and cannot read, with ranked fixes.
Prose Parity
Prose Parity is Lantad's metric for the share of a page's rendered, human-visible text that is present in the raw HTML an AI crawler receives, where 100% means a crawler sees everything a human visitor sees.
AI crawler
An AI crawler is an automated bot, such as GPTBot, ClaudeBot, or PerplexityBot, that fetches web pages so an AI system can read, train on, or cite what those pages contain.
Structured data for AI (schema markup)
Structured data is schema.org markup, usually written as JSON-LD, that labels a page's facts in a machine-readable format so AI systems can extract and attribute them without having to interpret the surrounding prose.
JavaScript added no new crawl paths, and rendering dropped nine locale links
Counted with an HTML parser over five real pages this scanner captured and stored on 15 July 2026, the raw HTTP response held 991 distinct same-host URLs and the rendered DOM held 982. Three URLs existed only after JavaScript ran, and every one of the three was a rewrite of a link already present. Nine of the twelve that disappeared were one page's translations of itself.
Three of five captured pages carried JSON-LD, and one held 96 of the 102 nodes
Parsed out of five real pages this scanner captured and stored on 15 July 2026, four JSON-LD blocks hold 26,299 bytes and 102 typed nodes. One page accounts for 24,865 of those bytes and 96 of those nodes, 39 of them describing a navigation menu.
Why we will not grade a page we could not measure
Every AI visibility tool prints a number. When the measurement failed, that number is a guess wearing a lab coat, so ours refuses to appear.
