AI visibility audit: 1,344 of 5,669 facts were absent from the HTML a crawler receives

What should an AI visibility audit check that a word count does not?

It should check the facts, separately from the prose, because the two fail independently. A page can serve a crawler every paragraph of its marketing copy and none of its prices, and a word-level reading of that page is close to perfect.

The extractor used here is deterministic and asks no model anything. It reads seven kinds of fact out of a document: prices from currency patterns and from markup, phone numbers and email addresses from tel: and mailto: links as well as text, street addresses from postcode and street patterns, opening hours from day and time patterns, question and answer pairs from accordion markup, and table rows from real table elements. Facts also come out of JSON-LD, which is where a price belongs in structured data as a property of an Offer node. The same HTML always yields the same facts, and the version number FACT_EXTRACTOR_VERSION, currently 1, changes whenever that output could change, so a stored reading can always say which extractor produced it.

Each fact then gets one of two statuses. Present means the initial HTML carried it, so a crawler that executes nothing still reads it. Absent, which the code calls hidden, means only the rendered page carried it. The comparison is deliberately asymmetric: a fact the raw HTML has and the render loses still counts as present, because crawlers read raw HTML and a fact cannot be taken away from them by a script.

Not every fact weighs the same. The weights in core/src/config.ts put a price, a phone number, a street address and opening hours at 3 points each, a question and answer pair and an email address at 2, and a table row at 1, on the reasoning that one row is rarely the whole answer to anything. That weighting is a decision somebody made rather than something measured, which is the distinction this blog draws on the methodology page and keeps drawing. It matters for answer engine optimization because the thing being optimised for is a quotation, and an engine cannot quote what it never received.

One thing to state plainly before any number below is read. This capability ships behind a flag, w2_score_v2, whose defaultOn value in core/src/config.ts is false. It is not scoring anybody's site in production today. The figures here come from running the shipped functions over the corpus directly, which is a measurement Lantad made with its own code, not a report from a live product.

Which facts were missing from the crawler's HTML

Email addresses were proportionally the worst at 209 absent of 552, or 37.9 percent. Table rows came next at 116 of 358, then phone numbers at 238 of 853 and prices at 662 of 2,464. Prices dominate in absolute terms simply because there are more of them: 2,464 price facts across 386 pages, against 254 opening-hours facts across 88.

Street addresses behaved differently from everything else, with only 41 of 602 absent. That is worth a sentence because it points at a mechanism rather than a habit. An address is usually typed into the page template or into a JSON-LD block in the document head, and both of those ship in the initial HTML. A phone number is just as often rendered as a tel: link inside a contact widget that a script builds after load, which is how 238 phone numbers went missing while the addresses beside them stayed. This blog found the same split from the markup side when it measured that 119 of 191 home pages with a clickable phone number named no telephone in their markup.

The concentration is the more useful finding. 765 of the 1,054 pages carried at least one fact, and on 579 of those every single fact was in the crawler's HTML. 186 pages were missing at least one. So this is not a condition most sites have mildly; it is a condition a minority of sites have severely. 68 pages had a weighted fact parity of exactly zero, meaning not one of their facts reached the crawler, and 75 pages came in under 0.25. Of the 186 pages missing anything, 69 were missing only prices, which is the pattern a storefront produces when its catalogue is fetched client-side. Lantad has measured the price half of this before on pricing pages specifically, finding that 17 of 93 SaaS pricing pages put a price in the markup, and the table rows have their own history too: when the blog counted real tables it found 87 of them across 1,079 home pages, so 358 table-row facts here come from a small number of table-heavy pages and the 32.4 percent is a figure about them rather than about the web.

Most of these pages were not empty shells

The obvious explanation for a missing fact is that the crawler got no page at all, and that explanation fails here. Of the 186 pages missing at least one fact, 151 had already sent the crawler 50,000 bytes or more of HTML. Only 20 sent under 5,000 bytes. The median page in the whole measured set sent 249,716 bytes and the median page missing a fact sent 235,333, which are the same page by any practical reading. These sites are serving a substantial document and leaving specific facts out of it.

Two of them were read by hand on 6 November 2026 to confirm the parse. u.ae, the United Arab Emirates government portal, sent LantadBot 968,619 bytes of HTML containing none of the 120 facts the rendered page carried: 50 phone numbers, 50 email addresses, 18 opening-hours statements and 2 street addresses, every phone number arriving as a tel: link that only exists once the page has run. dbs.com sent 305,985 bytes and withheld 33 phone numbers and 10 street addresses the same way. weizmann.ac.il sent 182,825 bytes and withheld 44 email addresses and 18 phone numbers. None of those is an empty shell, and 81 of the 151 large-page cases withheld a contact fact of exactly this kind.

The shells do exist and they are a different failure. heb.com answered LantadBot with 212 bytes, and the browser then rendered 2.8 megabytes carrying 50 prices. theregister.com answered with 2,101 bytes against 1.2 megabytes rendered. A page like that has not hidden its facts; it has declined to send a page, which this blog has measured separately as 11 of 271 pages returning no words to a crawler at all.

Whether any of this stops a given crawler depends on whether that crawler renders, and the honest answer is that most vendors do not say. Google's JavaScript SEO documentation, last updated 2026-03-04, states that Google processes JavaScript web apps in three phases, that Googlebot queues all pages with a 200 status for rendering, that the page may stay on this queue for a few seconds but it can take longer than that, and that if the content is not visible in the rendered HTML, Google will not be able to index it. So Google renders, with a delay it declines to bound. OpenAI's crawler documentation, read at source on 6 November 2026, lists four tokens and their purposes and says nothing whatever about JavaScript or rendering. That silence is the general case, and the blog counted it: 2 of 9 operators state a position either way. A fact absent from the initial HTML is therefore a fact whose availability depends on an undocumented behaviour, which is a weaker position than having it in the bytes.

Where the gap concentrates, by industry and by platform

The corpus is stratified by industry and by the platform a site is built on, which lets the same measurement answer both questions. By industry, finance was worst in absolute terms with 24 of 92 measured pages missing a fact, then ecommerce at 23 of 61, software as a service at 20 of 118, and healthcare and travel at 16 each. Ecommerce is the worst rate of any industry stratum: 23 of its 52 pages that carried any fact were missing one.

By platform the spread is wider and more interesting, because a platform decides how a page is assembled. Every one of the 25 framer sites carrying a fact had all of its facts in the crawler's HTML, and 2 of 49 wix and squarespace sites were missing one. Against that, 12 of the 32 shopify storefronts carrying a fact were missing one, and 9 of 38 wordpress sites. The hosted page builders that render on the server are doing the right thing by default, which is the same conclusion the Framer stack guide reaches from the other direction, and it is why the Shopify guide spends its length on what the storefront does after load rather than on markup.

That pattern is worth stating carefully, because it is about defaults rather than about merit. A framework that renders on the server ships facts in the HTML without anybody deciding to; a framework that hydrates on the client hides them without anybody deciding to either. The Next.js guide exists because the same codebase can do both depending on which rendering mode a route uses. This blog has measured the markup half of the same effect, finding that 27 of 404 home pages with no structured data in the HTML gained some when a browser ran the page, and the fact-level reading here is the same phenomenon with the facts a buyer actually asks about rather than the schema that describes them.

The government stratum deserves a note against itself. It had 9 pages missing a fact out of only 38 that carried any, and 9 of its pages had a fact parity of zero, the highest count of total failures in the corpus. Those are the sites whose phone numbers and opening hours are the answer to the question somebody is asking.

On more than half the pages this measure cannot apply at all

The inconvenient result of the run is about the measure rather than about the sites. 557 of the 1,054 pages carried fewer than three facts, and 289 carried none in either document. Three is the minimum in ANSWER_PARITY.minFacts in core/src/config.ts, and below it the code does not blend at all: it falls back to Prose Parity alone and the report says so. So on 52.8 percent of the corpus a fact-level audit has nothing to weigh, and the measure that this post is about quietly becomes the older measure it was meant to improve on.

That threshold is not arbitrary and the comment in the config explains why it is there: with one or two facts on a page, a single absent phone number moves the figure by a third or a half, which is a swing that describes the extractor's luck rather than the site. Raising the floor would make the measure steadier and apply it to fewer pages still. Lowering it would produce confident numbers from two data points. Neither is a better answer than publishing the share of pages it cannot speak about, which is 52.8 percent.

The 289 pages with no facts at all are mostly doing nothing wrong. A documentation home page, a news front page or a brand splash page has no price, no opening hours and no question and answer pairs to carry, and the static-docs stratum shows it plainly: only 7 of its 31 measured pages carried a single fact. A measure of facts is the wrong instrument for those pages and the honest output is silence, which is the same argument this blog has made about withholding a grade when a scan could not measure what it needed.

Where a page does carry facts, the related question is whether the ones it carries are the ones an engine wants, and that is a different gap from this one. The blog measured it on FAQ markup, finding that 298 of 1,277 declared answers were not on the page at all, which is the inverse failure: markup promising a fact the document does not contain, rather than a document containing a fact the markup and the initial HTML both miss. Readers who want to see the crawler-side document for a single page of their own can get it from what GPTBot sees, and the vocabulary for the whole subject is on the AI crawler page.

What this measurement does not show

It does not show that any AI crawler failed to read these facts. It shows that the facts were not in the initial HTML, and whether that matters to a particular crawler depends on whether that crawler renders JavaScript, which Google documents that it does and most other vendors do not address. Read the figures as a measure of what a site hands over without conditions, not as a count of lost citations.

The extractor has a false-positive mode this run found, and it is worth more than a footnote because it inflates one of the headline numbers. The price pattern in core/src/facts.ts guards against a digit following the amount but not a letter, so a news headline reading one billion pounds written with a currency symbol, a 1 and a B is read as a price of one pound. theregister.com contributed 25 such facts and every one of them is a figure in a headline rather than a price anybody can pay. Editorial pages are where this bites, so the 26.9 percent absent rate for prices should be read as a rate about a population that includes some non-prices, and the contact-fact figures, which come from tel: and mailto: links rather than from text patterns, are the sturdier half of this post.

Counts are capped. FACTS.maxPerType keeps 50 facts of one type per page so that a catalogue page stays bounded, and 14 type slots across 12 pages hit that cap, u.ae and heb.com and statssa.gov.za among them. Those pages carry more facts than are counted here, so the totals are a floor. One page per site was read, on one day, and it was the home page, which is the page least likely to carry a price and most likely to carry a phone number. A pricing page, a product page or a contact page would each give a different answer, and the figures here should not be read as a site-level verdict.

The corpus is a stratified sampling frame for crawlability rather than a census of the web, so a stratum count such as 12 of 32 Shopify storefronts indicates direction and not a market rate, and the crawlability study states how the frame was built. 365 of the 1,419 hostnames contributed nothing at all: 31 failed to answer a robots.txt request, 56 answered it with a 5xx that RFC 9309 tells a crawler to read as complete disallow, 11 disallowed LantadBot at the root and were not fetched, 252 did not answer the home page with HTML and a 200 status, 1 was unreachable, and 14 answered the crawler but failed to render. Finally, this is a reading of one slot in a composite and not a score: the whole-corpus scoring picture, where 972 of 1,419 sites were graded and parity took 60.7 percent of every point they lost, is the context this sits inside, and a per-page report with the absent facts listed is what a scan produces.