Do AI crawlers read iframes? 834 on 433 home pages, and 17 held 50 words

What an iframe is to a crawler, and why it is not part of your page

The definition is not ambiguous and it is not new. The MDN reference for the iframe element, last modified on 16 September 2026, opens by saying the element "represents a nested browsing context, embedding another document into the current one". The operative word is another. Two documents exist, each with an address, and nothing in the HTML obliges a consumer of the first to go and get the second.

Lantad's own extractor takes the conventional position, and it is worth stating plainly because this post is measuring against it. DROP_TAGS in core/src/extract.ts is a set of six element names whose subtrees are discarded before a single token is counted: script, style, noscript, template, svg and iframe. Text inside any of them never enters the corpus the scanner scores, which is the same rule that decides why a declarative shadow root contributes nothing and why 14 noscript elements on five captured pages held no text worth having. It is a decision this repository made rather than a finding about the world, and it sits underneath every prose parity number the product reports.

The strongest external evidence that the rest of the industry treats an iframe the same way is a tag that exists only because of it. On Friday 21 January 2022, Weizi Wang and Gary Illyes published a Search Central post introducing the robots tag indexifembedded. The problem it solves is stated in the post itself: media publishers want their content indexed when it is embedded on third party pages but do not want the media pages indexed on their own, so they add noindex, "However, the noindex tag also prevents embedding the content in other pages during indexing." The new tag works, in Google's words, "only when the page with noindex is embedded into another page through an iframe or similar HTML tag, like object".

Read that backwards and it is a specification of the default. The embedded document's own robots directive governs whether its content can be used inside the parent, because as far as the indexing system is concerned the two are separate pages that happen to be displayed together. The post also closes with a sentence that has aged into the central fact of this measurement: "Presently, only Google supports the indexifembedded tag."

Do AI crawlers read iframes? What 834 of them actually held

433 of the 1,076 reachable home pages carried at least one iframe, 834 elements in total, so the element is ordinary rather than rare. What it carries is the surprise.

688 of the 834 held an address that resolved to http or https once it was resolved against the page it sat on. 120 carried no src attribute at all, on 37 pages, which means the rectangle is filled by script after load and the bytes a fetch only reader receives contain nothing whatsoever; 41 of those 120 sit on one site, maroc.ma. A further 26 pointed at about:blank, a javascript: URL or a data: URL, where there is no second document to go and get. Those two groups are already decided before any crawl policy is consulted, and they are the same class of absence as the 11 of 271 pages that returned nothing readable until the bundle ran.

Lantad requested all 636 distinct addresses behind the remaining 688 frames, as LantadBot, on the same day. Sorting them by the host they point at, rather than by what anybody says they are for, gives the shape of the answer. 356 frames are tracking or advertising endpoints and hold 352 words between them. 150 are video players and hold 151 words. 30 are maps, 7 are social embeds and 2 are consent managers. That leaves 143 frames, on 81 pages, that are none of those things, and those 143 hold 8,367 of the 9,098 words found anywhere in an iframe on this corpus.

So the honest form of the question is narrow. It is not whether an AI crawler reads iframes in general, because the 545 tracking, video, map, social and consent frames hold 731 words between them and almost none of it was written by the publisher. It is whether the small remainder holds anything a reader would miss, and for most sites in this corpus it does not.

The single most common iframe on these home pages is a tag manager

354 of the 688 resolvable frames, spread across 331 separate pages, point at googletagmanager.com. That is a clear majority of every iframe with an address on this corpus, and every one of them is the same thing: the noscript fallback that Google Tag Manager's installation snippet asks you to paste immediately after the opening body tag, so that a page loaded without JavaScript still fires a tracking request.

It is worth pausing on what that means for the received advice. The warning against iframes is a warning about content, and the element's commonest use here is not content at all. It is a one pixel beacon that holds, across all 354 instances, a total of 352 words: boilerplate strings from the tag manager endpoint rather than anything a publisher wrote. A crawler that skipped every iframe on all 433 pages would lose those 352 words and would not have lost a sentence anybody intended a reader to see.

The distribution behind that head is long and thin. Video accounts for the next largest group, 84 frames pointing at youtube.com or youtube-nocookie.com across 37 pages, plus 8 at player.vimeo.com. 28 frames point at google.com, which covers maps, reCAPTCHA and Forms. After that the counts collapse into single sites embedding themselves: 19 frames on one page at thenationalnews.com, 11 on one page at bostonglobe.com, 7 on one at uct.ac.za. 90 of the 688 frames point at the same registrable domain as the page they sit on, across 33 pages, and 266 of the 834 elements carry a title attribute, which is the only label a reader without the rendered frame ever gets.

None of this is a claim about how any particular crawler behaves, because none of the vendors says. The OpenAI crawler documentation, Anthropic's crawler help article and Perplexity's bots guide were read in full on 21 September 2026 and the string iframe appears zero times in each of them, as does indexifembedded. That silence is consistent with the wider pattern this blog found when only two of nine vendors documented executing JavaScript at all.

The 17 frames that held prose, and the two that held most of it

Set a threshold of 50 words, which is roughly a short paragraph, and 17 frames on 12 sites clear it. Raise it to 100 words and 10 frames on 8 sites remain. Raise it to 200 and there are 4 frames on 4 sites. That is the whole of the problem on this corpus, and naming the cases is more useful than the rate.

Two of them hold two thirds of all the iframe prose on the corpus. fullsteamcrossfit.com embeds a review widget from services.leadconnectorhq.com, titled "Full Steam CrossFit Gym Happy Reviews", holding 3,314 words of customer testimonials. bls.gov embeds its own CPI inflation calculator from www.bls.gov, holding 2,758 words, most of it the month and year options in the form. Between them that is 6,072 words, 66.7 percent of every word found inside an iframe anywhere in this measurement.

The first of those is the case that should worry a business. Reviews are the single most quotable thing a local service has, they are written by other people, and an answer engine asked to recommend a gym has every reason to want them. Delegating them to a widget puts them behind a second address the parent page never asks anyone to follow, which is a different failure from the markup problem this blog described when review authenticity turned out not to be in the markup and a close relative of finding that only 2 of 172 home pages carried opening hours a machine could read. The same site, incidentally, appeared in the schema measurement published earlier on 21 September 2026 for writing a week of opening hours as one string with line feeds in it.

The remainder are product demonstrations and forms. littlebird.ai carries six frames from its own embeds subdomain, four of them substantial, titled "Littlebird product demo", "Littlebird routines and briefs", "Littlebird drafting" and "Littlebird meeting notes", holding 753 words between them, which is most of what the page says about what the product does. wits.ac.za embeds a social media wall from snapwidget.com carrying 90 words of caption text. abstractlab.co embeds 340 words of testimonials from embed.testimonial.to. mountsinai.org embeds 149 words from its own doctor subdomain, anvilogic.com 149 from media.anvilogic.com, appalshop.org 126 words of newsletter copy, gov.br 66 twice over, chop.edu 64 words of survey, census.gov 58 and 51 in two economic indicator widgets, and dailyui.co 128 words of event registration form from app.youform.com.

Twelve sites out of 1,076 is not an epidemic. It is also not nothing, and the twelve are not random: they are sites whose most persuasive content happens to be the content a third party tool renders. That is the pattern to check for, rather than the element.

26 embed addresses said noindex, and not one said indexifembedded

The 143 frames that were not tracking, video, maps, social or consent resolve to 125 distinct addresses. Lantad requested each of them once more on 21 September 2026, this time reading the response headers and the robots meta elements rather than the text, and the result is the finding this post exists for.

Start with how many answer at all. 85 of the 125 returned a 2xx status, 84 of those with an HTML content type. 37 returned a 4xx, of which 26 were a 404 and the rest 400, 401 or 403. Two returned a 5xx and one failed in the client. A third of these embed addresses, in other words, do not serve anything to a plain request made without a Referer header, a session or a browser. Some of that is deliberate protection and some of it is a widget that only works in context, and this measurement cannot separate the two. What it can say is that the address written into the parent page is not, for 37 of 125, an address that returns a document.

Now the directives. 26 of the 125 carry a noindex, 24 of them in a robots or googlebot meta element, 3 in an X-Robots-Tag response header, and 1 in both. Among them are embed.handelsblatt.com on handelsblatt.com, www.cbsnews.com on its own home page, go.us.ovhcloud.com on ovhcloud.com, 1jpgd.share.hsforms.com on bind.com.mx, play.mediaflow.com on 1177.se, and iframe.iono.fm on wits.ac.za. Two of the 26 hold real text: redcap.chop.edu with 64 words and app.youform.com with 128.

Not one of the 125 carries indexifembedded. Not one. The tag Google published in January 2022 for precisely this situation, and which its robots meta tag reference still documents, appears zero times across every embedded document on this corpus. For the 26 that combine noindex with no indexifembedded, the embedded content is refused twice over: dropped from the parent page because it is a separate document, and refused on its own address because the document says not to index it. That is the same directive this blog has measured elsewhere, when only two of nine crawler vendors documented what noindex does to them and when five of 391 X-Robots-Tag noindex headers turned out to be a captcha.

The charitable reading is that nobody meant to hide anything. A form vendor sets noindex on every hosted form by default, which is sensible for the form's own URL and says nothing about the publisher's intent for the page that embeds it. The uncharitable reading is that it does not matter what anybody meant. The directive is what ships.

What this does not measure, and what to check on your own site

The limits are worth more than the rate here, because the rate is low and a low rate is exactly what invites over-reading.

Only the home page of each site was read, so nothing here describes a product template, a location page or an article page, and a site that embeds nothing on its front door may embed everything on a listing page. No JavaScript was run, so any iframe injected after load was invisible to this measurement and the 120 frames with no src attribute are counted as empty on the strength of the bytes alone, which is what a fetch only reader gets and is not what a person sees. 343 of the 1,419 hostnames never reached the reachable set, so they are absent from every figure and they are not absent at random. The grouping of frames into tracking, video, maps, social and consent was done by matching the host against a named list, which is a judgement made in this measurement rather than a standard anybody publishes. And the word counts come from removing tags and dropping the same six subtrees core/src/extract.ts drops, which is the rule this scanner scores by and not a claim about what any other consumer does.

Most of all, this measured what is in the frames, not what any crawler did with them. No vendor documents the behaviour, so a statement about GPTBot or ClaudeBot here would be invention. What can be said is what an operator can verify without waiting for a vendor to publish, and that is the useful output: fetch your own page with no JavaScript, list every iframe, and request each address the way a crawler would, which is what the tool that shows the bytes a bot is sent does for a single URL and what the scanner's stated conduct commits it to doing politely.

Three checks follow, in order of how much they are worth. First, does the frame hold prose a reader would quote back at you, meaning reviews, specifications, prices, opening hours or an answer to a question people ask? If it does not, leave it alone; a video player and a map are fine where they are. Second, if it does, does that content also exist in the parent document in some form, even abbreviated? Duplicating reviews as text beneath the widget costs nothing and is the entire fix. Third, and only if the first two point at a real loss, is the embedded document itself indexable on its own address, which is where the noindex above becomes your problem rather than your vendor's. That ordering matters because the element is not the defect. Delegating your most quotable sentences to a second document is, and it belongs in the same family as offering a crawler no internal path at all and letting navigation account for half the text blocks on a page, both of which decide what a reader can reach before any question of AI visibility or how the score is built arises.