# Deep research kept cited links above 94 percent, and no model reached 77 on facts > A source attribution study posted to arXiv on 7 May 2026 by researchers at PricewaterhouseCoopers benchmarked 14 language models on 130 research queries. Twelve of the 14 kept cited links working above 94 percent of the time, while factual accuracy against those same sources ran from 24.4 to 76.8 percent. The dimension a publisher can change was the one already near its ceiling. - Canonical page: https://lantad.co/blog/deep-research-links-worked-facts-did-not - This file: https://lantad.co/blog/deep-research-links-worked-facts-did-not.md - Last substantive update: 2026-08-17 ## Key facts - **Published:** 2026-08-17 - **Category:** Findings - **Author:** Lantad - **Length:** 3405 words - **Takeaway 1:** A source attribution study from the Commercial Technology and Innovation Office at PricewaterhouseCoopers, posted to arXiv on 7 May 2026 as arXiv:2605.06635, benchmarked 14 language models on 130 research queries drawn from DeepResearch Bench and BrowseComp. - **Takeaway 2:** Twelve of the 14 models scored above 94 percent on Link Works, the study's test of whether a cited URL returned content, while Fact Check scores ran from 24.4 percent for OSS-120B to 76.8 percent for Claude Opus 4.5. - **Takeaway 3:** The paper names three causes of a failed link, being HTTP 404 errors, blocked access covering paywalls and bot detection, and connection timeouts, and publishes no split between the three. - **Takeaway 4:** In the study's search depth ablation, GPT-5.4's Fact Check accuracy fell from 78.6 percent at 2 tool calls to 16.7 percent at 150, while its Link Works score stayed above 97 percent at every one of the seven depths. - **Takeaway 5:** Lantad ran no part of this study, has measured none of these models, and reports every figure below from the paper itself, read on 17 August 2026. ## Summary A deep research agent reads dozens or hundreds of pages and returns a report with inline citations, and the citation is the whole promise: it says a human can go and check. Almost every argument for making a site readable by machines rests on that promise being worth something. If the link resolves and the page says what the report claims it says, the citation did its job. If either half fails, the reader has a footnote rather than a source. Researchers in the Commercial Technology and Innovation Office at PricewaterhouseCoopers put numbers on both halves. Their paper, [Cited but Not Verified: Parsing and Evaluating Source Attribution in LLM Deep Research Agents](https://arxiv.org/abs/2605.06635), was posted to arXiv on 7 May 2026 under a CC BY 4.0 licence, and it evaluates citations produced by 14 language models across 130 research queries along three separate dimensions: whether the URL returns content, whether the content is on the same topic as the claim, and whether the content actually supports the claim. Every figure in this post is read from that paper. Lantad measured no part of this. What this site can add sits in the first of those three dimensions, because whether a cited URL returns content to whoever fetches it is the only one of the three that a site owner has any say over, and the study's own numbers say it was already the dimension least in trouble. ## What the study measured, and how it separated three things people usually merge The paper's contribution is the separation more than any single number. Most work on citation quality collapses into one binary question, did the model attribute correctly, and the answer to that question hides at least three different failures. This framework asks them one at a time. Link Works is the first and the only one that runs without a language model in the loop. For each cited URL, a web content extractor retrieves the page, and the evaluator returns 1 if the URL returns accessible content and 0 if the request fails on an HTTP error, a timeout, or blocked access. Relevant Content is the second: a judge model is given the claim and up to 5,000 characters of the retrieved source and decides whether the source addresses the same topic. Fact Check is the third and the strictest: the same retrieved content is checked against the specific facts, numbers, dates and assertions in the claim, scoring 1 only if they are supported or consistent and 0 if they are contradicted, absent or uncertain. The paper states that the Fact Check evaluator was calibrated through manual review of 50 to 100 judgments. Before any of that, the reports are parsed rather than interpreted. As set out in [the paper's methodology](https://arxiv.org/html/2605.06635v1), a Markdown abstract syntax tree parser extracts citation and claim pairs with no model inference at all, normalising whitespace, stripping fenced code blocks so a URL inside a code sample cannot be counted, and recognising numbered references, footnote references, inline links, autolinks and ranges. One choice in that parser is worth knowing before reading any of the scores: when a citation appears at the end of a passage, the parser applies it backwards to all preceding uncited sentences in that passage. That is a defensible reading of how models actually cite, and it also means a single reference is being asked to support several sentences, which will tend to push the Fact Check number down relative to a stricter one to one reading. The setup is 14 models across OpenAI, Anthropic, Google and three open source alternatives, run against 130 research queries drawn from two existing benchmarks, DeepResearch Bench and BrowseComp. Evaluation ran with 10 concurrent agents and 15 concurrent evaluators, with five retries at five second delays to absorb transient failures. That retry logic matters for reading the link numbers honestly, because it means a single flaky response was not scored as a dead link. ## Link Works is the only one of the three a publisher can change Read the hero table by column rather than by row and the shape is obvious. Link Works clusters high: the paper states that 12 of the 14 models exceed 94 percent, and the two that do not are open source, with Llama 4 Maverick at 80.8 percent and OSS-120B at 83.9 percent. Fact Check spreads across the whole range, from 24.4 percent for OSS-120B to 76.8 percent for Claude Opus 4.5, which the paper calls a 53 percent spread and the most differentiating dimension of the three. The error analysis is where this becomes a question about websites rather than about models. The paper names three categories behind a Link Works failure: HTTP 404 errors where content was removed or the URL changed, blocked access covering paywalls and bot detection, and connection timeouts. It does not publish a split between the three, and that absence is worth stating plainly rather than guessing at, because the second category is the one a site owner configures and the first two are not the same problem at all. A 404 is a publishing decision that has already happened. Bot detection is a live rule that decides, per request, whether the fetch that arrives is served, and the difference between those two layers is the subject of [the two layers that decide whether AI can read your site](https://lantad.co/blog/two-layers-decide-if-ai-can-read-your-site). What the paper does quantify at that level is the extremes. GPT-5.4 recorded one failed link out of 2,159 evaluations, which is what the 100.0 percent in the table is a rounding of, while Llama 4 Maverick failed on 19.2 percent of its links and OSS-120B on 16.1 percent. Rate limiting was close to irrelevant: HTTP 429 responses affected fewer than 0.3 percent of evaluations and moved adjusted pass rates by less than 0.5 percent. That is a useful null result for anyone who assumed throttling was the main thing standing between an agent and a page, and it sits oddly against the finding that [no crawler vendor documents Retry-After](https://lantad.co/blog/no-crawler-vendor-documents-retry-after) anywhere in its published guidance. The blocked access category is the one this site exists to look at. A rule that returns a 403 to an unrecognised client is invisible in a robots.txt file, and [the Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309.html) is explicit that its rules are not a form of access authorization, so the file and the response are separate claims about the same page. A block can also run the other way and be harder to see: a site can serve [a 200 to a crawler it believes it has banned](https://lantad.co/blog/gptbot-bans-served-a-200-anyway). Paywalls have their own vocabulary that [names exactly two Google crawlers](https://lantad.co/blog/paywall-markup-names-two-google-crawlers) and no AI crawler at all, and the identity a block is keyed on is itself only [a claim rather than an identity](https://lantad.co/blog/a-user-agent-is-a-claim-not-an-identity). ## More search made the facts worse and left the links alone The ablation is the part of this paper that will be quoted most, and it is also the part that separates the two halves of a citation most cleanly. The authors varied the maximum number of tool calls across two models, GPT-5.4 and Claude Opus 4.6, at seven intervals: 2, 10, 30, 50, 70, 100 and 150. For GPT-5.4, Fact Check accuracy runs 78.6 percent at 2 calls, 45.9 at 10, 43.0 at 30, 38.0 at 50, 35.5 at 70, 37.2 at 100 and 16.7 at 150. For Claude Opus 4.6 the same series runs 80.0, 74.4, 69.2, 61.2, 61.7, 58.7 and 57.9. The paper describes the average decline across both models as approximately 42 percent, and gives the two individual declines as 62 for GPT-5.4 and 22 for Claude Opus 4.6. Those last two are differences between percentages rather than proportional changes, and the paper writes them with a percent sign, so read them as points and do not compound them with anything. The asymmetry is the finding. Across every one of those seven depths, for both models, Link Works and Relevant Content stayed above 92 percent. Only Fact Check moved. The authors read that as evidence of an information overload effect in synthesis rather than in source selection: the models keep finding accessible, on topic pages no matter how many they read, and lose the ability to state accurately what those pages said. The sharpest single drop is between 2 and 10 tool calls for GPT-5.4, from 78.6 percent to 45.9, which suggests the effect does not need an extreme search budget to appear. There is a second version of the same trade off in the cross model results. The paper reports that OpenAI models achieved 100 percent task success and generated the most citations, with GPT-5 Mini producing 1,272 total attributions, while their Fact Check accuracy spanned 39 to 59 percent. Anthropic models had lower task success at 83 to 97 percent and higher factual accuracy. Open source models made far fewer tool calls, 7 to 60 against 73 to 211 for the frontier models. The authors hypothesise attention dilution during synthesis and are careful to call it a hypothesis. That instability across runs will be familiar to anyone who has tried to measure this from the outside. A separate study covered here found that [AI visibility took seven runs per prompt to settle](https://lantad.co/blog/ai-visibility-took-seven-runs-per-prompt-to-settle), and the same discipline applies to these numbers: a single report from a single model on a single query is one sample of a process that does not repeat itself. It is also why [more citations did not mean more influence on the answer](https://lantad.co/blog/citation-count-is-not-answer-influence) is a separate question from this one. ## The verifier rendered JavaScript, and the crawlers it was checking are not documented to One sentence in the methodology changes how the Link Works column should be read, and it is easy to skim past. Section 3.3.1 says that for each cited URL a web content extractor capable of handling JavaScript-rendered pages retrieves the content. The instrument that produced those 94 to 100 percent figures was therefore a rendering fetch, not a plain one. That is the right choice for the study's own question, which is whether a citation can be verified at all. It is the wrong instrument for a different question, which is whether the crawler that produced the citation could have read the page in the first place. Those two fetches are not interchangeable, and a page that is empty until JavaScript runs will pass the first and can fail the second. Measuring the gap between a raw response and a rendered one is the whole reason [prose parity](https://lantad.co/glossary/prose-parity) exists as a metric here and why [the method used to score a page](https://lantad.co/methodology) fetches twice rather than once. How wide that gap is for AI crawlers specifically is not something this paper addresses, and it is not something anyone can settle from vendor documentation, because the documentation does not discuss it. A count published on this blog on 14 August 2026 found the words JavaScript, render, scroll, viewport and lazy appearing zero times across the crawler pages published by OpenAI, Anthropic and Perplexity, which is the finding behind [Google does not interact with your page, so scroll-loaded content never loads](https://lantad.co/blog/lazy-loading-and-crawlers-that-do-not-scroll). Absence of documentation is not evidence of absence of rendering. It is only a reason nobody outside those companies can plan against it. The practical consequence for a publisher is narrow and worth stating without inflation. A citation surviving this study's Link Works test tells you the URL was reachable by a capable renderer at evaluation time. It does not tell you the citing agent's own crawler reached it, and it does not tell you the page will be reachable tomorrow. The paper says the second part itself, listing temporal instability among its limitations: URLs accessible during evaluation may become unavailable through content removal, domain expiration or access policy changes, and source content itself may change. If you want to know what an unauthenticated non-browser client currently receives from one of your own URLs, that is an external observation, and [what GPTBot sees](https://lantad.co/tools/what-gptbot-sees) is the shape of it. ## What a publisher should take from a number that was already high The inconvenient reading of this paper, for anyone selling readability measurement, is the one worth leading with. If cited links already work between 94 and 100 percent of the time for every frontier model tested, then whatever is going wrong with deep research citations is mostly not that pages could not be fetched. Making a page more fetchable moves a number that was not the bottleneck in this study. Fact Check is a property of how a model synthesises what it read, and nothing in your HTML changes it. That is a real limit on what a scanner can promise, and it is the same limit that makes this site [withhold a grade](https://lantad.co/blog/why-we-withhold-a-grade) rather than issue a confident one when the evidence does not support it. Readability is a precondition, not a mechanism. It decides whether you are eligible to be cited accurately, not whether you will be. The second thing to take is a boundary in the study's design, and it cuts the other way. The framework evaluates citations that an agent already produced. Every URL in that dataset is a URL some model chose to cite, which means a site that was never cited never enters the denominator at all. The paper measures the quality of citations, not the distribution of them, so it says nothing about which sites get selected. That question is a different literature, and the closest measured answer covered here is that [four content factors decided the first citation and formatting was not one of them](https://lantad.co/blog/four-factors-decided-the-first-citation), alongside the finding that [ChatGPT citations are rare and tend to land on the homepage](https://lantad.co/blog/chatgpt-citations-are-rare-and-land-on-the-homepage). The third is a practical one about your own error surface. Of the three failure categories the paper names, one is fully within a publisher's control and shows up nowhere in a robots.txt file: whether the layer in front of your origin serves an unrecognised client. A 404 you can find in your own logs. A block by bot detection you generally cannot, because the request that was refused never reached the application that writes the log, and a 5xx during a robots.txt fetch is read by a well behaved crawler as a whole site closure, which is why [a 404 and a 503 on robots.txt are opposites](https://lantad.co/blog/robots-txt-404-and-503-are-opposites). Size is a quieter version of the same thing: content past a fetch limit is present on the page and absent from what was read, as in [the first two megabytes Googlebot reads](https://lantad.co/blog/googlebot-reads-the-first-two-megabytes). The paper carries two further limitations its authors state and this post will not soften. The Relevant Content and Fact Check judges are language models, calibrated by human review but still subject to position bias and self-enhancement effects, and the evaluation covers only models with web search, excluding enterprise retrieval over private corpora. Neither undermines the Link Works column, which needs no judge, and both are reasons to treat the Fact Check spread as a ranking rather than as an absolute rate. For the vocabulary underneath all of this, [generative engine optimization](https://lantad.co/glossary/geo) and [answer engine optimization](https://lantad.co/glossary/aeo) are defined here, [AI visibility](https://lantad.co/glossary/ai-visibility) is the outcome, and [an AI crawler](https://lantad.co/glossary/ai-crawler) is the thing that has to reach the page before any of it applies. What a scan of that kind can and cannot see at population scale is set out in [the crawlability study](https://lantad.co/research/crawlability-study). ## Questions and answers **What did the PricewaterhouseCoopers study actually measure?** It evaluated inline citations from reports written by deep research agents along three dimensions: Link Works, whether the cited URL returns accessible content; Relevant Content, whether the source is on the same topic as the claim; and Fact Check, whether the source supports the specific claim. The paper, arXiv:2605.06635, was posted on 7 May 2026 and covers 14 models across 130 research queries. **Does a working citation link mean the claim is accurate?** No, and separating those two is the paper's central point. Twelve of the 14 models kept Link Works above 94 percent while Fact Check ranged from 24.4 to 76.8 percent. In the search depth ablation, Link Works and Relevant Content stayed above 92 percent at every depth while GPT-5.4's Fact Check fell from 78.6 percent to 16.7 percent. **Can a site owner do anything about these numbers?** Only about the first of the three. Link Works failures come from HTTP 404s, blocked access including paywalls and bot detection, and connection timeouts, and the middle category is a configuration a publisher owns. Fact Check is a property of how the model synthesises what it read and no change to a page affects it. **Did the study test what an AI crawler receives from a page?** No. Section 3.3.1 states that its extractor was capable of handling JavaScript-rendered pages, so the Link Works figures describe what a rendering fetch retrieved at evaluation time. Whether the crawler that produced the citation could read the same page is a separate question the paper does not address. --- Lantad measures whether AI crawlers can actually read a page: it fetches as a non-rendering crawler, renders as a browser, and reports the gap. Free scan, one URL, no signup. Method and weights: https://lantad.co/methodology | All pages as markdown: https://lantad.co/md | Crawler policy: https://lantad.co/bot