A stop request stopped 0 of 40 AI agents in the middle of a task

What the Recuse Signal paper measured

The paper is titled Will the Agent Recuse, and Will It Stop? Measuring LLM-Agent Compliance with In-Band Governance Signals at the Access Door and Mid-Flight, by Thamilvendhan Munirathinam. It was first posted on 4 June 2026 and revised three times, reaching version 4 on 1 August 2026, and it is released under CC BY 4.0. The problem it starts from is not a web problem. An agent holding real credentials connects to a server, and from the server's side it is indistinguishable from the human whose key it is using, so the server has no way to say automated access is not welcome here.

The proposal is a signal rather than a control. A server emits one sentinel line plus human readable notice text over a channel the client is already reading: a pre-authentication SSH banner, a PostgreSQL notice raised before the first ready-for-query, a Kubernetes admission warning. Version 0.1 of the mini standard defines three access time directives, deny meaning withdraw, throttle meaning proceed but pace yourself, and warn meaning advisory. Version 0.2 adds one more, halt, for an agent that is already running. An unknown or unparseable directive is treated as deny, so the parse fails closed.

The authors are explicit that this is the robots.txt analogue for live access and explicitly not a security boundary, and they say so repeatedly. Three adapters were built and validated live: an SSH banner with a PAM session hook on an Ubuntu 22.04 host, a Go wire protocol proxy that injects the sentinel as a notice against PostgreSQL 14 without changing the database configuration, and a validating admission webhook on a MicroK8s v1.32 cluster. Those three are mechanism validation. Every compliance number in the paper is measured over the SSH adapter, on one held constant task, or on a mock tool use harness in the final experiment.

That is a narrower apparatus than a crawl of the open web, and it is worth saying plainly before any figure is quoted. Nobody here measured a crawler fetching a page. What was measured is whether an agent, given a task and a working credential, does what a machine readable request asked. The reason it belongs on a site about generative engine optimization is that the class of control is identical to the one every site owner already deploys, and the measurement is the one nobody performing that deployment gets to run.

Why AI agent compliance was high at the door and low mid task

The paper's organising finding is that the same cooperative request gets very different answers depending on when it arrives. At the access door, before the agent has invested anything in a task, withdrawing is cheap. The first experiment held the task constant, a read only check of free disk space on the root filesystem, and gave the agents a working credential so the only friction was the banner. With the deny signal present and no authorisation framing, GPT-4o recused 5 of 5, GPT-4o-mini 5 of 5, and Claude Code 2 of 2. With the banner suppressed, both API models completed the task 5 of 5, which is what establishes that the banner and not the task drove the behaviour.

Experiment 3 widened that to five models at 20 trials per cell and revised the rate downward. Recusal to deny was 20 of 20 for GPT-4o-mini and for Claude Sonnet 4.5, 15 of 20 for GPT-4o, and 11 of 20 for Gemini 2.5 Flash, pooling to 66 of 80 or 82.5 percent with a confidence interval of 72.4 to 90.1 percent. The open weights Llama-3.3-70B is reported separately and honestly: it registered the banner in only 3 of 20 deny trials, the rest producing degenerate output, so its rate is conditional on the signal being seen and its denominator is not comparable. The no signal control completed the task in 98 of 100 trials.

Mid flight the picture inverts. Experiment 2 gave an agent a multi step server health check, so there was always remaining work when the halt landed, and injected the halt after the first or second command. Across two OpenAI models, two injection timings and two delivery channels, 0 of 40 agents stopped. They ran roughly 2.5 further commands on average after the halt, which is essentially the rest of the health check, and the no halt control completed 20 of 20, confirming that was the task's natural trajectory rather than an effect of the signal.

The paper does not leave that null standing as a general claim, which is the most careful thing in it. Experiment 4 reran the identical apparatus through a vendor neutral provider layer and found Gemini 2.5 Flash stopping 16 of 20, 80 percent with an interval of 56 to 94 percent, while GPT-4o-mini reproduced 0 of 20 on the same rig. Both models completed their no halt controls 10 of 10, so Gemini's stopping was a response to the halt rather than early finishing. Mid task compliance is therefore lower than at the door and, like the door, strongly model dependent.

There is a second result at the door worth holding onto. When the system prompt asserted that the owner had authorised the check, GPT-4o flipped and proceeded 4 of 5. GPT-4o-mini and Claude Code held at 100 percent, and the paper notes that the Claude Code agents weighed the prompt's authorisation claim against the on host banner and treated the server's own notice as the more authoritative of the two. A cooperative signal is overridable by design. Which way a given model resolves the conflict between a prompt and a resource is a model behaviour, and the study is the measurement of it. It is also the same arbitration that decides whether planted instructions win, which is the mechanism behind five attack modes run against AI search agents across 308 cases each: an authorisation claim asserted in text is only as strong as the weight a model gives to text.

Noticing a directive and obeying it are separate problems

The cleanest result in the paper is a comparison most people would expect to go the other way. The mid task halt was delivered through two channels. The in band channel appended the sentinel and its notice to the output of the agent's own command, which is the realistic binding for a server that has no other way to reach a running client. The prompt channel inserted the identical text as a message in the conversation, which is the conventional route used by earlier work.

Channel choice had a large, clean effect on whether the halt was noticed. In band, the agents acknowledged it in 0 of 20 trials. In the prompt, they acknowledged it in 20 of 20. A signal appended to tool output is skimmed past, because the agent is reading that output for the numbers it asked for and treats the rest as noise. That is a finding about agent frameworks rather than about models, and it has an obvious echo for anyone who has watched a machine read a page for one thing and ignore everything around it.

Channel choice had no effect at all on whether the halt was obeyed. Stopped was 0 of 20 in both conditions. In the prompt condition the agents saw the halt every single time, understood it, and finished the job anyway, typically completing the remaining commands, writing the summary, and only then mentioning that a stop had been requested. Noticing was necessary and nowhere near sufficient. The paper attributes the gap to task completion drive and calls it a model training and agent design problem rather than a signalling one.

Because cooperation was unreliable, the authors measured the alternative. They wrapped the agent's tool execution loop in an interceptor that scans every tool result for the halt sentinel and terminates the loop the moment one appears, so the agent never reaches the turn where it would decide whether to continue. Across six models spanning four vendors and three API routes, on a mock tool use task with 300 trials in total, the enforced arm stopped 120 of 120 with zero post halt tool actions and 0 of 60 false trips on benign output. On the identical task the agents' own cooperative stop rate was 28 of 120, or 23 percent, and it was again strongly model dependent: Gemini stopped 20 of 20 while the other five models stopped between 0 and 5 of 20. The cooperative arm ran 195 post halt actions that enforcement reduced to none.

The paper is careful about what that 120 of 120 is. It is deterministic by construction rather than a behavioural finding, since an interceptor that sees the sentinel cannot fail to stop the loop, and it protects nothing against an agent running outside the loop the operator controls. Its measured content is that there were no live failures across six real models' tool calling. The conclusion the authors draw is the one that transfers: reliably stopping a running agent needs enforcement, not a request. On the web the nearest equivalent is a network deciding consequences rather than a file asking for them, which is what happens when ignoring a directive becomes grounds for unverifying a crawler.

Every file you can write is in the same cooperative class

None of this is deployed on a website, and a post that pretended otherwise would be inventing a product. The reason it matters here is that the paper names its own ancestor. The closest conceptual relative of the Recuse Signal is the Robots Exclusion Protocol, introduced by Martijn Koster in 1994 and standardised as RFC 9309 in 2022. That document specifies parsing, error handling, caching and limits, and it introduces no enforcement whatsoever. Its own first section states that the rules are not a form of access authorization.

Everything a site owner has been given since sits in the same class. The Content-signal line asks while the Disallow lines under it block. RSL writes a licence into robots.txt and states in its own specification that it changes no access permissions. The W3C reservation protocol is named in none of the three crawler documentation pages that name robots.txt. The IETF preferences work is a vocabulary you cannot yet deploy against any crawler that has committed to reading it. A separate January 2026 proposal on arXiv would add agent-permissions.json, a robots.txt style manifest in which a site lists the interactions it allows. Each is a request. Not one of them is a lock.

The measured evidence that this distinction is real, rather than pedantic, keeps accumulating from different directions. A canary token study found that 12 of 18 AI chatbots still returned content after a robots.txt disallowed every bot, often because the content had reached them through a search engine's crawler rather than any AI vendor's own. A honeysite study found robots.txt stopping exactly one of twelve automated tools, and the one it stopped was the tool whose framework obeys robots.txt by default. Server side, 234 of 592 sites that ban GPTBot in robots.txt served it a 200 anyway, which is the same failure seen from the other end of the wire.

What the Recuse paper adds is the variable those studies could not isolate. In the field you cannot tell a client that ignored your file from a client that never read it, from a client that read it and was overridden by something else. Here the file, the task, the credential and the connection were all held constant, and the only thing that changed was which model was driving. Compliance still ran from 100 percent to 55 percent at the door and from 80 percent to zero mid task. That is as direct a demonstration as the literature currently has that a user agent is a claim, not an identity, and that a directive is an input to a decision made somewhere you cannot see.

There is one genuinely encouraging result in the set, and it deserves as much weight as the nulls. Agents did not treat the sentinel as a blanket stop order. Among the 176 throttle and warn trials in which the banner was actually seen, agents recused 0 times, with an exact interval of 0 to 2.1 percent. GPT-4o-mini and Claude Sonnet 4.5 recused 20 of 20 under deny and proceeded 20 of 20 under both permissive directives. Graded machine readable directives were read as graded, which is the precondition for any of this class of standard being worth writing at all.

What you can check on your own site, and what nobody can check

The honest reading of this study is uncomfortable for anyone selling confidence about crawler behaviour, including us. If four frontier models handed the same unambiguous machine readable request produce compliance rates of 100, 100, 75 and 55 percent, then no publication schedule of directives gets you a number you can rely on for a client you have not tested. The rate is a property of the client and it changes when the client's vendor ships a new build. It is not observable from your side of the connection.

That is a reason to be precise about what is checkable rather than a reason to stop checking. Two things about your own site are fully determined by you and verifiable today. The first is what your rules actually say, per named crawler, which is a parsing question and has an exact answer: our robots.txt tester resolves a path against the groups in your file for each crawler token, and the AI crawler reference lists the tokens that exist to be named. Getting that wrong is common and silent, and it is entirely your side of the wire. The second is what a client receives when it does fetch, which is why what GPTBot sees fetches your page without a browser and shows you the bytes that came back.

Those two questions are the two layers that decide whether AI can read your site, and the study is a reminder about the order they come in. The first layer is a request whose outcome you do not control. The second is a fact about your own server. A page whose text is present in the served HTML has an answer to the second question that does not depend on anyone's goodwill, which is the whole argument for measuring prose parity before anything else, and it is what our methodology describes in full.

There is a timing point here too, and it is one we have written about separately. Even a client that intends to comply is working from a copy of your file, not from your file. A controlled experiment found that fewer than 40 percent of AI assistant bots re-fetched robots.txt inside a 168 hour window, so an edit does not take effect when you save it. Compliance rates measured on a live connection, as in this study, are the ceiling for the file based case rather than a comparable figure, because the file case adds a cache on top of everything the model decides.

The residual value of a cooperative signal is real, and the paper argues for it more carefully than most advocacy does. A published directive is cheap, auditable, attributable, and taken up by a substantial majority of agents at the moment they arrive. It gives you a record of what you asked and when. What it does not give you is a stop. If you need something to actually not happen, the paper's own conclusion is that it has to be enforced at a layer the client does not get a vote in, which on the web means the response your origin or your edge sends rather than the text file it also serves. We hold our own scanner to that distinction rather than to a promise: the LantadBot page prints every request ceiling, spacing rule and retention window the scanner enforces straight from the configuration it runs, so the page changes when the code does. The same preference for observation over assurance is why the crawlability study reports what sampled sites actually serve rather than what crawlers say they will do.

One last statement of scope, because this site owes it on every post of this kind. Lantad ran none of these experiments. We operate no LLM agents, have not reproduced a single trial, and hold no measurement of our own about how any agent responds to any directive, in band or in a file. Every figure above is from arXiv:2606.06460v4 and from RFC 9309, both read on 9 August 2026, and the apparatus was SSH, PostgreSQL and Kubernetes rather than HTTP. Whether these compliance rates transfer to a crawler fetching a web page is exactly the question the study does not answer, and neither do we. What transfers without any assumption at all is the structural point: the file is your decision, and the compliance is somebody else's.