# A user agent is a claim, not an identity > Every per-crawler number you have ever seen, including ours, counts requests that said they were a crawler. Verifying the claim needs an IP check, and most tools do not do one. - Canonical page: https://lantad.co/blog/a-user-agent-is-a-claim-not-an-identity - This file: https://lantad.co/blog/a-user-agent-is-a-claim-not-an-identity.md - Last substantive update: 2026-07-28 ## Key facts - **Published:** 2026-07-28 - **Category:** Guides - **Author:** Lantad - **Length:** 1008 words - **Takeaway 1:** A user agent is a request header the client chooses, so any program can send the exact string GPTBot uses and the receiving server cannot tell from the header alone. - **Takeaway 2:** Lantad labels every per-crawler figure as requests CLAIMING to be that crawler, in the column heading rather than in a footnote, because a fabricated crawler count is indistinguishable from a real one. - **Takeaway 3:** Verification is possible and costs a lookup: OpenAI, Anthropic and Perplexity each publish the IP ranges their crawlers fetch from, and a request from outside those ranges is not that crawler whatever its header says. - **Takeaway 4:** Cloudflare exposes a verified-bot signal that does this work, but it is gated to paid plans, so a tool cannot assume its users have it. - **Takeaway 5:** The practical consequence for a site owner is that an unverified spike in one crawler is a question rather than a finding. ## Summary The User-Agent header is a string the client picks. There is no signature on it, no registry binding it, and no step in HTTP where anyone checks. Sending one is exactly as hard as typing it. That means every per-crawler chart in every AI visibility tool, this one included, counts requests that SAID they were a given crawler. Most of the time that is fine, because most traffic has no reason to lie. It stops being fine the moment a number is used for a decision, and the honest fix is cheap enough that there is no excuse for not knowing which kind of number you are looking at. ## When the difference actually costs you something For a rough sense of whether AI crawlers reach your site at all, an unverified count is adequate. The failure mode is bounded: you might believe you are crawled slightly more than you are. It matters when the number drives an action. Three cases recur. A sudden spike in one crawler, read as growing interest, when it is a scraper wearing a well-known name to avoid a block. A per-crawler bandwidth bill attributed to a vendor that did not incur it. And a decision to block a crawler based on volume that was never that crawler in the first place, which costs you the real crawler's access while leaving the impostor untouched. The last is the worst, because the block is enforced by the same header the impostor is free to change. ## How verification actually works The check is an IP one. A vendor that wants its crawler trusted publishes the address ranges the crawler fetches from, and a request claiming that crawler from outside those ranges is not that crawler. OpenAI, Anthropic and Perplexity each publish ranges for their fetchers, and the more careful method adds a reverse DNS lookup confirming the address resolves to the vendor domain and forward-resolves back to the same address. Cloudflare does this work and exposes the result as a verified-bot signal, which is the least effort available if you are already behind it. The catch is that the signal is gated to paid plans, so a tool cannot assume its users have it, and a tool that quietly depends on it reports differently for different customers without saying why. None of this is expensive. It is a list of CIDR ranges, refreshed on a schedule, and a containment test per request. ## What Lantad does about it today Lantad does not verify crawler identity yet, and says so on every surface showing a per-crawler figure. The word claimed is in the column heading rather than in a note underneath, because a caveat a reader has to go looking for is a caveat that does not exist. The reason it is not built is ordering rather than difficulty. Verification is a scheduled job pulling published vendor ranges into a lookup structure, plus a containment test at the edge, and it is on the list. Until it ships, the number on the page is a count of claims and the page says the word. The general principle is worth more than the specific feature. When a tool shows a number about something it cannot directly observe, the useful question is not whether the number is large. It is what the tool would have had to do to know, and whether it says whether it did. ## Questions and answers **Can anyone really pretend to be GPTBot?** Yes. The User-Agent header is chosen by the client and no part of HTTP verifies it. Sending the exact string OpenAI publishes takes one line of code, which is why the header alone cannot establish identity. **How do I verify an AI crawler myself?** Check the request IP against the ranges the vendor publishes for that crawler, and optionally confirm reverse DNS resolves to the vendor domain and forward-resolves back. If you are behind Cloudflare on a paid plan, the verified-bot signal does this for you. **Does Lantad verify crawler identity?** Not yet. Every per-crawler figure Lantad shows is labelled as requests claiming that crawler, in the heading rather than in a footnote. Verification is a scheduled range pull plus a containment test at the edge, and it is on the roadmap. **Should I block crawlers that fail verification?** That is a reasonable policy, but note what it does and does not achieve. It stops the impostor using that name, and the impostor's next move is to send a different one. Blocking by identity works best combined with rate limits that apply regardless of the claimed name. --- Lantad measures whether AI crawlers can actually read a page: it fetches as a non-rendering crawler, renders as a browser, and reports the gap. Free scan, one URL, no signup. Method and weights: https://lantad.co/methodology | All pages as markdown: https://lantad.co/md | Crawler policy: https://lantad.co/bot