# How to get cited by Claude: three named bots, and one IP list that cannot tell them apart

> Anthropic's help centre article of 7 April 2026 names ClaudeBot, Claude-User and Claude-SearchBot, and says that disabling either of the last two may reduce your site's visibility. Lantad fetched the IP range file that page points at on 7 September 2026. It holds 26 prefixes covering 1,094 IPv4 addresses, and it is a single list for all three bots.

- Canonical page: https://lantad.co/blog/how-to-get-cited-by-claude-one-ip-list-three-bots
- This file: https://lantad.co/blog/how-to-get-cited-by-claude-one-ip-list-three-bots.md
- Last substantive update: 2026-09-07

## Key facts

- **Published:** 2026-09-07
- **Category:** Findings
- **Author:** Lantad
- **Length:** 3351 words
- **Takeaway 1:** Anthropic's crawler article, dated 7 April 2026 and read on 7 September 2026, states that disabling Claude-User prevents our system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search, and that disabling Claude-SearchBot prevents our system from indexing your content for search optimization.
- **Takeaway 2:** How to get cited by Claude starts with that asymmetry: the same article's consequence for restricting ClaudeBot names only training datasets and no visibility loss, so Anthropic itself writes the block training and allow search instruction that most crawler advice repeats.
- **Takeaway 3:** Lantad fetched the IP range file Anthropic's article links, at claude.com/crawling/bots.json, on 7 September 2026. It carried two keys, creationTime and prefixes, a creationTime of 2026-08-18T23:56:36Z, and 26 IPv4 prefixes covering 1,094 addresses, with no IPv6 prefix and no per-bot structure of any kind.
- **Takeaway 4:** OpenAI published four separate files on 7 September 2026, being gptbot.json with 21 prefixes, searchbot.json with 35, chatgpt-user.json with 207 and adsbot.json with 2, and Perplexity two, so on those two vendors an IP check names the crawler and on Anthropic's single file it names only the company.
- **Takeaway 5:** Anthropic's own article tells site owners not to use the list that way: alternate methods like blocking IP address(es) from which Anthropic Bots operates may not work correctly or persistently guarantee an opt-out, as doing so impedes our ability to read your robots.txt file.

## Summary

Anthropic runs three named crawlers, and its help centre publishes a table saying what each one is for and what happens when a site turns it off. That last column is the part worth reading for anyone asking how to get cited by Claude, because two of the three carry a stated visibility cost and the third does not. [Our platform page for Claude](https://lantad.co/how-to-get-cited/claude) lists the tokens and says they honour robots.txt. It does not quote what Anthropic says blocking each one costs, and it says nothing about the file sitting underneath the whole arrangement.

So we read both. The article carries the date 7 April 2026 and we read it on 7 September 2026, along with the IP range file it points at, which Anthropic names as the way to tell a genuine Anthropic crawler from a request that merely claims to be one. That file holds 26 prefixes, all IPv4, covering 1,094 addresses, and it is one list for all three bots. Nothing in it says which bot an address belongs to, and the two more explicit vendors in the same category publish a file per crawler instead.

This post is a reading of a vendor page and of published files, and nothing more. No Anthropic crawler was observed visiting anything, no page was scored, and no answer engine was asked a question, so every figure below is a property of a document we downloaded rather than of traffic that arrived. What [an AI crawler](https://lantad.co/glossary/ai-crawler) is, and why the token in a robots.txt file is not the same object as the client that shows up, is the background this rests on.

## How to get cited by Claude, and which of the three bots decides it

[Anthropic's crawler article](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) is unusual among crawler documentation in that it prints a consequence column. Most vendor pages name a token and describe what it collects. This one adds a third column headed with what happens when you disable it, and the three answers are not the same shape.

For ClaudeBot the consequence is about training and nothing else: when a site restricts ClaudeBot access, it signals that the site's future materials should be excluded from our AI model training datasets. No visibility is mentioned, because none is claimed. For Claude-User the sentence changes: disabling it prevents our system from retrieving your content in response to a user query, which may reduce your site's visibility for user-directed web search. For Claude-SearchBot it changes again in the same direction: disabling it prevents our system from indexing your content for search optimization, which may reduce your site's visibility and accuracy in user search results.

The page frames all three under four stated principles: that its collection of data should be transparent, that its crawling should not be intrusive or disruptive, that its bots respect do not crawl signals by honoring industry standard directives in robots.txt, and that they respect anti-circumvention technologies. Each of those is a commitment about conduct, and only the third is a thing a site can write down and check.

That is the answer to the question in the heading, written by the vendor rather than inferred from behaviour. Two of the three bots are named as paths to being seen, and the one that most site owners actually mean when they say they want to block AI training is not one of them. The standard advice in this category is to disallow the training crawler and allow the search crawler, and that instruction is only writable when a vendor publishes both names. We counted that once and found [six of nine vendors publish exactly one token](https://lantad.co/blog/six-vendors-publish-one-crawler-token), which makes Anthropic one of the three where the advice can be followed literally.

Our own registry carries a fourth Anthropic entry, the legacy token anthropic-ai, classed as training with no request user agent at all. It is not named anywhere on the April 2026 article, which describes three robots and calls them the three robots that Anthropic uses. A token like that is invisible in a log file by construction, which is a category we have written about before: [three AI crawler tokens that will never appear in your logs](https://lantad.co/blog/the-crawler-tokens-that-never-appear-in-your-logs) covers what a zero next to such a name does and does not mean. Keeping it in the registry is a decision to keep evaluating a rule that a site may still have written years ago, not a claim that Anthropic still ships it.

## One IP list, 26 prefixes, and no way to tell which bot arrived

The article ends by pointing at a list: if a crawler has a source IP address on this list, it indicates that the crawler is coming from Anthropic. The link goes to claude.com/crawling/bots.json, written here as plain text because that host is not registered for outbound links on this site. We fetched it on 7 September 2026 with this scanner's own user agent, following redirects, and it answered HTTP 200 with 1,162 bytes of JSON.

The document has two keys, creationTime and prefixes. The creationTime read 2026-08-18T23:56:36Z, which is 20 days before the day we fetched it. The prefixes array held 26 entries, every one of them an ipv4Prefix and none of them an ipv6Prefix: one /22, three /28 blocks and twenty-two single /32 addresses, covering 1,094 addresses in total. The response carried cache-control public, max-age=3600, and it carried neither an ETag nor a Last-Modified header, so a conditional request cannot ask this file whether it has changed since the last read. That is the same gap we found in two of the thirteen files in an earlier survey, where [crawler IP range files carried two dates that disagree](https://lantad.co/blog/crawler-ip-files-carry-two-dates-that-disagree) and two vendors offered no header to compare at all.

What the file does not contain is any division by bot. There is no per-crawler key, no label on a prefix, and no second file that we could find: we requested claudebot.json, claude-user.json and claude-searchbot.json under the same directory on 7 September 2026 and all three returned 404, which establishes that those three paths are absent rather than that no such file exists anywhere.

Set that against the two vendors whose files we read on the same day. OpenAI publishes four, named after four crawlers in [its own bot documentation](https://developers.openai.com/api/docs/bots): gptbot.json returned 21 prefixes with a creationTime of 2025-10-30, searchbot.json returned 35 with 2026-01-02, chatgpt-user.json returned 207 with 2026-09-04, and adsbot.json returned 2 with 2026-05-12. Perplexity publishes two, described in [its bots guide](https://docs.perplexity.ai/guides/bots), and they returned 8 and 4 prefixes with creationTimes of 2025-02-07 and 2025-10-17. We counted OpenAI's the first time in [four OpenAI crawler IP lists, opened and counted](https://lantad.co/blog/openai-crawler-ip-ranges-counted). On those two vendors, an address answers the question of which crawler. On Anthropic's, it answers only which company.

## Anthropic says not to block by IP, and gives the reason

The same article that publishes the list warns against using it as an enforcement tool. Opting out of being crawled by Anthropic Bots requires modifying the robots.txt file, it says, and alternate methods like blocking IP address(es) from which Anthropic Bots operates may not work correctly or persistently guarantee an opt-out, as doing so impedes our ability to read your robots.txt file.

The mechanism behind that sentence is worth spelling out, because it is the reason the coarse control destroys the fine one. A firewall rule that drops Anthropic's ranges drops every request from them, and the first request any well behaved crawler makes is for robots.txt. Block the ranges and the file that carries your actual preference is never fetched, so the per bot distinction the article spent three rows explaining stops existing for your site. You have replaced a statement that says allow one bot and refuse another with silence, and silence is not a directive. The [robots.txt tester](https://lantad.co/tools/robots-txt-tester) exists to answer the narrower question of what a given file says to a given token, which is the layer where this choice is actually expressed.

Two smaller details on the same page bear on how that file should be written. Anthropic supports the non-standard Crawl-delay extension to robots.txt, and calls it non-standard in its own words, which is accurate: the directive is absent from [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309.html), the standard that defines the robots exclusion protocol, and support for it is a per vendor promise rather than a guarantee. That gap has consequences elsewhere, and we wrote one of them up in [a crawl-delay directive that can unverify a crawler](https://lantad.co/blog/crawl-delay-can-unverify-a-crawler).

The other detail is the instruction to repeat the block on every subdomain you wish to opt out from. That is the protocol working as designed rather than an Anthropic quirk, because robots.txt is scoped to one authority and a rule on the apex governs nothing on a sibling host. We measured what that looks like in practice when [three of six sites served a different robots.txt on their second hostname](https://lantad.co/blog/check-for-robots-txt-fourteen-hostnames-eight-files). Anthropic also states it respects anti-circumvention technologies and will not attempt to bypass CAPTCHAs for the sites it crawls, which is a commitment about conduct and not something a site can verify from outside.

## The user agent is the only separator, and it is a claim

Put the two halves together and a gap appears. The IP list resolves the company: an address on it indicates the request came from Anthropic. The robots.txt file resolves the site's preference, and that preference is written per token. But nothing published connects the two, so the only thing on the wire that says which of the three bots is asking is the user agent string, and a user agent string is typed by whoever sends the request. We have argued the general form of that before in [a user agent is a claim, not an identity](https://lantad.co/blog/a-user-agent-is-a-claim-not-an-identity), and Anthropic's arrangement is a clean instance of it: verification is possible at the company level and unavailable at the bot level.

Our own registry makes that concrete in an uncomfortable way. It carries request user agent strings for ClaudeBot, Claude-SearchBot and Claude-User, and the header comment above them records that those three are the entries that could not be confirmed against vendor documentation, described in our source as widely reported strings we cannot source. Reading the article on 7 September 2026 confirms why. It publishes robots.txt tokens, and two robots.txt examples both written with ClaudeBot, and it prints no full user agent string anywhere on the page. The scale of that across the category is something we counted separately: [6 of 15 crawler tokens publish a User-Agent to match](https://lantad.co/blog/gptbot-detection-six-of-fifteen-tokens-publish-a-ua).

This is why a log line is weaker evidence than it looks. A request whose user agent contains Claude-User and whose source address sits in Anthropic's list is consistent with Claude-User, and it is equally consistent with any other Anthropic client, since one list covers all three. A request carrying the same string from an address outside the list is consistent with an impostor and also with a range Anthropic added after the file's creationTime of 18 August 2026. We wrote up the general problem of what a bot declares against what it proves in [a bot declares three things and proves one](https://lantad.co/blog/llm-crawler-detection-declared-three-verified-one), and the order in which those checks have to run in [AI crawler detection is an ordering problem](https://lantad.co/blog/ai-crawler-detection-is-an-ordering-problem).

For a site owner the practical consequence is small and specific. Write the preference in robots.txt, per token and per subdomain, because that is the only layer where the three bots are separable. Treat the IP list as an audit aid rather than a gate. Our own crawler publishes [its conduct policy and opt-out](https://lantad.co/bot) on the same principle, which is that the checkable thing should be written down rather than asserted.

## What this does not measure

Every figure above is a property of a document, not of traffic. We did not observe an Anthropic crawler fetching anything, we did not measure whether Claude cited any page, and we did not test whether a robots.txt rule naming Claude-User changes what Claude returns. Those are all separate questions, and the honest position on the last of them is that a fetch is a prerequisite for a citation rather than a cause of one. Our [methodology](https://lantad.co/methodology) sets out which parts of that chain this scanner measures, and [what our accumulated scans say](https://lantad.co/research) is the place where measured claims about real sites live.

The classes in our registry are settings, not findings. Calling ClaudeBot training and Claude-SearchBot search is a decision recorded in our own source so the product can evaluate them consistently, and Anthropic's page is where the underlying descriptions come from. If Anthropic changes what a bot does, the class in our file is a stale label until someone updates it, which is exactly the failure mode we described when [compliance records turned out to be falsifiable rather than proof](https://lantad.co/blog/compliance-records-are-falsifiable-not-proof).

Three small facts about the source itself belong in the record. The article's own date is 7 April 2026, so it is the vendor's statement about that day and we have no way to see edits made since. The page does offer a subscription form for notification of substantial changes to it, which is more than most crawler documentation provides and which puts the burden of noticing a change on the reader rather than on a diff nobody runs. It also asks anyone reporting a misbehaving bot to write from an email address on the domain in question, since a report about a site is otherwise hard to verify. And the URL we started from, on support.anthropic.com, answered with a 301 to support.claude.com when we requested it on 7 September 2026, so a bookmark or a robots.txt comment pointing at the old host still resolves, through a redirect, to the page quoted here.

This is the third of these platform readings. The first found that [Perplexity names two crawlers and deprecated an API](https://lantad.co/blog/how-do-i-get-cited-by-perplexity-two-crawlers), and the second found that [zero of 81 home pages blocked their own snippet](https://lantad.co/blog/how-to-get-content-cited-in-google-ai-overviews-zero-nosnippet), which is the single documented gate Google names for AI Overviews. The pattern across all three is that the vendor's own page is shorter and more specific than the advice written about it, and that the part a site actually controls is usually one file. What [AI visibility](https://lantad.co/glossary/ai-visibility) means here is the sum of those small controls being set correctly rather than any one lever, and the list of tokens we evaluate is published on our [AI crawlers reference](https://lantad.co/tools/ai-crawlers).

## Questions and answers

**Which Anthropic bot should I allow to be cited by Claude?**

Anthropic's article of 7 April 2026 attaches a visibility consequence to two of its three bots. Disabling Claude-User prevents retrieval of your content in response to a user query, which it says may reduce your site's visibility for user-directed web search, and disabling Claude-SearchBot prevents indexing your content for search optimization. The consequence it states for restricting ClaudeBot concerns training datasets only.

**Does Anthropic publish a separate IP list for each crawler?**

No. The article links one list, at claude.com/crawling/bots.json, and describes it as showing that a crawler is coming from Anthropic rather than which crawler it is. Fetched on 7 September 2026, that file held two keys and 26 IPv4 prefixes with no per-bot structure. Requests for claudebot.json, claude-user.json and claude-searchbot.json under the same directory all returned 404 on that date.

**Can I block Anthropic's crawlers by IP address instead of robots.txt?**

Anthropic advises against it in the same article, stating that alternate methods like blocking IP address(es) may not work correctly or persistently guarantee an opt-out, as doing so impedes its ability to read your robots.txt file. Blocking the ranges also removes the only layer where the three bots can be treated differently, because that distinction is written per token in robots.txt and nowhere else.

**Does Anthropic publish the user agent strings its bots send?**

Not on the article read on 7 September 2026. That page names the three robots.txt tokens and gives two robots.txt examples, both written with ClaudeBot, and prints no full user agent string. Lantad's registry carries strings for the three bots that our own source records as unconfirmed against vendor documentation, which is the honest status of them until Anthropic publishes the strings itself.

---

Lantad measures whether AI crawlers can actually read a page: it fetches as a non-rendering
crawler, renders as a browser, and reports the gap. Free scan, one URL, no signup.

Method and weights: https://lantad.co/methodology | All pages as markdown: https://lantad.co/md | Crawler policy: https://lantad.co/bot