BlogFindings
Crawlable URL structure: Google clarified hyphens over underscores, and 2,203 of 80,660 corpus links used an underscore
Google's Search Central changelog entry of 23 October 2026 records that it updated the URL structure best practices to clarify word separation guidelines using hyphens versus underscores. Lantad resolved every same-host link on the home page of 996 corpus sites on 25 October 2026 and found 2,203 of the 80,660 distinct URLs carried an underscore in the path, against 55,220 that carried a hyphen.
So we counted. This post reports what Google's page now says, with its dates, and then reports a measurement Lantad ran on 25 October 2026 over the home page links of its crawlability corpus. The short answer is that the thing Google chose to clarify is the thing the corpus already gets overwhelmingly right, and that three other practices on the very same page, none of which got a clarification, are broken far more often. That is not a criticism of the guidance. It is an argument about where a site owner's attention is worth spending.
In short
- Google's Search Central changelog carries an entry dated 23 October 2026 titled Updated guidance on crawlable URL structures, whose stated change is to clarify word separation guidelines using hyphens versus underscores, and whose stated reason is to synchronize the documentation with the episodes of a video series; the page it refers to carries Last updated 2026-10-21 UTC.
- Lantad requested robots.txt and then the home page of all 1,419 hostnames in this repository's two committed corpus seed files on 25 October 2026 as LantadBot with no JavaScript executed, resolved every link in the returned HTML, and found 80,660 distinct same-host URLs across the 996 home pages that offered at least one.
- Of those 80,660 URLs, 2,203 carried an underscore in a path segment and 55,220 carried a hyphen, so the separator Google recommends outnumbered the one it advises against by about 25 to 1; 168 sites served at least one underscore URL and only 10 of them used underscores with no hyphen anywhere on the page.
- The practices on the same Google page that no video series was made about are the ones this corpus breaks more often: 4,797 URLs on 224 sites carried an uppercase letter in a path that both Google and RFC 3986 treat as case sensitive, 1,703 URLs on 94 sites ended in a bare number rather than readable words, and 133 URLs on 67 sites were addressed only by a query string.
- Lantad measured URL shape on home page links only, not citations: the composite score this scanner ships weights parity at 0.5, access at 0.25, structure at 0.15 and schema at 0.1, and URL wording is not one of those four components, so nothing in this post says a hyphen earns an AI citation.
| Documented practice | What Google's page says | What 80,660 corpus URLs did, 25 October 2026 |
|---|---|---|
| Word separation | Recommends hyphens instead of underscores, because hyphens clearly signal a word boundary | 2,203 used an underscore, 55,220 used a hyphen |
| Readable words | When possible, use readable words rather than long ID numbers in your URLs | 1,703 ended in a bare number, 88 in a long hex string |
| Case | Google Search's URL handling is case sensitive, like any other HTTP client following IETF STD 66 | 4,797 carried an uppercase letter in the path |
| Parameters | Whenever possible, shorten URLs by trimming unnecessary parameters | 5,445 carried a query string, 133 were addressed only by one |
| Fragments | Do not use fragments to change the content of a page | 14 href values on 7 sites routed through a fragment |
What is a crawlable URL structure, and what did Google change on 23 October 2026?
A crawlable URL structure, in Google's own framing, is a set of properties that let a crawler fetch a page and let a reader guess what is on it before clicking. Google's latest documentation updates page carries an entry dated 23 October 2026 under the heading Updated guidance on crawlable URL structures. Its What field reads: updated the URL structure best practices to clarify word separation guidelines using hyphens versus underscores. Its Why field reads: to synchronize the documentation with the episodes of The Non-Canonical Cabinet video series.
Two dates are worth separating, because they do not agree and the difference is the kind of thing this blog says out loud rather than smoothing over. The changelog entry is dated 23 October. The page it points at carries Last updated 2026-10-21 UTC, read on 25 October 2026. Both are Google's own timestamps on Google's own pages, so the honest reading is that the page was edited on the 21st and the changelog recorded it on the 23rd, and this post quotes each date to the page that carries it.
What the page recommends has not moved. It says it recommends separating words in your URLs when possible, and specifically recommends using hyphens instead of underscores, because underscores are commonly used for denoting concepts that should be kept together, for example by various programming languages to name functions, whereas hyphens clearly signal a word boundary. Its not-recommended example is a path with underscores in both the directory and a query parameter, and its other not-recommended example is two words joined with no separator at all.
Five further practices sit on the same page and none of them received a clarification. It says to use readable words rather than long ID numbers. It says to use words in the audience's language, transliterated where applicable. It says Google Search supports URLs as defined by IETF STD 66, that reserved characters must be percent encoded and that characters in the non-ASCII range should be percent encoded. It says to shorten URLs by trimming parameters that do not change the content. And it says not to use fragments to change the content of a page, because Google Search generally does not support URL fragments.
None of that is new, and none of it is about AI. It is Google Search guidance, published by Google, for Googlebot. The reason it belongs on this blog is that URL shape is one of the few things any crawler reads before it spends a request, and what an AI crawler can reach is the thing this scanner exists to measure. So the useful contribution here is not a restatement of the advice. It is a count.
Reading the source
- GET developers.google.com/search/updates entry dated 23 October 2026
- entry title Updated guidance on crawlable URL structures
- entry What field clarify word separation guidelines using hyphens versus underscores
- entry Why field synchronize with the episodes of a video series
- GET developers.google.com/search/docs/crawling-indexing/url-structure Last updated 2026-10-21 UTC
- practices documented on that page separation, readable words, language, STD 66 encoding, parameters, fragments, case
Hyphens against underscores: what 80,660 corpus URLs actually use
Lantad requested /robots.txt once from each of the 1,419 hostnames in this repository's two committed corpus seed files on 25 October 2026, as LantadBot/1.0, redirects followed, from one network location, with no JavaScript executed and no browser involved. 1,071 answered with a text file, 36 answered with an HTML page rather than robots text, 266 answered a non-200 status and 46 failed in transport. The file was parsed with the parser this product ships and the site root was evaluated for our own token, which closed the home page to LantadBot on 51 hosts; those were not requested, which is the same rule our scanner applies to a paying scan. Be precise about what closed them, because the two causes are not the same thing: 11 were closed by a disallow rule in a file that parsed, and the other 40 by a 5xx response to the robots.txt request, which RFC 9309 tells a crawler to read as complete disallow. A 404 on robots.txt allows every crawler and a 503 blocks them all, so 40 of those 51 sites never refused anything; their file was simply unreachable.
Of the hosts the files permitted, 1,073 returned an HTML home page. Every anchor href in that HTML was resolved against the final response URL, which gave 175,640 href attributes, 166,206 that resolved to an http or https URL, and 140,278 that stayed on the same host. Collapsed to distinct path plus query, that is 80,660 URLs across the 996 home pages that offered at least one, a median of 55 per page and a maximum of 1,248. This is the same method behind an earlier count of the links a home page actually offers, and it measures the paths a crawler meets first rather than a whole site.
Against those 80,660 URLs: 2,203 carried an underscore in at least one path segment, which is 2.73 percent, and 55,220 carried a hyphen, which is 68.46 percent. By site, 168 of the 996 served at least one underscore URL and 938 served at least one hyphen URL. 158 sites served both, which is what you would expect from a site that has changed its URL conventions once and kept the old paths alive. Only 10 sites used underscores with no hyphen anywhere on the page, and on 24 sites the underscore URLs outnumbered the hyphen URLs. 48 sites used neither separator, and those are small pages rather than tidy ones: their median home page offered 4 distinct same-host links against 55 across the corpus, so there was little for a separator to separate.
That ratio is the finding, and it is inconvenient for the news hook that produced this post. The separator Google recommends outnumbers the one it advises against by about 25 to 1 in link volume, and the sites carrying any underscore at all are about one in six. A clarification was published, and the corpus had already converged on the recommendation years before it arrived. A reader looking for the highest-value change to their own URLs will not find it here, and saying so is cheaper than implying otherwise.
One thing this does not establish, and the distinction matters for anybody reading this as generative engine optimization advice. Google's stated reason for preferring hyphens is that they signal a word boundary, which is a claim about parsing a string. It is not a claim that changing a URL moves a ranking, and no measurement in this post observed any engine's behaviour at all. Our crawlability study measures what a crawler can read; it does not measure what an answer engine chooses.
Which stacks still ship underscores?
The corpus is stratified, which turns a flat percentage into something a reader can act on. The platform seed file holds 392 hostnames in ten strata chosen by the technology a site is built on, and the industry seed file holds 1,027 in eight strata chosen by sector. Splitting the 2,203 underscore URLs across those eighteen strata puts almost all of the remaining defect in a small number of places.
Three strata produced no underscore URL at all. The Framer stratum served 496 distinct URLs with none, the Webflow stratum served 1,436 with none, and the static documentation stratum served 426 with none. That is what a generated URL looks like: the path comes from a slug field the platform controls, and the platform picked hyphens. Readers on those stacks can stop reading this section, and the practical guides at our Framer notes and our Shopify notes have nothing to add about separators because there is nothing to fix.
The no-code stratum is the opposite case and by a wide margin the worst rate measured. 143 of its 607 distinct URLs carried an underscore, which is 23.56 percent, concentrated on 4 of its 28 sites. That stratum is the same one where 10 of 42 home pages served a crawler no words at all, so a URL convention is the smaller of its two problems, and the ordering matters: a path nobody can read is a cosmetic issue next to a page with no prose in it.
After that it is sectors rather than stacks. Government sites produced 405 underscore URLs across 6,645, which is 6.09 percent on 19 of 82 sites. Ecommerce produced 480 across 10,399, which is 4.62 percent but on 18 of 52 sites, the highest site-level incidence in the corpus. News produced 327 across 9,838. Healthcare, finance, education and travel all landed between 1.5 and 2.8 percent. The pattern is legible and it is not about taste: these are sectors running long-lived systems where the URL was minted by a content management system or a case management application a decade ago, and the hand-authored strata built on modern site builders do not have the problem because they never had the chance to acquire it.
Two single sites are worth naming because they show the shape of the concentration rather than a trend. On 25 October 2026 zooplus.de served 296 underscore URLs out of 435 distinct links on its home page, and ansa.it served 283 out of 396. In both cases the underscore is the site's convention rather than an exception inside it, which is the only version of this defect that would be worth a migration, and a migration is the one fix on this page that carries real risk: changing a URL means a redirect, and 305 of 313 home pages already answer at more than one live URL without trying.
| Stratum | Distinct URLs | With an underscore | Share | Sites with one or more |
|---|---|---|---|---|
| bubble-nocode | 607 | 143 | 23.56% | 4 of 28 |
| government | 6,645 | 405 | 6.09% | 19 of 82 |
| ecommerce | 10,399 | 480 | 4.62% | 18 of 52 |
| news | 9,838 | 327 | 3.32% | 8 of 59 |
| healthcare | 9,042 | 251 | 2.78% | 22 of 87 |
| finance | 9,070 | 238 | 2.62% | 27 of 85 |
| education | 8,895 | 176 | 1.98% | 19 of 94 |
| saas | 9,161 | 34 | 0.37% | 13 of 115 |
| framer | 496 | 0 | 0.00% | 0 of 30 |
| webflow | 1,436 | 0 | 0.00% | 0 of 40 |
| static-docs | 426 | 0 | 0.00% | 0 of 30 |
ID numbers, uppercase paths and query-only URLs are the larger defects
The same Google page carries four other practices, and the corpus breaks three of them more often than it breaks the separator rule. None of the three received a clarification or a video episode.
Case is the largest. 4,797 URLs on 224 of the 996 sites carried an uppercase letter somewhere in the path, which is 5.95 percent of the corpus, more than double the underscore rate. Google's page states that, like any other HTTP client following IETF STD 66, Google Search's URL handling is case sensitive. RFC 3986, which is STD 66, says the same thing more precisely in its normalization section: the scheme and host are case-insensitive and should be normalized to lowercase, and the other generic syntax components are assumed to be case-sensitive unless a scheme specifically defines otherwise. The path is one of those other components. Mixed case is not itself an error, and a site that serves one canonical casing consistently has no problem. The failure mode is a site that serves both, which is a duplicate rather than a defect in the string, and it is the reason this corpus already carries measurements of robots.txt matching under case and of pages that declare no canonical at all. Ecommerce carried 1,378 of the 4,797 and government 842.
Opaque paths come next. 1,703 URLs on 94 sites ended in a bare number after stripping a file extension, and 88 on 8 sites ended in a hexadecimal string of twelve characters or more. Google's wording on this is to use readable words rather than long ID numbers when possible. The concentration is extreme: theregister.com served 358 numeric-ending URLs out of 489 distinct links, rwth-aachen.de 223 of 232, and dollargeneral.com 162 of 250. By stratum, news produced 584 and education 361, while finance produced 2 across 9,070 URLs. State the heuristic plainly, because it is a heuristic: this test asks whether the final path segment is all digits, so a path ending in a year is counted alongside a path ending in a database key, and the count is an upper bound on genuinely opaque URLs rather than a census of them.
Query-only addressing is the smallest count and the clearest defect. 5,445 URLs carried a query string, which is ordinary, but 133 URLs on 67 sites placed the entire address in the query with the path left as the site root or an index file. Google's not-recommended example for readable URLs is exactly that shape. A page reachable only as a parameter on the root is a page whose identity depends on a string the server may reorder, drop or ignore, and when one of those is lost in a redirect the request lands on the home page with a 200, which is the mechanism behind soft 404 responses on 125 of 1,094 sites. Fourteen href values on 7 sites went further and routed entirely through a fragment, the one practice Google's page states flatly that Search generally does not support.
Finally, a practice this run could not test honestly. Google says non-ASCII characters should be percent encoded in the href. 419 URLs on 77 sites carried a percent-encoded character after resolution, but the resolver normalizes encoding as it parses, so this run cannot separate an href that arrived already encoded from one encoded on our side. That is a gap in the measurement rather than a finding about the sites, and the only honest thing to do with it is name it. The nearest real measurement this corpus holds is on the other side of the same question: 22 robots.txt rules carried a raw non-ASCII path and the shipped matcher bound none of them in encoded form.
| Practice | URLs | Sites | Heaviest stratum | Measurement note |
|---|---|---|---|---|
| Uppercase in a case-sensitive path | 4,797 | 224 | ecommerce, 1,378 | Not an error alone; a defect when both casings answer |
| Path ends in a bare number | 1,703 | 94 | news, 584 | All-digit final segment, so a year counts too |
| Path ends in a long hex string | 88 | 8 | ecommerce, 54 | Twelve or more hex characters in the final segment |
| Carried a query string | 5,445 | 382 | ecommerce, 1,330 | Ordinary; counted to frame the next row |
| Addressed only by a query string | 133 | 67 | healthcare, 23 | Path was the root or an index file |
| Percent-encoded character present | 419 | 77 | not reported | Resolver normalizes encoding, so raw form was not measured |
What this measurement does not say about AI visibility
This post is a count of strings, and the distance between a count of strings and an AI citation is the part a measurement blog has to state rather than imply.
Start with what the scanner itself believes. The composite score this product ships weights four components: parity at 0.5, access at 0.25, structure at 0.15 and schema at 0.1. Those are settings somebody chose, not findings, and they are quoted here as settings. URL wording is not one of the four. A site could migrate every underscore in its corpus and move its AI visibility grade by nothing at all, because what the grade reads is whether the words on the page reach a crawler, which is prose parity, and whether the crawler is allowed in, which is access. What GPTBot actually sees on a page is a different question from what its URL spells.
Next, what was and was not measured. Lantad made at most two requests per hostname on 25 October 2026, both from one network location, with no JavaScript executed. The unit is the home page, so the median site contributed 55 URLs and no site contributed its whole URL space; a crawler that follows links for days would meet paths this run never saw. No engine was asked anything, no citation was observed, and no ranking was checked, so no sentence here supports a claim that a URL convention changes an answer. Sites that refused our agent were not measured either: 51 hosts were closed to LantadBot at the root, 11 by a written rule and 40 by a 5xx on robots.txt, and 248 answered the home page with something other than HTML; all of those are absent from every figure above rather than counted as compliant. Our agent documentation and our methodology say what we send and what we refuse to infer.
Then, the standing caveat on this corpus. The 1,419 hostnames are two committed sampling frames built for a crawlability study, stratified by platform and by sector, not a random draw from the web. Every proportion in this post describes those frames on that date. The stratum figures in particular rest on small denominators once a stratum is split by site, and the three zero-underscore strata hold 30, 40 and 30 sites respectively, which is enough to say a convention holds there and not enough to call it universal.
Last, the thing worth taking away, which is an ordering rather than a number. Google published a clarification about the smallest of its URL practices, and the corpus was already at 25 to 1 in favour of the recommendation. On the same page, the practices nobody clarified are broken two to three times as often. If a reader wants a priority list for their own site from this measurement, it runs: make sure one casing answers, give pages readable paths rather than keys, stop addressing content only by parameter, and treat separators as a question for new URLs rather than a reason to migrate old ones. For whether an answer engine can read the page those URLs point at, the per-crawler access check, the robots.txt tester and the guidance on getting cited in Google AI Overviews are the places that question is actually settled.
- Distinct same-host URLs resolved from 996 home pages 80,660 on 25 October 2026, median 55 per page, maximum 1,248
- Underscore and hyphen counts in path segments 2,203 against 55,220, across 168 and 938 sites
- Case, ID shape, query and fragment properties 4,797 uppercase, 1,703 numeric-ending, 133 query-only, 14 fragment-routed
- Raw percent-encoding as written in the href The resolver normalizes encoding, so the raw form was not observed
- Joined words with no separator at all Google's greendress case needs a dictionary this run did not use
- Whether any engine cited any of these pages No engine was asked and no citation was observed
- URL wording in the shipped composite score 0 pts The four weighted components are parity 0.5, access 0.25, structure 0.15, schema 0.1
Lantad
Published .
Google's documentation on URL structure has said for years that hyphens are preferable to underscores. On 23 October 2026 its changelog recorded an update to that page, and the stated purpose of the update was to clarify the same advice rather than to change it. That is an unusually small documentation event, and it raises a question worth answering with a count instead of an opinion: on real sites, is this still a defect anybody has?
Common questions
What did Google change about crawlable URL structure in October 2026?
Its Search Central changelog carries an entry dated 23 October 2026 titled Updated guidance on crawlable URL structures. The stated change is to clarify word separation guidelines using hyphens versus underscores, and the stated reason is to synchronize the documentation with a video series. The page it refers to carries Last updated 2026-10-21 UTC and its recommendation, hyphens rather than underscores, is not new.
Do underscores in a URL hurt AI visibility?
Nothing in this measurement supports that claim. Google's stated reason for preferring hyphens is that they signal a word boundary when a string is parsed, which is Google Search guidance, and Lantad observed no engine behaviour and no citation. The score this scanner ships weights parity, access, structure and schema, and URL wording is not one of those four components.
How common are underscores in real site URLs?
Across 80,660 distinct same-host URLs resolved from 996 corpus home pages on 25 October 2026, 2,203 carried an underscore in a path segment and 55,220 carried a hyphen. 168 of the 996 sites served at least one underscore URL, 10 used underscores with no hyphen anywhere, and the Framer, Webflow and static documentation strata served none at all.
Which URL problem should a site fix first?
On this corpus, case. 4,797 URLs on 224 sites carried an uppercase letter in a path that Google and RFC 3986 both treat as case sensitive, which is more than double the underscore rate, and the failure it leads to is two URLs answering for one page. Opaque numeric paths on 94 sites and query-only addressing on 67 sites both outrank separators as well.
See what AI can read on your site
Run a free scan and get a graded report of exactly what AI crawlers can and cannot read, with ranked fixes.
AI crawler
An AI crawler is an automated bot, such as GPTBot, ClaudeBot, or PerplexityBot, that fetches web pages so an AI system can read, train on, or cite what those pages contain.
AI visibility
AI visibility is whether AI systems can access, read, and faithfully represent your web pages, the prerequisite for being cited in an AI answer.
Generative Engine Optimization (GEO)
Generative Engine Optimization (GEO) is the practice of structuring and publishing content so that generative AI engines can fetch it, read it in full, and cite it when they answer a question.
Scan a website for all pages: 985 links found, 200,712 URLs declared
Six pages captured on 15 July 2026 exposed 985 unique internal paths between them. On 29 August 2026 all six of those sites named a sitemap in robots.txt, five of the named sitemaps resolved, every one of the five was an index rather than a page list, and together they declared 200,712 URLs.
Duplicate content and AI crawlers: 305 of 313 home pages answered at more than one live URL
Lantad requested robots.txt from all 1,419 hostnames in this repository's two committed corpus seed files on 15 October 2026, then asked the 313 home pages that answer at their own apex for four more URL forms of the same page. 305 of the 313 served the same home page at two or more live URLs, 45 of those 305 declared no canonical at all, and the documentation of three AI crawler vendors never uses the word canonical.
Internal links and AI crawlers: 84 of 1,091 home pages offered a crawler no path at all
Lantad requested the home page of all 1,419 hostnames in this repository's committed corpus on 21 September 2026 and read the raw bytes with no JavaScript executed. 1,091 answered HTTP 200 with HTML. 84 of those carried no internal link a crawler could follow, and 55 of the 84 contained no anchor element at all. On the 1,007 that did offer a path, the first one was requested too.
