Home / Markdown index

Every page on this site, in markdown

Prefer one flat file? llms.txt lists every page, and llms-full.txt carries the long form. Crawler policy and the scanner's own user agent are on /bot.

Start here 3

• Fix Sprint and done-for-you AI visibility work

  Paid engagements where Lantad fixes what the scan found, rather than only reporting it.

  /services.md from /services

• Lantad pricing

  The scan is free and needs no account. Paid plans add monitoring, multi-page scanning and prompt tracking.

  /pricing.md from /pricing

• Lantad: see what AI can read on your site

  Lantad fetches your page the way a non-rendering AI crawler does, renders it the way a browser does, and reports the difference. Free, one URL, no signup.

  /index.md from /

How it works 7

• Compare two pages or two points in time

  Puts two reports side by side so a change is attributable rather than remembered.

  /compare.md from /compare

• How the Lantad score is calculated

  The full method: what is fetched, how prose parity is computed, how access is probed per crawler, and every weight in the composite.

  /methodology.md from /methodology

• Lantad API for developers

  Programmatic access to scans and reports, including the JSON a scan returns.

  /developers.md from /developers

• LantadBot: the scanner crawler policy

  What LantadBot fetches, how it identifies itself, and how to block it. It never spoofs another crawler's user agent on the baseline fetch.

  /bot.md from /bot

• Prompt tracking across answer engines

  Tracks a set of prompts over time across several answer engines, so a change in what they say is visible rather than anecdotal.

  /prompts.md from /prompts

• Scan many pages at once

  Runs the same measurement across a set of URLs and reports them together, so a pattern across a site becomes visible.

  /multiscan.md from /multiscan

• What AI says about you

  Asks answer engines real questions about your brand and reports what comes back, including whether you are named at all.

  /what-ai-says.md from /what-ai-says

Free tools 5

• AI crawler reference: every token and what it is for

  The 15 AI crawler tokens Lantad tracks, grouped by purpose: training, search indexing, and fetches triggered by a person asking an assistant.

  /tools/ai-crawlers.md from /tools/ai-crawlers

• Free AI crawler tools

  Four free checks that need no account: what GPTBot sees, a per-crawler robots.txt tester, an llms.txt checker, and the AI crawler reference.

  /tools.md from /tools

• llms.txt checker

  Fetches your /llms.txt and reports whether it exists and how it is structured. Ships beside our own published finding that the evidence for llms.txt is weak.

  /tools/llms-txt.md from /tools/llms-txt

• robots.txt tester for AI crawlers

  Parses a robots.txt and reports, for each of the 15 named AI crawler tokens, whether it is allowed at the site root and which rule decided.

  /tools/robots-txt-tester.md from /tools/robots-txt-tester

• What GPTBot sees on your page

  Fetches your page with GPTBot's published user agent and no JavaScript, then shows the text a non-rendering crawler actually receives.

  /tools/what-gptbot-sees.md from /tools/what-gptbot-sees

Definitions 9

• AI crawler

  An AI crawler is an automated bot, such as GPTBot, ClaudeBot, or PerplexityBot, that fetches web pages so an AI system can read, train on, or cite what those pages contain.

  /glossary/ai-crawler.md from /glossary/ai-crawler

• AI visibility

  AI visibility is whether AI systems can access, read, and faithfully represent your web pages, the prerequisite for being cited in an AI answer.

  /glossary/ai-visibility.md from /glossary/ai-visibility

• Answer Engine Optimization (AEO)

  Answer Engine Optimization (AEO) is the practice of making a page readable, allowed, and structured enough that AI answer engines can use it as a source when they compose an answer.

  /glossary/aeo.md from /glossary/aeo

• Entity confidence

  Entity confidence is a free diagnostic on every graded Lantad report that scores, out of 100, how completely the JSON-LD in your crawler-visible HTML declares one identity and how far that identity is anchored by identifiers which resolve outside the page.

  /glossary/entity-confidence.md from /glossary/entity-confidence

• Generative Engine Optimization (GEO)

  Generative Engine Optimization (GEO) is the practice of structuring and publishing content so that generative AI engines can fetch it, read it in full, and cite it when they answer a question.

  /glossary/geo.md from /glossary/geo

• GEO and AI visibility glossary

  Definitions for the vocabulary this field uses loosely: 8 terms, each with what it means and how it is measured.

  /glossary.md from /glossary

• llms.txt

  llms.txt is a proposed plain-text markdown file placed at a site's root that gives AI systems a curated map of its most important content, so models can find and read what matters without parsing every page.

  /glossary/llms-txt.md from /glossary/llms-txt

• Prose Parity

  Prose Parity is Lantad's metric for the share of a page's rendered, human-visible text that is present in the raw HTML an AI crawler receives, where 100% means a crawler sees everything a human visitor sees.

  /glossary/prose-parity.md from /glossary/prose-parity

• Structured data for AI (schema markup)

  Structured data is schema.org markup, usually written as JSON-LD, that labels a page's facts in a machine-readable format so AI systems can extract and attribute them without having to interpret the surrounding prose.

  /glossary/structured-data.md from /glossary/structured-data

Fix guides 6

• Fix AI crawler readability on Framer

  A published Framer site is pre-rendered to static HTML and is usually readable without JavaScript, so when ChatGPT misses your content the cause is specific: a code component, an embed, or an interaction that keeps text out of the HTML.

  /fix/framer.md from /fix/framer

• Fix AI crawler readability on Lovable

  Lovable server-renders new projects by default and serves older client-rendered projects a rendered copy to recognized crawlers, so many sites are readable. The gap is browser-fetched content and any crawler Lovable does not recognize.

  /fix/lovable.md from /fix/lovable

• Fix AI crawler readability on Next.js

  Next.js hands AI crawlers fully rendered HTML when you use SSR, static generation, or server components, and an empty shell only when your content is rendered client-side or loaded in useEffect after the page mounts.

  /fix/nextjs.md from /fix/nextjs

• Fix AI crawler readability on React (single-page apps)

  React is not invisible to AI, but a client-rendered single-page app ships an empty div and builds the page in the browser, so a crawler that does not run JavaScript receives a shell with almost no text.

  /fix/react.md from /fix/react

• Fix AI crawler readability on Shopify (Liquid)

  Shopify themes are server-rendered Liquid, so most of your store is already readable to AI crawlers, and the real gap is the content third-party apps and widgets inject with JavaScript after the page loads.

  /fix/shopify.md from /fix/shopify

• Per-stack fix guides

  How to make a site readable to AI crawlers on 5 specific stacks, with the steps that apply to each rendering model.

  /fix.md from /fix

Platform guides 5

• How to get cited by ChatGPT

  ChatGPT can only cite a page its crawlers are allowed to fetch and can read without JavaScript, so being permitted and readable comes first, and any citation follows from there.

  /how-to-get-cited/chatgpt.md from /how-to-get-cited/chatgpt

• How to get cited by Claude

  Claude reaches your page through named Anthropic crawlers that honor robots.txt and work from your raw HTML, so being cited starts with being fetchable, allowed, and readable without JavaScript.

  /how-to-get-cited/claude.md from /how-to-get-cited/claude

• How to get cited by each AI platform

  What 4 named platforms actually read, and what to change so they can read you.

  /how-to-get-cited.md from /how-to-get-cited

• How to get cited by Google AI Overviews

  Google AI Overviews are built from Google's Search index, so getting cited starts with a page Googlebot can crawl, index, and read, and no site can promise the citation itself.

  /how-to-get-cited/google-ai-overviews.md from /how-to-get-cited/google-ai-overviews

• How to get cited by Perplexity

  To be cited by Perplexity, a page first has to be readable: it must fetch cleanly, carry its real text in the raw HTML before any JavaScript runs, and stay open to PerplexityBot in robots.txt.

  /how-to-get-cited/perplexity.md from /how-to-get-cited/perplexity

Comparisons 4

• Lantad compared with other AI visibility tools

  Honest comparisons against named alternatives, including what each of them does better than Lantad.

  /vs.md from /vs

• Lantad compared with Otterly.AI

  Otterly.AI tracks what AI engines already say about your brand; Lantad measures and fixes whether AI crawlers can read your site in the first place.

  /vs/otterly.md from /vs/otterly

• Lantad compared with Peec AI

  Peec AI tracks what AI assistants already say about your brand; Lantad measures and fixes whether AI crawlers can read your site in the first place.

  /vs/peec.md from /vs/peec

• Lantad compared with Profound

  Profound tracks what AI assistants say about your brand across many engines; Lantad tests and fixes whether AI crawlers can read your pages in the first place.

  /vs/profound.md from /vs/profound

Findings 11

• A user agent is a claim, not an identity

  Every per-crawler number you have ever seen, including ours, counts requests that said they were a crawler. Verifying the claim needs an IP check, and most tools do not do one.

  /blog/a-user-agent-is-a-claim-not-an-identity.md from /blog/a-user-agent-is-a-claim-not-an-identity

• Five structural signals that tell an AI who you are

  We re-measured the five structured-data signals behind Entity Confidence against 30 pages fetched in one day, removed two of them, and added two nobody was checking.

  /blog/five-signals-that-tell-ai-who-you-are.md from /blog/five-signals-that-tell-ai-who-you-are

• The AI Overviews opt out does not live on your site

  Google began testing a Search Console setting on 3 June 2026 that excludes a site from AI Overviews and AI Mode. It is an account setting rather than a file, so no external scan of your site can tell you whether it is set.

  /blog/ai-overviews-opt-out-does-not-live-on-your-site.md from /blog/ai-overviews-opt-out-does-not-live-on-your-site

• The AI preferences standard you cannot deploy yet

  The IETF vocabulary for expressing AI usage preferences reached its sixth draft on 28 April 2026. The companion draft that defines where to write one down expired on 1 May 2026, so there is still nothing on your site for a crawler to read.

  /blog/the-ai-preferences-standard-you-cannot-deploy-yet.md from /blog/the-ai-preferences-standard-you-cannot-deploy-yet

• The AI Visibility Index

  A live index built from real scans, showing how readable the measured web is to AI crawlers, with the sample size stated beside every figure.

  /research.md from /research

• The Lantad blog: findings from real scans

  Dated, argued posts built on measured scan data, plus guides and product notes. Every figure carries its measurement date.

  /blog.md from /blog

• Three AI crawler tokens that will never appear in your logs

  Google-Extended, Applebot-Extended and anthropic-ai are robots.txt product tokens that send no user agent of their own, so a log search for them returns zero forever, and that zero does not mean what it looks like.

  /blog/the-crawler-tokens-that-never-appear-in-your-logs.md from /blog/the-crawler-tokens-that-never-appear-in-your-logs

• Two layers decide whether AI can read your site, and robots.txt is the weaker one

  Cloudflare said on 1 July 2026 that from 15 September newly onboarded domains will block Training and Agent bots by default on pages that display ads. Your robots.txt will not mention it, because robots.txt was never the layer that answers.

  /blog/two-layers-decide-if-ai-can-read-your-site.md from /blog/two-layers-decide-if-ai-can-read-your-site

• What a crawler actually meets on a real storefront

  Six real captures, measured on one day in July. Two stores on the same platform produced opposite outcomes, and the difference was not their content.

  /blog/what-a-crawler-meets-on-a-real-storefront.md from /blog/what-a-crawler-meets-on-a-real-storefront

• What the evidence actually says about llms.txt

  Ahrefs measured 137,210 domains in May 2026 and found 97 percent of valid llms.txt files were never fetched at all. We ship an llms.txt tool, and we think you should know this.

  /blog/what-the-evidence-says-about-llms-txt.md from /blog/what-the-evidence-says-about-llms-txt

• Why we will not grade a page we could not measure

  Every AI visibility tool prints a number. When the measurement failed, that number is a guess wearing a lab coat, so ours refuses to appear.

  /blog/why-we-withhold-a-grade.md from /blog/why-we-withhold-a-grade

Company 3

• Lantad privacy policy

  What Lantad stores, for how long, and the rights that apply. Written from what the code does rather than from a template.

  /privacy.md from /privacy

• Lantad terms of service

  The terms that apply to using Lantad, including acceptable use of the scanner and the limits of what a report asserts.

  /terms.md from /terms

• Markdown index for AI crawlers

  Every public page on lantad.co in markdown, grouped and linked, so a crawler can take the content without parsing HTML.

  /md.md from /md