Nineteen counted brand mentions, and about two were the company

What the first live run reported, and what was actually in those answers

The prompt tracker's first scheduled run fired at 06:00 UTC on 13 August 2026. The repository's deviation log records the shape of it: 186 runs, being 31 prompts across three engines and two countries, of which 151 produced a scored extraction and 35 failed before any answer came back. Of the 151 classified answers, 19 were counted as mentions of this brand. That is 12.6 percent, and the dashboard rendered it beside a Visibility Score of 17 out of 100.

The truth was near 1 percent. The word lantad is a Tagalog adjective meaning exposed or in full view, and the engines knew that better than they knew the company. Of the 19 counted mentions, eight were answers defining the adjective and citing dictionaries, three resolved the name to a different business, being a real-estate investing app called Landa or a vendor called Lanta AI, and six were engines naming the brand only in order to say they had never heard of it. That leaves roughly two answers genuinely about this company, a rate near 1.3 percent rather than 12.6.

Two of the worst cases are stored verbatim in the golden set and can be read today. One is the live answer to the prompt asking what lantad is, which opens by defining the adjective as exposed to view and cites kaikki.org and en.wiktionary.org. Another is the live answer to a prompt asking whether lantad is legitimate, which says it could not find a company by that name, offers Landa instead, and cites a technology news report and a review site. In both, the brand name is unambiguously present in the text. In neither is the answer about the brand.

The damage did not stop at one number. The same judge call also harvests the other entities an answer recommends, which is what feeds a share-of-voice comparison, and on a dictionary answer it dutifully harvested the Tagalog synonyms halata, malinaw and nakita as competitors. The panel then reported this brand as first of six tracked entities. Every downstream figure inherited the same error, which is the ordinary consequence of a bad primitive and part of why this site keeps insisting that a citation count is not answer influence. A rate is only ever as good as the event it counts.

The fault was ours in the specification rather than the model's in the answer. The original brief defined a mention as the brand name or domain appearing in the answer text, which is the literal reading, and the judge prompt never told the model what the brand was, so it was asked whether the brand was mentioned and could only look at letters. Both halves were fixed together, and the published definition on our methodology page now states both tests instead of one. Auditing our own scale against our own evidence is a habit here rather than an exception: the same exercise applied to the scan product found that a robots.txt blocking every citation crawler still grades B.

Presence is a question about characters, identity is a question about meaning

The fix is a division of labour, and it is worth stating in the abstract because nothing about it is specific to a Tagalog collision. Deciding whether a string of characters occurs in a piece of text is a question no model is needed for, and a model is a bad choice for it: the deterministic answer is cheap, auditable, and identical every time it is asked. Deciding what those characters refer to is a question about meaning, and there is no lookup table for it. A matcher cannot tell this company from an adjective. A model can, given a sentence saying what the company is.

So the two are computed separately and then combined. The alias matcher runs over the answer text against five declared spellings, being Lantad, LANTAD, lantad.co, Lantad AI and LantadBot, deduplicated on their normalised form so that the first two are one needle. It tokenises rather than substring matches, which is why the possessive in a phrase such as Lantad's costs nothing, and it accepts a single edit for aliases of four characters or more, a threshold named in core/src/tracker.ts as FUZZY_MIN_ALIAS_CHARS. Longer aliases claim their tokens first, so Lantad AI counts once rather than as a phrase hit plus a bare hit.

The judge is asked a different question and must return a verdict on the name from a closed set of five: brand, other_entity, homonym, brand_unknown or absent. The brand's own one-line description and its category terms go into that prompt, because the judge cannot separate a company from a word without a sentence telling it what the company is. A mention is then the conjunction of the two: the characters are present, and the judge confirms they mean this company. That conjunction is the whole correction, and it is the reason a strong entity confidence signal matters more for an ambiguous name than for a coined one.

Citation is deliberately not gated on that verdict, and the asymmetry is the interesting part of the design. A citation is your domain used as a source, and that fact holds however the engine read your name, so cited-without-mentioned is a real state the tracker reports rather than an inconsistency it smooths over. The entity universe is gated, though: entities and cited domains are taken only from answers about the tracked market, which is what stops a dictionary's sources being read as a competitive citation layer. That distinction between what an observation proves and what it is allowed to imply is the same one behind why we withhold a grade rather than guess it, and it is why rival names are extracted rather than typed in.

One consequence lands on customers with ordinary-word brand names, and it landed here on 22 August 2026. A new project's brand is created with an empty description on purpose, because inventing one would teach the judge something false about somebody else's company. The prompt builder had been rendering those honest empties unconditionally, so an undescribed brand's judge read three dangling assertions, one of them the words it is followed by nothing at all. That is not a judge with less evidence, it is a judge told the brand is nothing. It now omits what nobody knows and states the tie-break instead.

What the 55 stored cases hold, and how they were chosen

The golden set is a single JSON file holding 55 answers, each with the engine's own cited URIs, the judge reply that was returned for it, and a hand label recording what a person decided the correct output was. The suite started at 50 and grew to 55 when the incident above supplied five shapes nobody had thought to invent: the Tagalog dictionary answer, both of the same-named companies, an engine disclaiming knowledge, and, importantly, the one live answer that genuinely was about this company and arrives in the same shape as the collisions. That last case is what stops a fix trading a false high for a false zero, which is the failure mode of every over-correction.

The labels are not evenly spread, and they are not meant to be. Across the 55, 31 are labelled as a mention, 18 as carrying a citation of this domain, and 16 as having a measurable position in a ranked answer. Eleven of the 55 carry no citation container at all, which is different from carrying an empty one: an engine that reported no sources is not the same as an engine that reported zero sources, and 20 further cases carry an explicitly empty list. Collapsing those two states would silently convert an unmeasured run into a measured absence.

The distribution is the point at which honesty about this post's own evidence is owed. These 55 answers were selected to cover the families of failure, not sampled from traffic. Six carry deliberately malformed judge replies, one is an empty answer, and several encode near-miss spellings no engine has ever actually produced. A suite built that way is adversarial by construction, so every agreement figure below describes performance on hard cases and understates what the same components would score on a random day's answers. It is a regression test rather than a benchmark, and reading it as a benchmark would be exactly the sort of confident wrong number this post is about.

Five of the 55 come from real answers returned by real engines on 13 August 2026, and the other 50 were written. That ratio is itself worth stating, because it bounds how much any figure here can be said to describe the live web. What the suite does support is a claim about the code: whatever an engine returns, if it takes one of these shapes, the pipeline resolves it the way a person did. That is the same class of guarantee as the stored page fixtures behind our scan verdict, it is what lets the prompt tracking surface state a number at all, and it is the sort of evidence that separates measurement from the general advice that fills most writing about answer engine optimisation.

Taken at face value, the judge was wrong on 11 of the 55

Here is the measurement this post exists for. Replaying the file on 22 August 2026 and scoring each stored judge reply against the hand label on all four fields, being mentioned, cited, position and name verdict, the judge alone agrees on 44 of 55 cases. The reconciled pipeline, which combines that same reply with the deterministic matcher and the engine's own citation list, agrees on 55 of 55. The floor the test enforces is 98 percent agreement, a value set in core/src/config.ts as goldenAgreementMinPct, and that number is a threshold somebody chose rather than anything measured.

Counted by field across those 11 disagreements, the judge got the citation wrong seven times, the mention wrong five times, the name verdict wrong three times and the position wrong twice. The totals exceed 11 because several cases are wrong in more than one place, and the failures run in both directions, which is what makes any one-sided correction useless.

Four are worth reading individually. In one case the answer names Lentid, two edits away from the brand and therefore not a match, and the judge reports a mention at position one regardless: a hallucinated mention with a rank attached to it. In the mirror case the answer names Lanted, a single substitution and therefore a genuine hit, and the judge reports no mention at all. In a third, on an answer entirely about a rival, the judge claims both a mention and a citation, and the URL it offers as evidence for that citation is the rival's own pricing page. In a fourth, the judge claims a citation and supplies no URL whatsoever, on an answer where the engine reported no sources.

Each of those falls to a different rule, and none of the rules is clever. Presence belongs to the matcher in both directions, so the hallucinated mention is dropped and the one-letter misspelling is kept. A citation claim is honoured only when its own URL evidence names the brand domain, so a rival's pricing page cannot become our citation and a claim with no URL is not a claim. Position exists only where a mention exists, so overturning the mention takes the rank with it. Deterministic citation evidence runs the other way too: two of the 11 are cases where the engine's own source list carries the brand domain, once behind a www prefix and once in upper case, and the judge missed both while a hostname comparison did not.

The general lesson is not that this particular judge is weak. It is that a language model asked to read an answer will produce a plausible reading of it, and a plausible reading is not a measurement. Where a cheaper and more reliable method exists for part of the question, spending the model on that part buys nothing and costs accuracy, which is a claim about AI visibility tooling rather than about models. Independent work keeps finding the same asymmetry in this domain, including the result that cited links held up far better than the facts drawn from them across 14 models, and the result that most AI citations point at other companies rather than at the brand being asked about.

A judge that fails to answer must not read as a brand that was not mentioned

Six of the 55 cases exist for a failure that has nothing to do with reading comprehension: the judge reply does not parse at all. One is prose where JSON was asked for. One is JSON truncated mid-object, which is what a clipped output cap produces. One returns the right keys with the wrong types, one omits a required key, one returns an entities list of the wrong shape, and one is an empty answer with an empty reply. Three of the six are stored as raw strings, because they are not JSON in the first place. A fourth raw string sits in the set and is not one of the six: a reply wrapped in a code fence, which parses once the braces are sliced out.

The temptation with an unparseable reply is to treat it as a negative result, because a boolean field needs filling and false is the convenient default. That is the error this design refuses. A run with no verdict reports no mention, because nothing is entitled to claim one, but it keeps a null verdict and the daily rollup drops it from the denominator entirely. It lands in neither half of any rate. The practical difference is what a customer sees on a day the judge is unavailable: a gap in the series, rather than a confident zero with a downward arrow beside it, which would be a fabricated regression on a day when nothing was measured at all.

The deterministic half still works during that outage, and the stored cases prove it rather than assert it. One of the six carries an answer that plainly names the brand and an engine citation of lantad.co, with a judge reply written in prose. The reconciliation reports no mention, because identity was never confirmed. It reports the citation as true, because that rests on a hostname comparison needing no model. Position is left unmeasured. Three fields, three different honest answers, out of one broken reply.

These six are also excluded from the separate harness that puts the golden questions to a live vendor model, and the reason is worth recording: no correct model can produce a malformed reply, so grading a model on cases designed to be malformed would make that gate unreachable rather than strict. The offline replay still runs them, because the offline replay is testing the pipeline rather than the model. Keeping straight what is being tested and what is merely being replayed is the same discipline as reporting that a compliance record is falsifiable rather than proof, and it is why our research page states the limits of a sample instead of burying them.

What this does not prove, and what to check on your own site

The limits first, because they are load-bearing. This is a replay of stored replies against stored labels, so it measures the code and not any model: no vendor engine was called for this post, and nothing here says how often a live judge errs on live answers. The 55 cases were chosen to be difficult, so 44 of 55 is a score on a hard suite, and reading it as a general accuracy figure for LLM judges would be a misuse of it. The 100 percent reconciliation figure is agreement with labels a person wrote, so it inherits whatever that person got wrong, and a regression suite that always passes is also a suite that has stopped finding anything. Fifty-five answers about one brand with an unusually ambiguous name is not a sample of anything, and the underlying instability of this kind of measurement is documented independently: repeated runs of the same prompt disagree enough that seven runs per prompt were needed before a visibility rate settled, and a five point confidence interval on citation share needed roughly 40 queries on one platform and 150 on another.

Two things follow for anyone buying this category rather than building it. First, ask what a vendor counts as a mention, and specifically whether a name appearing in the text is sufficient on its own. If it is, the number is an upper bound, and the gap between it and the truth grows with how ordinary your brand name is. Most of the category is not a crawler at all: our own reading of ten AI visibility products found six of them put prompts to models and read the answers back, which is precisely the step this post is about. Second, ask what the tool reports on a day its grader fails, because one that prints a zero for an unmeasured day will show you a trend that is partly an outage.

The half of this you can act on sits on your own pages, and it is entity disambiguation. If an engine cannot tell your company from a word or from a similarly named business, the correction is to publish the identity rather than to hope a model infers it. The schema.org vocabulary carries a property for exactly this: sameAs, defined in version 30.0 as the URL of a reference web page that unambiguously indicates the item's identity, offering a Wikipedia page, a Wikidata entry or an official website as its examples. Google's Organization structured data documentation, carrying Last updated 2026-04-15 UTC, lists sameAs among recommended rather than required properties, and advises placing the markup on a home page or a single page describing the organisation rather than on every page of a site.

That is what our own free entity confidence diagnostic scores, and its weights are settings rather than findings: a declared identity node is worth 35 points as a gate, and sameAs is worth 25, the largest non-gate weight, because sameAs is the only property in that set whose own definition is an identity-resolution statement. None of it guarantees an engine will read your name correctly, and this site will not claim otherwise until it has measured the effect. What it does mean is that the facts an answer engine would need are present in the structured data it can extract rather than absent and left to inference. Whether any of that reaches an engine at all still depends on the layer underneath, which is whether an AI crawler is allowed to fetch and read the page in the first place, and on the answers a given engine is already giving about you, which is what our answer tracking surface reports.