How AI Site View measures a website

Every number on your report comes from somewhere, and this page is that somewhere. What we test, what each check is worth, what counts as evidence, and where the method stops. If a result ever disagrees with this page, the page is the thing that is wrong.

---

Version and changelog

A measurement that changes silently is not a measurement. Observed results carry their scan time in UTC and this version number, so a result can always be traced back to the rules that produced it. Your report reads, for example, Observed 31 Aug 2026, 14:32 UTC · Methodology v1.1.

Scores produced under different methodology versions are not directly comparable. Every stored scan keeps the version that produced it, so history stays honest rather than tidy.

The AI Site View Score, 0 to 100

Two components. Nothing else contributes to the headline number, and no component is estimated to make the panel look complete.

Your report shows both components natively, never rescaled into each other: 52 of 60 plus 31 of 40 is 83 of 100. If you can't add the parts up to the total on your own report, the report is lying to you.

When a probe does not answer

A crawler probe that times out twice reports "could not be determined during this scan". It leaves the denominator entirely and the remaining scored weights scale back up. A timeout is never counted as a block, because a slow network is not a policy decision. Crawlers blocked by robots.txt or denied at the server score zero of their weight, because that is a real barrier.

Two gates that sit outside the arithmetic

• No score at all. If a control request cannot reach the site, or the site blocks our prober outright, there is no score. A failed measurement is reported as a failed measurement, never as a low number.
• Capped at 30. A homepage carrying noindex is capped at 30. The cap is applied after the two components are composed, never folded inside one of them, so you can still see which part of the site is healthy underneath the barrier.

Grade letters A to F are bands over the same number and are defined once, in the methodology module the whole product reads from.

Every result is a point-in-time observation. It describes what happened when we asked, from where we asked, on the date stamped on your report.

The four diagnostic states

The state leads your report, because "your crawlers are blocked" is more useful than "you scored 41". The number stays; it just stops being the first thing you read.

The crawler matrix

Nine crawler identities are examined on every scan. Six of them decide part of your score. The other three are reported because you should know about them, and scored at nothing because they answer a different question.

The six scored weights sum to 42, which is the crawler share of Technical Access.

Why training crawlers do not move your score

GPTBot, ClaudeBot and Google-Extended are presented together as a training-access preference, with each one marked allowed, blocked or unknown.

A publisher who blocks model training and welcomes search retrieval has made a coherent decision, and a tool that marks them down for it is scoring its own opinion.

Why Google-Extended is never probed

Google-Extended is a robots control token, not a crawler identity we can meaningfully impersonate through an HTTP probe. We evaluate it through robots.txt policy only.

What a probe can and cannot prove

We never report a successful probe as proof that a crawler is able to reach you. A probe result is evidence about a probe, not a guarantee about the genuine crawler's full production behaviour. So every crawler result resolves into one of four honest verdicts.

Only one of those four is definitive about policy: a robots.txt disallow is a statement you published, so we can report it as fact. The rest are observations, and they are worded as observations.

Every crawler result opens an evidence drawer carrying the probe identity, the HTTP status code, the robots.txt decision with the matched rule where one applies, the test origin, the observed timestamp, and this limitation:

Server-delivered content

We check whether important content exists in the HTML returned directly by the server, without requiring browser execution. This is a durable machine-readable baseline regardless of how individual crawlers render JavaScript.

That framing is deliberate. Rendering behaviour varies between crawlers and changes over time, and many automated crawlers rely heavily on server-delivered HTML. We measure the thing that stays true either way, rather than a claim about any particular crawler's JavaScript engine on any particular day.

How every finding is labelled

Four labels, and the label is part of the finding rather than a disclaimer at the bottom of the page. You should always be able to see how hard the evidence behind a statement actually is.

The UNKNOWN discipline

Missing evidence is never converted into an assertion about your business. This is the single rule that most separates this report from a confident-sounding one, and it holds for every fact and every indicator.

The difference is not politeness. The first sentence is about our scan and is true. The second is about your company and might be completely false, because the pricing could be on a page we did not inspect.

What counts as established

Eleven business facts are read from what the server returned. Each one lands on clear, partial or unknown, and the qualifying evidence for each is fixed in advance rather than decided in the moment. The extraction is fully deterministic: there is no language model anywhere in this path, because a language model will always produce a confident sentence, including about an empty page.

Open any fact below for the acceptance criteria. The criteria are declared in api/_methodology.js, implemented by the extractor in api/_profile.js, and pinned by a fixture suite across fourteen site archetypes so a loosened rule breaks a test rather than quietly starts lying.

The ten diagnostic indicators

Each indicator is derived deterministically from the facts above, lands on clear, partial or unknown, and carries the same provenance and evidence that facts do. Cross-source consistency can additionally land on conflict.

• Business identity completeness. Name, type and description considered together.
• Offering clarity. Whether specific products or services are established.
• Audience clarity. Whether who you serve is established.
• Location clarity. Whether where you operate is established.
• Pricing clarity. Whether prices connect to specific things you sell.
• People and leadership clarity. Whether anyone is named.
• Trust and proof clarity. Whether claims are checkable.
• Contact clarity. Whether a machine-readable way to reach you exists.
• Policy clarity. Whether policy documents are findable.
• Cross-source consistency. Declared names compared across sources. A conflict leads your blind spots, because two different names for one business is the fastest way to become two half-businesses to a machine.

llms.txt is detected, and never scored

If your site publishes an llms.txt, we find it and report it. It earns you nothing, and that is on purpose. There is no measured evidence today that its presence changes how machines read a website, so awarding points for it would be scoring a fashion and inviting you to spend an afternoon on a file that may do nothing.

Being told what does not count is part of what you are here for. A checker that quietly rewarded every new convention would look generous and be useless.

What AI Site View will not pretend to know

The limits are not a disclaimer. They are the product. Every one of these is something a competitor could show you as a confident number today.

One line covers the rest of it: AI Site View reports what it can prove, labels what it infers, and says UNKNOWN when the evidence stops.