How readable is the web to AI crawlers?
Every number below is live, aggregated from real Lantad scans and updated as the web is scanned. Each page was fetched once like an AI crawler (no JavaScript) and once in a real browser, then diffed. The methodology is public. Cite freely with a link.
What crawlers actually get
The single biggest gap is content that never arrives: text rendered by JavaScript that most AI crawlers do not execute. Across the sample, the average page ships 69% of its rendered content in the raw HTML a crawler reads. Structured data is the weakest link.
| Mean Prose Parity (content reaching crawlers) | 69% |
|---|---|
| Mean bot access (crawlers allowed to read) | 92% |
| Mean structure score | 82% |
| Mean structured-data score | 50% |
Grade distribution
Based on 1,097 pages that produced a full grade. Pages that could not be graded honestly (bot walls, auth walls, non-HTML) are excluded from the shares. Snapshot: 2026-09-16.
How does your site compare?
Most pages score worse than their owners expect. Run a free scan and see exactly where yours lands, then keep it above the line as your site changes.