When an AI agent fetches your homepage, it does not render CSS or execute JavaScript the way a browser does. It receives a blob of text and pays for every token. Most of that blob is noise.
What agents actually read
A typical SaaS marketing page might ship:
• 45 KB of HTML
• Navigation, footer, cookie banners
• Inline styles and class names
• Script tags and JSON config blobs
• The actual value proposition buried in nested divs
An LLM tokenizer sees all of it. A 45 KB page might cost 12,000 tokens. The same content as clean Markdown might be 2,500 tokens — an 80% reduction.
Why HTML is expensive
| HTML component | Token impact |
|---|---|
| Navigation chrome | High — repeated on every page |
| CSS class names | Medium — meaningless to agents |
| Script blocks | High — often largest single chunk |
| Semantic content | Low — what agents actually need |
Agents do not benefit from your design system. They need the words, structure, and data.
The efficiency scan methodology
ADN's free scan at https://www.agent-delivery.network/#scan:
1. Fetches your URL as a machine visitor would
2. Extracts readable text from the HTML response
3. Estimates token count for the raw payload
4. Generates a clean Markdown equivalent
5. Compares token counts side by side
No signup. One request. Real numbers from your URL.
Who benefits from Markdown serving
Not every bot should receive Markdown:
• answer_engine, live_fetch, developer_agent — Markdown is the right shape
• bulk_training — Corpus export is even more efficient (one fetch vs thousands)
• legacy_seo_social — Passthrough HTML, always
Intent classification ensures the right class gets the right shape.
From measurement to action
Token waste is not just an agent problem. When training crawlers walk your sitemap page by page, every wasted token is a wasted origin request. Publishers on ADN's bulk corpus path serve one export instead of thousands of HTML fetches.
Start with the scan. If your token ratio is above 3:1 (HTML vs Markdown), you have room to improve — either by serving agent-friendly shapes or by fixing what your origin publishes.