Back to Learn

Help

Googlebot passthrough: the one rule we never break

Serving Markdown to search crawlers can destroy SEO. ADN's hard safety rule: Googlebot and social unfurlers always receive untouched passthrough from your origin.

Making your site agent-friendly must never come at the cost of search rankings. That is why ADN has one non-negotiable rule: > Googlebot and social unfurlers always receive untouched passthrough.

Why this matters

Many "agent-friendly" proposals suggest serving Markdown or simplified HTML to all bots. That is dangerous for SEO:

Googlebot expects your canonical HTML. Substituting Markdown can cause indexing mismatches.

Social unfurlers (facebookexternalhit, Slackbot, Twitterbot) parse OG tags from your real HTML. Serving a different shape breaks previews.

Procurement teams rightly ask: "Will this hurt our rankings?" The answer must be an unqualified no.

How ADN enforces it

Machine traffic is classified before your origin is touched. The legacy_seo_social class includes:

Googlebot (all variants)

facebookexternalhit

Slackbot, Twitterbot, LinkedInBot

Other search and social preview crawlers

These requests are proxied to your origin and returned unchanged. No Markdown substitution. No content stripping. No caching layer that alters the response body.

What other classes receive

Agent-friendly optimization targets different visitors:

answer_engine — Clean Markdown + JSON-LD for AI answer indexers

bulk_training — Corpus export on agreed terms

developer_agent — MCP tools and llms.txt for IDE agents

live_fetch — Fast Markdown for user-triggered chat fetches

Each class gets the shape its job needs. Search engines get exactly what you published.

Verifying in production

After connecting a domain, open Analytics in the ADN dashboard. Filter by class legacy_seo_social. Every row should show response shape passthrough. If it does not, that is a bug — not a feature.

SEO + GEO together

Traditional SEO and generative engine optimization are complementary:

SEO — Googlebot sees your real HTML. Rankings and snippets stay under your control.

GEO — Answer engines and agents get structured content they can quote without burning tokens on markup noise.

ADN routes both from the same gateway. The classifier decides; your origin stays authoritative for search.

Frequently asked

Will ADN change what Googlebot sees?
No. Googlebot is classified as legacy_seo_social and always receives your origin response unchanged. ADN never rewrites, compresses, or substitutes Markdown for search crawlers.
What about Bingbot and other search engines?
Answer-engine indexers like Bingbot may receive Markdown when classified as answer_engine — because they consume content for AI answers, not traditional ranking. Googlebot and social unfurlers are always passthrough.
Can I audit passthrough in production?
Yes. Traffic-by-class analytics in the ADN dashboard shows every request class and response shape. You can verify Googlebot rows always show passthrough.

Continue reading

Run the free efficiency scan on your URL to see what machines receive today.