Every new AI agent that fetches web content creates another integration problem. Sites block bots. Bots evade blocks. Everyone wastes tokens parsing HTML navigation chrome. And agent builders face the same N² scaling wall on the other side.
The N² integration problem
Consider a recruiting platform with sourcing agents that need candidate data from 50 employer ATS systems. Each ATS has different auth, payload schemas, rate limits, and compliance requirements. That's 50 bespoke integrations for one agent.
Now add a second sourcing agent from a different vendor. Another 50 integrations. With N agents and M sites, you get N × M integration pairs — each requiring auth, schema mapping, retry logic, and audit trails.
This is the same problem HTTP solved for human browsers in 1991. Before HTTP, every client-server pair needed custom protocols. HTTP standardised request/response semantics so any browser could talk to any server.
Agent Delivery Protocol (ADP) proposes the same standardisation for machine traffic: discover capabilities via manifest, negotiate response shape, audit via delivery envelopes.
Why block-or-evade fails
The web's default equilibrium is adversarial:
• Sites deploy WAFs and bot management that block machine traffic
• Agents spoof user agents, rotate IPs, and ignore robots.txt
• Both sides lose: sites burn origin compute on blocked requests; agents waste tokens on HTML parsing
A 2026 scan of SaaS marketing sites found that 37% of bot traffic is live user-triggered fetches (ChatGPT-User, Perplexity-User) — users asking an AI to check pricing or docs. These are high-intent, latency-critical requests that WAFs routinely block.
Blocking is not a strategy. It is a tax on both sides.
The cooperative alternative
ADN adopts an invitation-over-interception model:
1. Discovery — sites publish machine-readable manifests at /.well-known/adn.json with links to Markdown, DocLang, MCP tools, bulk corpus, and product catalogs
2. Content negotiation — agents request Accept: text/markdown and get clean content without HTML chrome
3. Intent routing — the gateway classifies traffic into nine intent classes and serves the response shape each job needs
4. Identity ladder — anonymous agents get correct responses; registered agents get better limits; Web Bot Auth agents get highest trust
5. Graceful degradation — when limits apply, responses include machine-readable upgrade instructions, never a silent wall
What this looks like in practice
A training crawler (GPTBot) hitting your docs site gets a bulk corpus export with delta feed — not 500 individual HTML pages.
A live user asking ChatGPT to check your pricing gets fast Markdown — latency-critical, never rate-limited.
An IDE agent (Cursor) indexing your API reference gets MCP tools for structured search and fetch.
Googlebot gets untouched passthrough — non-negotiable safety rule.
Same site. Four different jobs. Four different response shapes.
The agent side
On the agent side, the problem is identical. Register once at the agent directory, get an API key, and route through any ADP-compliant gateway with a single header:
GET /docs/api.md HTTP/1.1
Accept: text/markdown
X-ADP-Agent-Key: agt_your_keyNo bespoke integration per site. No HTML parsing. No user-agent spoofing.
What comes next
ADP v0.1 defines discovery, intent classification, response negotiation, and delivery envelopes for audit trails. Future versions will add clearing hooks for agent-to-agent settlement — the infrastructure layer for multi-party agent workflows in recruiting, procurement, and fintech.
Start here:
• Run the free efficiency scan: https://www.agent-delivery.network
• Register your agent: https://registry.agent-delivery.network/register
• Read the open spec: Agent Delivery Protocol v0.1