Cooperative machine traffic — Agent Delivery Network

The web's machine traffic problem is not a technology problem. It is an incentive problem. Sites block bots because bots cost money and provide no signal. Agents evade blocks because they need content to do their job. Both sides lose. ADN changes the incentives with a cooperative invitation model.

Invitation over interception

Three approaches exist for handling machine traffic:

Approach	How it works	Problem
Identity-gated	Serve signed bots well, block everyone else	90%+ of traffic is unsigned
Hosted proxy	Point a hostname at ADN; all traffic on it flows through us	Uptime on the hostnames you proxy
Invitation + edge connector	Serve anonymous traffic correctly; edge connector captures traffic that ignores invitations	Requires compatible edge stack

ADN chose invitation plus hosted proxy or edge connector because it serves the entire traffic spectrum — from anonymous curl scripts to Web Bot Auth agents — without touching human browser traffic on edge-connector paths.

How invitation works

Sites publish discovery artefacts that tell agents where to find structured content:

Manifest at /.well-known/adn.json — links to all machine-readable resources

Navigation at /llms.txt — page index in Markdown

Link headers on HTML responses — RFC 8288 discovery without scraping

DNS-AID at _agent.<domain> — TXT record pointing to MCP endpoint

Agents that honour these invitations get better content in better formats — without the gateway ever intercepting their traffic.

Content negotiation as universal fallback

Only 3 of 7 major agents send Accept: text/markdown by default. ADP supports URL suffix fallback:

GET /docs/api/index.md
GET /docs/api.md
GET /docs/api.dclg
GET /index.dclg

Agents can also negotiate DocLang via Accept: application/vnd.doclang.document+xml for structured XML with explicit headings, tables, and code blocks.

Even a totally anonymous script gets correct Markdown or DocLang with zero identity — graded by behaviour score, not blocked.

The identity ladder

Cooperation is rational at every tier:

Tier 0 (anonymous): Full content via negotiation. Conservative rate limit. Visible path to register.

Tier 1 (verified bot): Class-specific defaults. Higher throughput for known crawlers.

Tier 2 (rDNS/IP range): Bulk corpus access for training crawlers with confirmed identity.

Tier 3 (registered agent): Better limits across every site on the network. Free registration at https://registry.agent-delivery.network/register.

Tier 4 (Web Bot Auth): Highest trust. Commercial terms available.

Every tier is independently useful. No tier gates basic service.

Why agents register

Registration takes 30 seconds and buys something immediately:

• Visibly higher rate limits on the first request

• Access to MCP tools and bulk corpus on participating sites

• Listed in the public agent directory

• Machine-readable terms acceptance recorded for compliance

The network effect: every additional site makes registration more valuable to agents. Every additional agent makes the network more valuable to sites.

Why sites cooperate

Sites that publish discovery artefacts and enable hosted proxy or an edge connector get:

• Origin offload — machine traffic served from edge cache, not your servers

• Per-class control — different policies for training crawlers vs live fetches

• Token savings — agents get Markdown instead of parsing HTML nav chrome

• Analytics — traffic-by-class breakdown in the dashboard

The free efficiency scan at https://www.agent-delivery.network shows the ROI before you commit.

Edge connector safety contract

For sites that need full offload while keeping custom routing, the edge connector runs at the customer's edge with a fail-open guarantee:

• Classification happens in Cloudflare Snippet or Next.js middleware

• If classification fails → request goes to origin

• If gateway is down → request goes to origin

• Human browser traffic → always goes to origin

10 automated tests verify fail-open behaviour. This is not optional — it is a safety contract.

Machine-readable compliance

Every degradation response includes upgrade instructions:

{
  "error": "rate_limited",
  "upgrade": {
    "register": "https://registry.agent-delivery.network/register",
    "terms": "https://gateway.example.com/terms.json"
  },
  "retry_after": 3600
}

No silent walls. No mystery 403s. Cooperation has a clear, frictionless path.