Bulk corpus, not page-by-page
Where a site publishes one, take the whole corpus and a delta feed instead of walking a sitemap. Faster for you, cheaper for them.
For agent, crawler and framework builders
Register once. Get a better deal on every site in the network.Thirty seconds, no cryptography.
If you build crawlers, agent frameworks or IDE tooling, ADN is the cooperative side of the trade. Declare who you are, what you are for and how to reach you, and every site on the network serves you structured content at higher limits — instead of a shell, a challenge page or a 402 you were never going to pay.
- Web Bot Auth
- MCP
- DocLang (.dclg)
- Content negotiation
- llms.txt
Sound familiar?
Where it hurts today
- 01
You get a shell, a challenge, or a 402
Cloudflare alone returns over a billion 402s a day and most agents walk away rather than pay. The site loses representation, you lose the content, and nobody is better off.
- 02
You burn context on markup
Fetching a rendered page means paying for navigation, cookie banners and inline scripts before a single useful token arrives.
- 03
Every site is a bespoke parser
Structure has to be re-derived per domain, and it breaks the next time somebody ships a redesign.
What you get
What ADN does about it
Clean content, honestly labelled
Markdown by content negotiation or an /index.md URL, DocLang via /index.dclg or Accept header, with JSON-LD where it exists and machine-readable terms attached to the response.
MCP where it is offered
Sites that publish tools let you call them directly — search, fetch, look up — rather than driving a storefront or docs UI like a browser.
Limits that improve with cooperation
The identity ladder runs from anonymous to registered to Web Bot Auth signatures. Every rung up raises your rate limits across every site on the network at once.
How to join
Three steps, none of them cryptography
Registration is the first rung of the identity ladder. Everything above it is optional and raises your limits further.
- 01
Register at the gateway
Name, purpose, contact and the user agents you send. No standards body, no key ceremony, no review queue.
- 02
Accept the network terms
One set of machine-readable terms covering every participating site, so you are not negotiating with each domain separately.
- 03
Send traffic as yourself
Identify honestly and honour the rate posture you are given. Behaviour is a signal: honouring robots.txt, respecting 304s and keeping path diversity sane all move you up.
Where you land
RegistrationFree
Always
The agent side of the network is acquisition, not revenue. There is no paid tier for agents and there is no plan to add one.
What registration does not buy
It is not a licence and it is not a bypass. Each site still sets its own policy per class, commercial scraping is metered rather than free, and abusive patterns — credential stuffing, pagination loops — are refused with a machine-readable reason no matter who is asking.
Questions
The ones that come up first
- What does it cost?
- Nothing, now or later. A registry that charged agents would not be worth registering with — the value to sites is that the directory is complete and neutral.
- Do I need Web Bot Auth to benefit?
- No. Registration is the first rung and needs no cryptography. Signed requests are a later rung that raises your limits further; both are supported, neither is required to start.
- Will my agent be listed publicly?
- Yes — the directory is public, which is the point. Site owners decide policy from it, and being findable there is how a well-behaved agent earns better treatment before it ever sends a request.
Register an agent
Thirty seconds at the gateway's own domain, and the limits apply across every site on the network immediately.
Not quite you?