Google's Agent Discovery Spec Wants to Be robots.txt for AI

The agent economy spent two years solving the wrong half of the problem. MCP standardized how an agent connects to a tool; agent-to-agent protocols standardized how agents talk to each other. But nobody had answered the question that comes before both: how does an agent find the right capability in the first place? On June 17, 2026, Google published the Agentic Resource Discovery (ARD) specification — an open protocol for publishing, discovering, and verifying AI capabilities across the open web — and it arrived with a coalition that makes it hard to dismiss.

What it actually is

The mechanism is deliberately boring, which is the point. ARD lets any website publish a machine-readable file at /.well-known/ai-catalog.json that lists the agents, MCP servers, skills, and APIs that site offers, described by capability. Registries can crawl and index those files; agents can query them to discover what exists and how to use it. If that pattern sounds familiar, it should — it is the same move that robots.txt made for crawlers and sitemap.xml made for search engines, repurposed for a web that is increasingly read by software acting on a user's behalf.

ARD frames its job as answering three questions an agent faces at runtime: where does the right capability live, which one should I use, and how do I verify it is safe to connect to. That third clause is the one that separates a real standard from a hopeful one. Discovery without verification is a phishing vector — an agent that will autonomously connect to whatever it finds is an agent that will eventually connect to something hostile. By baking provenance and verification into the spec rather than leaving it to each implementer, ARD treats trust as infrastructure instead of an afterthought.

The spec is licensed under Apache 2.0, sits at v0.9 (Draft) on GitHub under ards-project/ard-spec, and builds on the AI Catalog data model from the Linux Foundation's AI Catalog Working Group. None of that is glamorous. All of it is the connective tissue that decides whether a standard becomes load-bearing or becomes another abandoned RFC.

The coalition is the moat

The reason to take ARD seriously is the names on the launch. Contributors include Cisco, Databricks, GitHub, GoDaddy, Google, Hugging Face, Microsoft, Nvidia, Salesforce, ServiceNow, and Snowflake — a list that spans the cloud platforms agents run on, the registries where they live, the enterprise systems they need to reach, and the silicon they run on. Getting Microsoft and Google to co-sign anything is rare enough; getting GitHub and Hugging Face — the two places developers already publish models, MCP servers, and agents — into the same spec is what gives ARD a plausible path to ubiquity.

Standards do not win on technical elegance; they win on adoption gravity. The history of the web is littered with cleaner specifications that lost to uglier ones with bigger coalitions. ARD's authors clearly learned that lesson. By anchoring to existing publishing surfaces — the platforms where capabilities are already listed — they lower the cost of adoption to near zero: if your tools live on GitHub or Hugging Face, your catalog can be generated rather than hand-built. That is how you cross the chicken-and-egg gap that kills most discovery protocols, where no one publishes because no one reads and no one reads because no one publishes.

Why discovery is the real bottleneck

It is worth being precise about why this matters now. The first wave of agentic software lived inside walled gardens: an assistant could use the tools its vendor wired up, and nothing else. That works until you want an agent to do something its maker never anticipated — book a niche service, query a specialized dataset, call an API that launched yesterday. In a closed model, every new capability requires an integration someone has to build by hand. In an open-discovery model, a capability that publishes an ARD catalog is reachable the moment it exists, by any agent that can read the spec.

That shift changes the unit of competition. When discovery is open, the advantage moves from who has the most pre-wired integrations to who builds the most useful capability and describes it well enough to be found. It rewards the long tail — the small tool that does one thing precisely — in the same way search rewarded the long tail of web pages over the curated directories that preceded it. For a web full of capabilities that no single platform could ever enumerate, machine-readable discovery is not a convenience. It is the precondition for the agent economy being an economy rather than a collection of company-store silos.

The asterisks

Restraint is warranted. A v0.9 draft with eleven backers is a strong start, not a settled outcome — specs at this stage routinely fracture into incompatible dialects or get quietly strangled by a competing proposal from someone who wasn't in the room. Notably absent from the launch list are some of the labs whose agents matter most, and a discovery standard that the largest agent platforms route around is a standard in name only. Verification, the spec's best idea, is also its hardest: making trust machine-checkable at web scale is exactly the problem that has defeated every prior attempt at a "safe by default" internet protocol.

But the diagnosis is correct, and that is the thing worth holding onto. The agent economy has been building roads and traffic lights while forgetting it needed a map. ARD is an attempt to draw one in the open, with enough of the industry holding the pen that it might actually get used. For Flux readers who live in the directory of what AI can do, this is the standard that decides whether that directory stays human-curated or becomes something agents can read for themselves.