The /.well-known/ files and headers that tell AI agents how to use your site

There is a quiet convention for telling an AI agent how to use your site: a small set of files under /.well-known/ and a few HTTP headers, all in predictable places so an agent can find them without guessing. Most sites expose none of them, and most sites do not need all of them. This is the map of what exists in 2026, what each one does, and the short list actually worth shipping. It is the deepest layer of agent readiness, the one almost nobody has reached yet.

01 · The conventionWhat /.well-known/ is, and why agents use it

The idea is old and boring, which is exactly why it works. Instead of every service inventing its own place to publish metadata, they agree on one prefix: /.well-known/. A client that wants your security contact, your certificate-validation file, or your login configuration knows precisely where to look. You have met it before:

• /.well-known/security.txt lists who to contact about a vulnerability.
• /.well-known/acme-challenge/ is how Let’s Encrypt verifies you own the domain.
• /.well-known/openid-configuration tells an app how to authenticate against you.

AI agents now use the same shelf. Rather than scrape your homepage and infer what you are, an agent can fetch a known path and read a structured answer. That is the whole move: predictable location, machine-readable content, no guessing.

02 · The manifestsThe files that describe you to an agent

Several competing manifests live under /.well-known/, and they answer different questions. They sit alongside the root-level files an agent also looks for, like your llms.txt and sitemap. The split that matters is declarative versus functional: some files just say who you are, others expose things an agent can actually call.

A declarative file like ai-agent.json, a vendor-neutral identity spec published in early 2026, is cheap: a short JSON object with a name and description, plus optional capabilities and contact. It tells an agent who you are. It does not let it do anything.

{
  "name": "Pagewatch",
  "description": "Monitors any webpage for changes and alerts you instantly.",
  "capabilities": ["change-monitoring", "webhooks"],
  "contact": "support@pagewatch.dev"
}

The functional one is the MCP server card. The Model Context Protocol is the standard agents use to call tools, and a server card at /.well-known/mcp/server-card.json lets an MCP client discover and connect to your tools automatically, instead of a user wiring it up by hand. This is the powerful end of the spectrum, and the expensive one, because you need an actual MCP server behind it.

03 · Content negotiationThe headers that hand agents clean content

Files under /.well-known/ tell an agent what you are. Headers tell it how to read you. The cheapest and most useful thing here has a name now: serving markdown to agents.

When an agent fetches a page it wants clean text, not your navigation, scripts, and styling. There are four interchangeable ways to give it that, and any one is enough:

• Honour an Accept: text/markdown request header by returning markdown.
• Publish a .md version at the same path, so /pricing has a sibling /pricing.md.
• Add a link to the markdown version in your HTML head.
• Send a Link: response header pointing at the markdown version.

<link rel="alternate" type="text/markdown" href="/pricing.md" />

Cloudflare, which coined the “markdown for agents” label, pairs these with its own Content-Signal and x-markdown-tokens headers, but the conventions work on any web server. There is also a small, growing set of authentication headers (HTTP message signatures, used by Web Bot Auth) that let a well-behaved agent prove which company it belongs to, so you can trust it rather than guess from a spoofable user-agent string. That ties into your crawler-access decisions, and it matters more for high-traffic sites than for most indie products.

04 · The honest triageWhat is actually worth shipping

Almost nobody exposes any of this, which makes it tempting to either ignore it entirely or over-build it. Both are mistakes. Sort it by cost and payoff:

The rule of thumb: declarative is cheap, functional is expensive. A few lines of JSON declaring who you are costs nothing and helps a curious agent. An MCP server that exposes real, callable actions costs real engineering, and only pays off if your site has something to do, like a booking, a price check, or an API. A blog does not. A configurator or a checkout does. Build the functional layer when an agent calling it would genuinely help your user, not because a scanner handed you a red mark.

05 · VerificationHow to check what you expose

You can check most of this from a terminal in a minute. Request a page the way an agent would, and see what comes back:

curl -s -H "Accept: text/markdown" https://yourdomain.com/pricing | head
curl -s https://yourdomain.com/pricing.md | head

Then hit the well-known paths directly, and confirm they return JSON rather than a 404 or your HTML not-found page:

curl -s https://yourdomain.com/.well-known/ai-agent.json
curl -s https://yourdomain.com/.well-known/mcp/server-card.json

If you would rather not do it by hand, a few free checkers cover the same ground: Cloudflare’s isitagentready.com for the broad picture, markdown-for-agents checkers for content negotiation, and Nilkick’s readiness report for these signals alongside the rest. Expect almost everything to come back missing. That is normal, and it is the opportunity, because this is the layer where being early costs the least.