Essay
What if websites could explain themselves to AI agents?
Why I started building Descuff, and why I think the web may need a better machine-facing layer.
AI agents have become surprisingly good at using websites. Give one a task like finding a black hoodie under £60, checking whether medium is available, and adding it to a basket, and a capable browser agent can inspect the page, locate search, interpret results, open a product, find the size selector, and eventually add the right item.
That is genuinely impressive. But there is something slightly strange about the process.
The application already knows what a product is. It already knows what sizes exist. It already knows how search works. Somewhere underneath the interface there may already be functions or endpoints that amount to:
Yet the agent may still have to reconstruct those capabilities from a user interface designed for a person.
What if the application could explain itself?
The web is built for us, not for agents
A website communicates remarkably well to a human. We understand that a large text box near the top of a shop is probably search. We understand that a red button saying “Delete account” deserves more caution than a small “Learn more” link. We understand menus, forms, icons, tabs, visual grouping and dozens of conventions without really thinking about them.
Agents have had to learn the same environment. Google's current guidance for agent-friendly sites describes screenshots, raw HTML/DOM, and the accessibility tree as important ways an agent may understand a page. Modern agents can combine all three, but screenshot analysis can be slow and expensive in token usage.
A 2025 research project called FocusAgent describes web-agent observations that can exceed tens of thousands of tokens. Its approach reduced accessibility-tree observation size by more than 50% while maintaining strong baseline performance.
Why send the agent so much irrelevant interface information if the application already knows what matters?
Machine interfaces can help
This is not only a theoretical argument. One ACL 2025 study compared agents operating on WebArena using browser interaction, API interaction, and a hybrid of both.
The API-based agent almost doubled the browsing-only result in that experimental setup. The hybrid agent performed best overall: 24.1 percentage points above browsing alone.
That does not prove APIs are always better than browsers. The more interesting result is the direction: a structured machine interface can reveal capabilities that are harder to use reliably through a human interface alone.
Token cost is not imaginary either
Cloudflare has started serving an optional Markdown representation of websites specifically for agents. Their Markdown for Agents feature lets an agent request Accept: text/markdown.
In Cloudflare's documented example, one page went from an estimated 12,345 tokens to 725 tokens: roughly a 94% reduction for that particular page.
Making software better for agents may often mean giving it less context, but better structured context.
The machine-facing layer is already forming
I originally thought this problem might require inventing a new format. The more I looked into it, the more obvious it became that this would probably be the wrong approach. Several standards and proposals already address different pieces of the problem.
llms.txt
The llms.txt proposal provides a concise Markdown entry point containing information and links useful to language models. It does not make a website actionable by itself, but it can help an application explain its content.
OpenAPI and API Catalog
OpenAPI gives machines a structured description of HTTP APIs. RFC 9727 adds automated API discovery through /.well-known/api-catalog, because machine usability requires more than an API existing. The machine has to discover that it exists.
Schema.org and WebMCP
Structured data identifies products, organisations, articles, events and other entities. WebMCP goes further by letting websites expose named tools with descriptions, input schemas and execution behaviour. Chrome opened a WebMCP origin trial in Chrome 149 in June 2026, and Lighthouse can now audit registered WebMCP tools.
A different architecture
The machine-facing side does not have to be one giant protocol. For one application it may mean llms.txt and JSON-LD. For another it may mean OpenAPI, API Catalog and WebMCP. For a simple informational site, the correct answer may be very little at all.
Being agent-ready should not mean generating every AI-related file currently trending on GitHub.
Descuff starts with existing software
New applications can deliberately adopt these standards. Most software already exists. A SaaS product built five years ago may already have hundreds of routes, authentication, database models, forms, internal APIs, server actions, billing, permissions, background jobs and dozens of user workflows.
The information required to make it easier for agents to use is often already there. It is simply expressed through implementation rather than through an explicit machine-facing contract.
What does this application do?
Which capabilities are public?
Which require authentication?
Which operations modify state?
What should not be exposed at all?
That is what led me to start building Descuff.
Descuff turns an existing website into an interface AI agents can understand and use.
It is an open-source experiment in approaching agent readiness from the application itself. The aim is not to invent a Descuff protocol. The idea is to understand the application, understand its capabilities, choose appropriate existing standards, implement them, and independently verify that they actually work.
The current public version focuses on Next.js applications and can inspect routes, APIs, Server Actions, forms, authentication boundaries and existing standards. But those implementation details are not really the long-term thesis. The principles behind them are.
Four principles behind Descuff
Understand before generating
It is easy to generate an llms.txt file, JSON, or an OpenAPI document that looks convincing. The difficult part is deciding whether what was generated corresponds to the application that actually exists.
GET /api/team could publicly list sports teams, or it could return the currently authenticated user's private company team. Those require completely different treatment. So the first useful layer is evidence: routes, APIs, forms, authentication, runtime behaviour, symbols, relationships and existing standards.
Structure and semantics are different problems
A parser can tell you POST /api/slots/query exists. Determining that this operation means check_availability is a semantic question. An LLM can be useful in the middle layer, but it should not be allowed to turn “this looks like checkout” into “expose this payment operation to agents” without evidence, safety checks and validation.
The application does not need one label
One early version of Descuff tried to classify applications into categories like ecommerce, SaaS, booking and content. This works until you meet real software. What matters more is understanding what capabilities actually exist.
Do not trust generated code because an agent says it finished
The workflow separates understanding, implementation and validation. A coding agent can implement the proposed changes, but that agent does not get to decide that the job succeeded.
Does this route really exist?
Is this data actually public?
Is the generated schema valid?
Can a browser discover the WebMCP tool?
Did an authenticated capability accidentally become public?
Chrome's WebMCP documentation makes a similar point: merely defining tools is not enough, and developers are encouraged to run evaluations. This matters because agent-facing interfaces create new security boundaries, including indirect prompt injection risks.
Descuff asks what the application should expose, lets a coding agent implement that, then asks for proof that it was implemented correctly.
So I tried to break Descuff
A project that analyses code can look clever if you only test it against applications created specifically for the project. So I selected eight unrelated public Next.js repositories covering substantially different application shapes.
The repository list, exact commit SHAs and test records are stored in Descuff itself. The point was not to get a nice number. It was to find assumptions. It found quite a few.
The first external audit: 8 repos, 11 failures
In the recorded Phase 10 audit snapshot, those eight repositories exposed 11 concrete issues.
They included incorrect application classification, missed Server Actions, incorrect authentication assumptions, bad standards recommendations, monorepo discovery failures, validation false positives and unsupported Next.js patterns.
A checkout route does not make an app ecommerce
The Next.js SaaS Starter contained Stripe checkout functionality. An early classifier saw that evidence and classified the application as ecommerce. Reasonable pattern match. Wrong conclusion. The fix moved classification toward weighted evidence.
GET does not mean public
Operations involving /api/team and /api/user were initially treated as public reads. They were not. That resulted in stronger authentication-aware capability classification.
Not every Next.js app lives in the root directory
Formbricks exposed another assumption. Its relevant application lives inside a larger monorepo. The fix added nested Next.js application discovery.
Sometimes the best recommendation is nothing
Tailnext is mostly a static site. An early plan recommended OpenAPI, API Catalog and WebMCP even though the application had no meaningful APIs or agent-callable capabilities. The fix was not to build more generators. It was to make Descuff generate less.
A capability snapshot
That is not “Descuff has 100% accuracy.” Twenty-four manually selected capabilities across eight repositories is not a statistically meaningful general benchmark. What it shows is that the external hardening exercise recorded failures, converted them into regression cases and improvements, and reran the same expectations after fixes.
Why the idea is worth pursuing
The ecosystem is moving quickly, so I do not know which individual standards will matter most five years from now. WebMCP is still experimental. llms.txt is a community proposal rather than an IETF standard. New agent protocols will appear. Some existing ones will probably fade away.
But the direction feels much less speculative than it did even a year ago. Cloudflare is converting HTML to Markdown specifically for agents. Chrome is experimenting with browser-native structured tools for agents. Lighthouse can audit registered WebMCP tools. The IETF has standardised automatic API discovery through RFC 9727.
There is even a 2025 position paper whose title summarises the idea well: “Build the web for agents, not agents for the web.” Descuff is not an implementation of that paper, but it explores a closely related developer-tooling question.
How do we get the enormous amount of software that already exists from here to there?
I do not want to replace browser agents
Browser interaction is enormously useful. It provides a universal fallback. If a site exposes no structured machine interface, an agent can still use it. If an application has an unusual workflow, the browser may still be the best option.
The objective is not to kill the browser. It is to stop forcing it to be the only interface available to a machine.
The application already knows a lot about itself
A mature application already knows what entities exist, what users can do, which inputs operations require, which routes are authenticated, which actions mutate state, which actions are dangerous, how data relates, what APIs exist and what workflows exist.
We encode this information in code, routing, schemas, types, database models, middleware, permissions, tests and APIs. Then an agent arrives at the rendered website and starts trying to infer some of it again.
The implementation of those layers will change. The idea should not have to.
What comes next
I am deliberately trying not to turn Descuff into a race to support the largest number of standards. There is much more interesting work left in understanding applications correctly.
- Richer semantic understanding of mixed-domain applications.
- Better mapping from implementation details to real user capabilities.
- Deeper structural context.
- Easier integration with the coding agents developers already use.
- Stronger browser and runtime verification.
- More external application testing.
- Eventually supporting frameworks beyond Next.js.
The last decade of web development has been dominated by making applications increasingly pleasant for humans to use. That should continue. Accessibility matters. Semantic HTML matters. Good UX matters. None of this is an argument against the human web.
But AI agents are becoming another consumer of software. And they are not humans.
Maybe the future is not just building agents that become infinitely better at reverse-engineering every application. Maybe part of it is building applications that are better at explaining themselves.
That is the idea behind Descuff. And for now, I am interested in seeing how far it goes.
References
- Google — “Build agent-friendly websites.”
- Song et al. — “Beyond Browsing: API-Based Web Agents,” Findings of ACL 2025.
- Kerboua et al. — “FocusAgent: Simple Yet Effective Ways of Trimming the Large Context of Web Agents.”
- Lù et al. — “Build the web for agents, not agents for the web.”
- Chrome for Developers — WebMCP.
- IETF RFC 9727.
- Cloudflare — Markdown for Agents.
llms.txtproposal.- Descuff Phase 10 external audit.