How to optimize a website for AI assistants: Google's new audit, llms.txt, and what good SEO already covers

In my previous post I showed, on data from 21 domains, that AI assistants send more and more traffic and hand it out under different rules than Google. The natural next question: what exactly should you do on a website to reach for that traffic. The answer is partly boring, because most of the work overlaps with good SEO, and partly fresh, because since May 2026 Google itself measures how ready a page is for AI agents with a dedicated audit. I ran my own sites through it and here is what came out.

Google already grades pages for agents

In May 2026, Lighthouse, the engine behind PageSpeed Insights, got a new category next to performance, SEO and accessibility: Agentic Browsing. It evaluates whether a page can be read and operated by an AI agent acting without a human. The result is not a 0–100 score but a count of passed checks, e.g. 3/3.

Four things are checked:

Audit	What it checks	Why an agent cares
Accessibility tree	Is the page’s semantic structure well-formed	An agent parses structure, not looks; good a11y is its map of the page
Layout stability (CLS)	Do elements jump around after load	A screenshot-driven agent clicks where something was a moment ago
llms.txt	Does a valid descriptive file sit at the domain root	A machine-readable table of contents for the site
WebMCP	Are forms annotated and agent tools registered	The agent calls the site’s function instead of guessing form fields

I ran the audit through the API on my own sites. karol-leszczynski.pl and inkmagnet.com pass the scored part (accessibility tree and CLS) in full, with mobile performance at 96–97/100. The llms.txt and WebMCP audits are informational for now, they don’t lower the result, and both come up empty for me, because none of my 21 domains has an llms.txt. The category is flagged as under development, so the set of checks will change. The direction is unambiguous though: Google now treats agents as the page’s third audience, next to the human and the crawler.

You can check this without the API: type an address into pagespeed.web.dev and the category shows up in the report alongside the others.

Four-fifths of the work you already know from SEO

Before we get to the new parts, one thing needs saying plainly, because the market has started selling “AI optimization” as separate magic. An assistant looking for sources behaves like a very demanding crawler: it fetches HTML, reads the content, judges the fit to the question. Everything that solid technical and content SEO has done for years works for this channel too.

The overlap is concrete: page speed (an agent has a time budget like Googlebot), semantic HTML with a sane heading hierarchy, content available without executing scripts, no text locked inside images, and answering the user’s questions plainly instead of in generalities. If a site has this done, it gets most of “AI optimization” for free. If it doesn’t, no llms.txt will make up for it.

The difference lies in the weighting. Classic ranking leans heavily on domain authority, while an assistant, at the moment of answering, weighs how well the content fits one specific, long question. That is why in the data from my previous post sites weak in Google can get 20–60% of their traffic from AI: the content answered the question even though the link profile barely existed.

What to add on top of good SEO

First, robots.txt. Assistant crawlers (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended) respect the file, and a fair share of sites block them wholesale, sometimes unknowingly, because the rule came with a template or a “protect your content from AI” guide. That is a legitimate business decision, but it has to be made consciously: a site closed to the bots will not be cited or recommended. My domains let all of these bots in, and that is not an accident.

Second, structured data. JSON-LD with Organization, Service, Product, FAQPage or Article types hands the machine facts in a form it doesn’t have to guess from prose: what you sell, for how much, where you operate, which questions you handle. For classic Google this has been standard for years; for a model assembling an answer these are ready-made blocks of fact. Schema won’t replace content, but it removes ambiguity.

Third, llms.txt: a markdown file at the domain root with the site’s name (an H1 heading), a concise description of what the site is, and links to its most important content. Honestly: it is a community specification, and hard evidence that the major providers use it at scale does not exist today. Two things speak for adding it anyway: Lighthouse already audits it, and the cost is half an hour of work. The price is low and the payoff uncertain, but possible.

Fourth, on the horizon, WebMCP: annotations that let an agent call a site’s function (search, an order form) as a tool instead of clicking blind. Today it is an unscored audit, and implementing it makes sense mainly in applications where an agent could genuinely get something done for the user. Worth knowing it exists; not yet worth rebuilding a site for.

How to check whether it works

You measure the effect in two places. Visits: the acquisition report in GA4 with the “session source” dimension filtered to chatgpt.com, copilot.com, gemini.google.com, perplexity.ai. Technical readiness: the PageSpeed report with the Agentic Browsing category, plus a manual test, meaning pasting your own address into an assistant with a question your site should answer and seeing what the model read out of it.

For my own sites this happens as part of ordinary website care, because it is the same set of chores: keeping speed up, HTML correct, structured data in place, and content that answers questions. If you want someone to do it on your site together with the rest of the maintenance, the scope and pricing are on my website care page.

And if you’re wondering whether to pay anyone separately for “AI positioning” at all, I wrote a separate post about that — the short version: first read what exactly you would be paying for twice.