llms.txt probably is not doing what you think
Publishing llms.txt is a fifteen-minute job with an appealing story attached. The measurements do not support the story.
The pitch is reasonable: give language models a clean plain-text map of your site and they will understand and cite you better. It costs almost nothing and it feels like the obvious modern equivalent of a sitemap.
Then you look at whether anything fetches it.
What the measurements show
- A server-log study across roughly 900 domains, running from late 2025 into 2026, recorded around 1,200 requests to
/llms.txt. Not one came from a frontier AI crawler. The majority was a commercial data aggregator, most of the rest was human browser traffic, and the remainder was security scanners. - A citation study across roughly 300,000 domains found no association between publishing the file and being cited in AI answers. The prediction model got better when the variable was removed, which is what noise looks like.
- Google's own documentation states the file has no effect on rankings or AI summaries.
- No major model provider has publicly committed to using it as a signal.
Why the correlation looked convincing
Sites that publish llms.txt tend to be sites that pay attention: they have structured content, they publish regularly, they are technically well maintained. Those things drive citations. The file arrives alongside them and takes the credit.
It is worth being honest about how easy this is to fall for. Comparing three of your own sites, finding the one with the file has far more citations than the two without, and concluding the file did it, is a sample of three with an obvious confounder. Convincing, and not evidence.
Should you still publish one?
Probably, but for the right reason and with the right expectation. It is cheap, it is harmless, and it has a genuine use: an agent or a person already on your site reading a compact factual summary. Documentation sites and developer tools get real value from that. Adoption is far higher in that segment than on the web generally.
What it is not is a discovery mechanism. Nothing goes looking for it.
Make it accurate and keep it current, because the failure mode is worse than the upside. A stale llms.txt quoting last year's prices to something that does read it is more damaging than not having one. Check every URL in it resolves.
What the same studies say does correlate
A large brand-level analysis found the strongest associations with AI visibility were mentions of the brand elsewhere on the web, and mentions in video content. Domain authority and raw backlink counts correlated weakly. The authors are explicit that correlation is not causation.
The practical reading is that being talked about by other people matters more than markup you control. That is slower, harder and less satisfying than adding a file, which is exactly why the file is so popular.
The general habit
When a new optimisation file or tag appears, ask one question before adopting it: has anyone measured whether the thing it is aimed at actually requests it? Server logs answer that in an afternoon, and they answer it better than any amount of published advice.
Need help with any of this?
These notes are free and always will be. If you would rather someone just set it up, or you are stuck on something similar, get in touch at hello@opsira.io.