What is llms.txt — and does it actually do anything?
llms.txt is a proposed standard: a Markdown file at your domain root that gives AI models a curated summary of your site and links to its most important pages. It is a community proposal, not an official standard, and no major AI provider has committed to reading it. It is cheap to add and currently unproven.
Updated 19 July 2026
What it is
The idea is straightforward, and borrowed from a file everyone already understands.
robots.txt tells crawlers where they may go. llms.txt is proposed to tell
language models what your site is about — a Markdown file at
https://yourdomain.com/llms.txt containing a short description of the
organisation and a curated list of the pages that matter, so a model does not have
to infer your structure from navigation and guesswork.
A minimal one looks like this:
# Acme Cigars
> A UK online tobacconist specialising in Cuban cigars, established 1994.
## Core pages
- [Shop](https://example.com/shop): Full catalogue
- [About](https://example.com/about): History and credentials
- [Shipping](https://example.com/shipping): UK and international delivery
Some sites also publish llms-full.txt containing the full text of key pages.
The honest status
This is where most write-ups on llms.txt become less than candid, so plainly:
- It is a community proposal, not a W3C or IETF standard.
- No major AI provider has publicly committed to reading it. Not OpenAI, not Google, not Anthropic.
- There is no reliable public evidence that publishing one changes whether an assistant names your brand.
- Server logs showing AI crawlers fetching
/llms.txtprove the file was requested. They do not prove it influenced an answer.
Anyone telling you llms.txt is essential for AI visibility is ahead of the evidence.
So should you publish one?
Probably yes — but for honest reasons, not hopeful ones.
The case for: it costs perhaps thirty minutes. It cannot hurt. If adoption does arrive, you are already there. And writing one is a genuinely clarifying exercise: stating in fifty words what your organisation is and which ten pages matter is a discipline most sites fail, and the answer is useful for your homepage copy, your structured data and your meta descriptions regardless.
The case against giving it any priority: it is unproven, and every hour on llms.txt is an hour not spent on things with demonstrated effect — being readable, being clearly described, and being named on sources assistants actually retrieve.
Our position: publish one, spend half an hour, then go and do something that moves the measurement.
What to do instead, in priority order
- Make sure crawlers can read you at all. A site that requires JavaScript to render its content, or that blocks AI user-agents, has a problem no metadata file solves. This is the single most common blocker we see.
- Structured data on real content. Organization, Product, FAQPage where they genuinely apply.
- A clear, consistent description of who you are, identical across your site, your profiles and your listings.
- Get named on sources assistants trust. This remains the biggest lever and the least automatable.
- Then, sure, llms.txt.
You can check items 1 to 3 on your own site with a free scan — those are exactly the readable signals it measures.
A note on blocking
There is a real decision underneath this one. Some publishers block AI crawlers deliberately, to stop their content training a model that will answer without sending them a visitor. That is a defensible position.
But it is a trade, and it should be made knowingly: a site an assistant cannot read is a site it cannot cite. We have measured sites that block our crawler and then wonder why they are invisible in AI answers. The two facts are the same fact.