What Is llms.txt? The 2026 Spec, a Real Example, and Who Reads It

llms.txt is a proposed Markdown file, served at yoursite.com/llms.txt, that gives AI agents a short summary of your site and a curated list of links worth reading. Google says its Search ignores the file, and no AI search engine or assistant documents reading it to decide what to show or cite.

So what is llms.txt good for? Mainly coding agents and developer tools that choose to fetch it, on documentation sites. This guide covers the spec as revised in August 2026, a complete example file you can copy, what server logs show about who actually requests it, and how to decide whether your site needs one.

What llms.txt is for

The llms.txt proposal was published by Jeremy Howard on September 3, 2024. Its pitch: web pages wrap their information in navigation, ads and JavaScript, so an agent is better served by "concise, expert-level information gathered in a single, accessible location."

The intended reader is an agent working on a task right now, not a crawler building an index. The spec says it expected llms.txt to be "mainly useful for inference rather than training," and that agents should "view or search llms.txt to find the information they need, then follow the relevant links." It also says the files "are used most heavily for software documentation, where coding agents follow them to find API references and tutorials."

The AI labs publish llms.txt files for their own developer docs: OpenAI, Anthropic, Google's Gemini API and Perplexity all serve one (we requested each in October 2026 and got HTTP 200). Publishing a file for your own docs is not the same as reading everyone else's, and none of them says its search product does the second.

What changed in llms.txt v2 (August 2026)

The spec was revised in August 2026, after two years of adoption, and most of the guides we reviewed still describe v1. The changes page lists what moved:

The llms.txt format, section by section

A file that follows the spec contains these parts, in this order:

1. An H1 with the name of the site or project. This is the only required section.
2. A blockquote with a short summary holding the key information needed to understand the rest of the file.
3. Zero or more detail sections of any Markdown type except headings: paragraphs, lists.
4. Zero or more H2 sections, each a "file list." Every item is a Markdown link, [name](url), optionally followed by a colon and a note.

An optional byte-order mark may come first. The file can live at the root (/llms.txt) or in a subfolder such as /docs/llms.txt, where it covers only that folder. No other heading levels belong in the file.

A complete llms.txt example

Here is a spec-correct file for a fictional B2B software company. Copy the structure, replace the facts, and keep every number identical to what your own pages say.

# Example Analytics

> Example Analytics is a product analytics platform for B2B SaaS teams. It records in-app events and shows which features drive retention. Founded in 2019, headquartered in Austin, Texas.

Facts in this file are current as of October 2026. Prices are in US dollars, billed monthly.

- Plans: Starter $39/month, Team $119/month, Enterprise priced on request
- Integrations: Segment, Snowflake, BigQuery, HubSpot
- Example Analytics is not affiliated with Example Corp

## Docs

- [Quickstart](https://www.example.com/docs/quickstart.md): Install the SDK and send a first event
- [API reference](https://www.example.com/docs/api.md): REST endpoints, authentication and rate limits

## Product

- [Pricing](https://www.example.com/pricing.md): Every plan, its limits and what it includes
- [Security](https://www.example.com/security.md): SOC 2 report, data residency and subprocessors

## Optional

- [Changelog](https://www.example.com/changelog.md): Release notes since 2023

Why each part is there:

• The blockquote carries the facts an assistant most often gets wrong about a company: what it is, when it was founded, where it is based.
• The dated details block tells an agent how old the facts are. Prices change; a file with no date gives no way to judge.
• The disambiguation line handles a shared name. If your brand is easy to confuse with another company, say so in plain words.
• The links point to Markdown versions (.md) where you have them, because the spec says links "should therefore point to LLM-friendly content."
• The Optional section holds links an agent can skip when it needs a shorter context.

Under v2 you can also point each page at its Markdown version and its covering llms.txt with one HTTP header, set in your server or CDN without touching the pages:

Link: </docs/quickstart.md>; rel="alternate"; type="text/markdown", </llms.txt>; rel="describedby"

How to create an llms.txt file

1. Decide the scope. A root file for the whole site, or a /docs/llms.txt for documentation only.
2. Write the H1 and blockquote. Name, one-sentence description, and the two or three facts you most need repeated correctly.
3. Add a dated details block. Plain paragraphs or a list, no headings.
4. Add H2 file lists pointing to the pages that answer real customer questions: pricing, security, docs, policies.
5. Publish it as plain text at the path you chose. The AI labs' own files are served as text/plain or text/markdown.
6. Confirm it loads. Run curl -sI https://www.example.com/llms.txt and check for a 200 status and a text content type, not an HTML error page.
7. Test it the way the spec suggests: "asking an agent questions about your content, giving it only your llms.txt as a starting point."

Several platforms generate the file automatically. The spec's integrations list includes Mintlify, GitBook, Yoast SEO, AIOSEO and Wix. If yours does, read what it generated: an auto-built file can list pages you would never put in front of an agent.

Does llms.txt work? What the evidence says

It depends on which job you mean. For agents reading documentation, coding agents do fetch it: Claude-Code was one of the top AI requesters in the Ahrefs logs below, though "fetched" does not mean "read," and whether reading it improves what they do has not been measured. For being cited by AI search, there is no evidence it does anything, and the one platform that has spoken says it ignores the file.

Google's statement is in its AI optimization guide: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities)." Google added a clarifying note on June 15, 2026: the files won't negatively or positively affect visibility, and it is fine to keep them for other systems that use them.

The vendor pages are OpenAI's crawler documentation, Anthropic's crawler article and Perplexity's crawler page. The Lighthouse audit sits under "agentic browsing" and makes no ranking claim; its stated benefit is that without the file "agents may spend more time crawling the site."

What the server logs show

Ahrefs studied 137,210 domains with traffic in May 2026, about 38,000 of which had a valid llms.txt. Of those files, 97% received zero requests that month.

The requests that did arrive skew away from AI search. SEO audit tools made 21.7% of them, and AI retrieval bots, the kind that fetch pages for answers, made 1.1%. GPTBot was the most frequent AI fetcher at 4.51%, and Claude-Code, Anthropic's coding agent, out-fetched every other AI retrieval, assistant and training bot but two, which fits the coding-agent use the spec describes.

Ahrefs' own caveats are worth repeating: its customers skew technical, so real adoption is likely lower, and "fetched" does not mean "read." The sample of files that got any traffic is small, about 1,100 domains.

The fair verdict: the defensible claim is "no engine documents consuming it and no evidence exists," not "proven useless everywhere." Google's statement covers Google only, and the coding-agent use is real.

llms.txt vs robots.txt vs sitemap.xml

The three files are often lumped together. They do different jobs, and only one of them can remove you from an AI answer.

robots.txt is the one with documented consequences. OpenAI says sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers," and Meta says "allowing Meta-WebIndexer in your robots.txt file helps us cite and link to your content in Meta AI's responses" (Meta's crawler documentation).

Even robots.txt is necessary, not sufficient. Perplexity says its user-triggered fetcher "generally ignores robots.txt rules," and OpenAI's ChatGPT search help page says eligibility also requires that your "website host or content delivery network allows traffic" from its crawler. A firewall can block a bot your robots.txt allows.

Should you add llms.txt to your website?

Whatever you decide, the file is one more place your facts are published. A file that says $39 while your pricing page says $49 is a contradiction you wrote yourself.

llms.txt not working? Check these

• The URL returns 200 with text, not your HTML 404 page. Ahrefs filtered out files that were HTML error pages served at the llms.txt path.
• Your bot protection lets it through. The file sits behind the same CDN rules as every other URL, so an agent blocked from your site is blocked from it too.
• You are looking for the right signal. No AI answer reports that it used your llms.txt. The only evidence of use is your own server logs: filter requests to /llms.txt by user agent.

If AI gets your brand wrong, fix these first

1. robots.txt, per retrieval bot. Check OAI-SearchBot, PerplexityBot, Claude-SearchBot and Meta-WebIndexer individually, not one blanket rule.
2. Firewall and CDN access. Request a page as each crawler and look for 403 or challenge pages. CDN defaults can change without you touching anything; see our breakdown of the Cloudflare AI crawler default.
3. Your own facts. Pricing, founding year, headquarters and leadership should read the same on every page, in your schema, and in your llms.txt if you have one.

The same evidence standard applied to structured data is in our guide to schema markup for AI, and the wider list of popular tactics is in our fact-check of GEO strategies. To see what ChatGPT and Google AI Overviews say about you right now, try the free AI visibility checker.

Frequently asked questions

Is llms.txt still used?

Yes, by a narrow audience. Coding agents and developer tools fetch it from documentation sites, and the spec was revised in August 2026. But Ahrefs found 97% of valid files got zero requests in May 2026, and no AI search engine documents reading it.

Does llms.txt really work?

For agents reading docs, coding agents do fetch it, but fetched does not mean read, and no one has measured whether it changes their output. For AI search visibility, there is no evidence: Google says its Search ignores the file, and OpenAI, Anthropic and Perplexity document nothing about reading other sites' llms.txt.

Is llms.txt mandatory?

No. It is a community proposal, not a standard any engine requires, and Chrome's Lighthouse calls it "optional at the moment." Your robots.txt and firewall settings matter far more to whether AI can reach you.

Does Google use llms.txt?

Not for Search. Google's AI optimization guide says Google Search "doesn't use" AI text files and ignores them, and that publishing one will neither help nor hurt. Google adds that it is fine to keep the file for other systems that use it.

What is the difference between llms.txt and llms-full.txt?

llms.txt is an index: a summary and links. llms-full.txt is a convention from documentation platforms such as Mintlify that "combines your entire documentation site into a single file." It is not part of the llmstxt.org spec.

To check whether AI crawlers can reach your site at all, which matters more than any text file, run a free 99Visibility audit.