What is llms.txt?
llms.txt is a Markdown file at a website's root or under a subpath that uses a heading, a summary and link lists to tell large language models and AI agents what the site does and where its important content is.
Agents read this compact index first and follow links for detail only when needed. Jeremy Howard proposed it in September 2024; it remains a proposal open to community input.
What the file contains
Under the v2 proposal, a file contains the first four parts below, in this order; the fifth is advice for the site's other pages.
| Part | Content |
|---|---|
| H1 | Site or project name; the only required part. |
| Blockquote | A short summary with the key information needed to understand the rest of the file. |
| Details | Optional unheaded paragraphs or lists with more detail and how to read the links. |
| H2 file lists | Markdown link lists under H2 headings, each link optionally followed by notes. A section named Optional holds secondary links that can be skipped. |
| .md pages | Markdown versions of important pages, with .md appended to the URL or replacing the extension. |
Version 2 and llms-full.txt
The proposal page is labeled v2, last modified August 10, 2026. Per its change notes, a file covers the pages under its path and the most specific file applies; rel="alternate" and rel="describedby" links should point to a page's Markdown version and its covering llms.txt; and v1's context-expansion tooling has left the proposal, so the Optional section no longer has a special role.
The common llms-full.txt is not part of the v2 format; it is a practice of some platforms. Mintlify, for example, combines an entire documentation site into one llms-full.txt.
llms.txt vs robots.txt and sitemap.xml
llms.txt is not access control. robots.txt tells automated tools what access is acceptable; llms.txt is a guide an agent reads on demand, when it needs information while helping a user. It cannot allow or block any crawler, so AI crawlers are still managed with robots.txt (see the AI crawler policy guide). A sitemap.xml lists all indexable pages; llms.txt is a curated overview for LLMs.
Does anyone read it? Official positions and data
Google Search Central's generative AI optimization guide names llms.txt: Google Search itself does not use such files, and maintaining one neither helps nor harms visibility or rankings in Google Search. Its AI features page adds that appearing in AI Overviews or Google AI Mode needs no new machine-readable or AI text files.
Chrome's Lighthouse checks llms.txt in its agentic browsing audits, since without it agents may spend more time crawling a site; a missing file is marked Not Applicable because providing one is currently optional.
An Ahrefs study published in June 2026 analyzed 137,210 domains using Ahrefs Web Analytics with traffic in May 2026: 28% had an llms.txt file, and among the roughly 38,000 domains with a valid file, 97% received no requests for it all month. AI bots made zero requests for llms.txt files that did not exist. Ahrefs notes its sample skews technical and SEO-aware, so 28% is an upper bound; a fetch is not a read, so every figure is a ceiling on actual use.
In Mintlify's test of 2,400 runs of Claude Code and Codex on 20 Mintlify documentation sites, adding a link to llms.txt on pages cut dead URLs by about 90%, while answer accuracy varied little across formats. Every site Mintlify hosts publishes an llms.txt, so Mintlify has a stake in the result.
Scope and limits
llms.txt is not a ranking signal and is not guaranteed to be read by any AI service; there is no evidence yet that publishing one leads to citations in AI Overviews or ChatGPT. The proposal says it is used most for software documentation, where coding agents follow it to find API references and tutorials. Mintlify's test also covered coding agents and technical docs, so it does not transfer directly to a typical brand website.
If coding agents read your docs, or agents search and act on your site for users, publishing one is cheap, but link to it from your pages so agents are more likely to find it. This falls under AX (agent experience). For Google rankings or AI Overviews, start with robots.txt, the sitemap and page content.
An outdated or tampered file misleads every agent that reads it; Ahrefs recommends version-controlling it like code and restricting who can edit it. The LS-Labs research library has no observations yet of how llms.txt files are read.
Sources and scope
- Jeremy Howard: The /llms.txt file, v2
- Jeremy Howard: Changes, v2 (August 2026)
- Mintlify docs: llms.txt
- Louise Linehan, Ahrefs (2026): llms.txt study of 137K sites
- Google Search Central: Optimizing for generative AI features
- Google Search Central: AI features and your website
- Chrome for Developers: Lighthouse llms.txt audit
- Aadit Shah, Mintlify (2026): Docs URL Benchmark
Sources as checked for the source-language entry on 2026-10-26; this English edition adds no new claims.