llms.txt is a plain Markdown file placed at the root of a website, at yoursite/llms.txt, that tells an AI agent what the site is and which pages matter. It is a proposal, not a standard that every engine follows. Below is what the proposal says, what four AI vendors say on their own pages (read on 9 November 2026), and what we found when we checked our own file against the format.
What is llms.txt?
The llmstxt.org page, written by Jeremy Howard and published on 3 September 2024 (last modified 10 August 2026), calls it "a proposal to standardise on using an /llms.txt file to provide information to help agents use a website". The file is Markdown: a page of text that a person and a language model can both read without parsing HTML menus and scripts.
What goes in the file?
By the proposal, in this order:
- An H1 with the name of the project or site. It is the only required part.
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail.
- H2 sections that hold lists of links. Each link is written
[name](url), optionally followed by a note.
An H2 section called "Optional" holds secondary links that an agent can skip when it needs less context. The file can sit at the site root or at any path inside it, covering the pages under that path.
Do the AI engines read it?
The proposal itself says agents "are expected to view or search" the file. It names no AI assistant or search engine that reads it; the tools it lists generate the file, audit for it or let an agent fetch it. What the vendors say, as of 9 November 2026:
| Vendor | What its own page says about llms.txt |
|---|---|
| AI features and your website does not name llms.txt. It says: "You don't need to create new machine readable files, AI text files, or markup to appear in these features", and that there are no additional requirements beyond being indexed and eligible to show in Search with a snippet. | |
| OpenAI | The crawler overview does not say ChatGPT reads llms.txt. It lists OAI-SearchBot (ChatGPT search), GPTBot (training), ChatGPT-User (user actions) and OAI-AdsBot, and says sites opted out of OAI-SearchBot will not be shown in ChatGPT search answers (though they can still appear as navigational links). OpenAI links its own llms.txt as the index of its documentation. |
| Anthropic | The help article on crawling does not mention llms.txt. It names ClaudeBot, Claude-User and Claude-SearchBot and says they honour robots.txt. |
| Perplexity | The bots page links Perplexity's own documentation index at llms.txt. It describes PerplexityBot and Perplexity-User and recommends allowing PerplexityBot in robots.txt. It does not say the bots read a site's llms.txt. |
So publishing an llms.txt for your own documentation is common, but none of these four pages says its product reads yours. Google says outright that you do not need such a file. We have not tested whether any engine reads ours, and we do not claim it.