Short answer. llms.txt is a markdown file at your site root (/llms.txt) that gives large language models a compact, machine-readable map of your content: a short project summary plus a curated list of your most important pages as links. It does not replace robots.txt or sitemap.xml — it helps AI agents understand your site structure faster and cite your material more accurately.
What llms.txt is and why it matters
The llms.txt format is a way to tell AI systems which content on your site is the most valuable, in a format that is easy for an LLM to process. A normal HTML page is full of navigation, ads, scripts and styling — all noise to a model. The llms.txt file strips the noise and serves the essence as clean markdown.
- Context efficiency. The model spends fewer tokens parsing a page.
- Citation accuracy. AI assistants (ChatGPT, Claude, Perplexity) are more likely to link to the right page.
- Priority control. You decide which sections to surface first.
llms.txt is a hint, not an access directive. It answers "what do you have that matters", not "where am I allowed to go". Access is governed by robots.txt.
File structure
The file consists of an # heading with the project name, a short summary in a > blockquote, and topical ## sections with link lists in the form - [Name](url): description.
# Enterno.io
> Uptime monitoring and website analysis platform: 48+ free tools covering HTTP headers, SSL, DNS, and AI-readiness checks.
## Tools
- [AI-readiness check](https://enterno.io/ai-check): analyzes llms.txt, schema, content negotiation
- [SSL checker](https://enterno.io/ssl): certificate expiry and chain
- [DNS Lookup](https://enterno.io/dns): A, AAAA, MX, NS, TXT records
## Documentation
- [API v4](https://enterno.io/api/docs): REST API for all tools
- [robots.txt guide](https://enterno.io/articles/robots-txt-guide): crawler directives
## Optional
- [Blog](https://enterno.io/articles): SEO and monitoring articles
The ## Optional section signals to the model that these links can be skipped under a limited context budget.
Where to put it and how to serve it
The file must be reachable at https://your-domain/llms.txt with the correct MIME type. Many sites also serve /llms-full.txt with a full catalog of every URL, plus per-cluster versions such as /llms-glossary.txt.
Enterno.io, for example, serves llms.txt, llms-full.txt, per-cluster files and a markdown version of every page — and it honors the Accept: text/markdown header. More on that in the content negotiation for AI article.
Table: related files
| File | Purpose | Audience |
|---|---|---|
| llms.txt | Map of key content in markdown | LLMs / AI agents |
| robots.txt | Access rules for crawlers | Search and AI bots |
| sitemap.xml | Full list of URLs for indexing | Search engines |
| .well-known/ai-plugin.json | AI plugin descriptor | AI platforms |
How to validate llms.txt
After publishing, confirm the file is served from the root path, contains valid markdown, and that every link works. To check your whole site's AI readiness — presence of llms.txt, structured data, correct content negotiation — use a free checker.
Don't dump utility or private pages (login, cart, admin) into llms.txt. The file is a showroom of your best content, not a mirror of your sitemap.
FAQ
Is llms.txt mandatory?
No, it's an optional format. But it's free, simple, and improves the odds that AI assistants cite your site correctly.
Does llms.txt replace sitemap.xml?
No. A sitemap helps search engines index everything; llms.txt is a short, curated map of priority content for LLMs. See the sitemap.xml guide.
How often should I update it?
Whenever you ship important new sections. Many teams generate llms.txt automatically from a database via cron, so it always stays fresh.
Do search engines see it?
The format targets LLMs, but nothing stops search bots from reading it. It has no direct effect on classic search ranking.
How is llms.txt different from robots.txt for AI?
robots.txt controls bot access (may/may not crawl), while llms.txt describes content (here's what matters). For AI bot directives, read the robots.txt and AI bots article.