What is llms.txt and does your site need one?
By haylow.ai editorial team
In short
llms.txt is a proposed convention: a plain-text Markdown file at the root of a site that summarises what the site is and links to its most useful pages, written for AI systems to read. It is not a standard that every assistant honours.
What is llms.txt?
llms.txt is a plain-text file, written in Markdown, placed at the root of a website. It gives a short description of the site and lists links to the pages most useful to a language model, so a system reading it does not have to work that out from navigation and scripts.
The idea is similar in spirit to a sitemap, but written for reading rather than for crawling. A sitemap lists every URL. An llms.txt file lists the few that matter, with a sentence about each.
Do AI assistants actually use it?
There is no guarantee. llms.txt is a proposed convention, and we are not aware of a published commitment from the major assistants that they read it for ranking or citations. Treat it as low-cost groundwork, not as a lever you can measure on its own.
If you publish one, watch your own results rather than assume an effect. The reliable way to know whether anything changed is to track how often assistants name and cite your brand before and after.
How is it different from robots.txt and sitemap.xml?
robots.txt says which crawlers may fetch which paths. sitemap.xml lists all indexable URLs for search engines. llms.txt is a short, human-readable summary and reading list. They do different jobs, and llms.txt does not grant or block access.
robots.txt
Access rules for crawlers, including AI crawlers such as GPTBot and ClaudeBot.
sitemap.xml
A complete machine-readable list of the URLs you want indexed.
llms.txt
A short curated summary and set of links for AI systems to read.
What should an llms.txt file contain?
A title, a one or two sentence description of what the site is, and grouped links to your most useful pages, each with a short note. Keep it factual and current, and generate it from your real page list so it never drifts.
haylow.ai publishes its own llms.txt at /llms.txt. It is generated per request, so new blog posts and comparison pages are listed without a redeploy.
To check whether your own site is set up for AI crawlers, including robots.txt access and llms.txt, use the free AI visibility checker at /tools/ai-visibility-checker.
Key takeaways
- llms.txt is a proposal, not a guaranteed ranking or citation factor.
- It is a short Markdown file at /llms.txt: a summary plus links to your best pages.
- It does not replace robots.txt, which controls crawler access.
- It is cheap to publish and keep current, so many sites add one, but do not expect it to move results on its own.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal. Adoption varies and no assistant is guaranteed to read it, so it should not be treated as a ranking factor.
Will llms.txt make ChatGPT mention my brand?
Not by itself. Assistants decide what to mention from many signals, including the sources they can retrieve. Track your brand's mentions to see whether any change helps.
Should I block AI crawlers instead?
That is a business decision. Blocking crawlers in robots.txt can stop them fetching your pages, which can reduce how often you are cited. If you want to be cited, allow them.
Where does the file go?
At the root of your domain, so it is served at yourdomain.com/llms.txt, as plain text.
See what AI says about your brand
Add one domain and get your first visibility report across five assistants. Free to start, no card required.