llms.txt is a proposed, plain-text file placed at the root of a website that summarises the site and lists its most useful pages for large language models. It is an emerging convention, and not every AI system reads it. Adding one is cheap and harmless if kept accurate, but it should not replace clear pages or be expected to guarantee visibility.
- llms.txt is a proposal, not a formal standard enforced by every AI provider.
- It is a simple text file that describes your site and links to key pages.
- It supplements, and does not replace, readable pages and robots.txt.
- Keep it accurate and short; do not expect guaranteed results.
What is llms.txt?
The idea is simple: a text file at an address like yoursite.com/llms.txt that tells language models, in a compact and readable form, what your site is about and where its important content lives. It is written in a lightweight Markdown style, with a title, a short description and organised lists of links with one-line explanations.
It was proposed as a convenience for AI systems that have limited space to read a page and would benefit from a clean map rather than a crawl of menus, scripts and adverts. As with any proposal, adoption is voluntary, and each AI company decides for itself whether to use it.
How is it different from robots.txt and sitemaps?
Robots.txt tells crawlers what they may and may not fetch. A sitemap lists the URLs you want search engines to know about. llms.txt does neither of those jobs: it is a curated guide that explains your site to a reader, picking out the pages that best answer common questions and describing them in plain language.
Because they serve different purposes, they work side by side. You still need correct access rules, still benefit from a sitemap and still need pages that are easy to read. llms.txt is an optional extra on top of that foundation.
- robots.txt: rules about what crawlers may fetch.
- sitemap: a list of pages for search engines to discover.
- llms.txt: a human-readable guide to your best content.
- Your pages themselves: still the real source of information.
What goes inside an llms.txt file?
Begin with the name of the business and a two or three sentence summary stating what you do, for whom and where. Follow with sections grouping your key pages, such as services, guides, pricing policy, support and contact. Under each, list the link with a short, factual description.
Be selective. The point is to highlight the pages that best represent you, not to dump every URL. Use plain language, avoid marketing superlatives and include only facts that are true today, since the file is only useful while it remains accurate.
- A title and short factual summary of the business.
- Grouped links to services, guides and policies.
- One-line plain descriptions for each link.
- Contact and location details that match your site.
Do AI systems actually read it?
Honestly, support is uneven and hard to verify from outside. Some tools and developers use llms.txt files, particularly in documentation and coding contexts, but there is no public guarantee that the major assistants consult it when answering questions about your business. Treat claims otherwise with scepticism.
That uncertainty is a reason to keep expectations modest, not a reason to ignore it. The effort is small, and the same exercise of choosing and describing your best pages tends to improve your site architecture anyway.
Should my website have one?
For most businesses a short, accurate file is a reasonable low-cost addition, especially if you publish documentation, guides or many services. It is worth less if your site has only a handful of pages that are already clear, and it is certainly not worth paying a large fee for.
Do the fundamentals first: readable pages, accurate structured data, consistent business facts and deliberate crawler access. Add llms.txt afterwards, and put a reminder in your content review to update it whenever services or key pages change.
How do I create and maintain one?
Write the file by hand or generate it from your site structure, save it as plain text and upload it to the site root so it loads at the expected address. Check that it opens in a browser and that every link works and reflects the live URL.
Review it whenever you launch or retire a service, change contact details or restructure pages. An outdated file points systems at stale information, which is worse than having none.
Step by step
- Choose your key pages. Select the pages that best explain your services, policies and expertise.
- Write the summary. Describe the business in two or three factual sentences.
- List and describe links. Group links by topic with a short plain description for each.
- Upload to the root. Save as llms.txt and place it at the top level of your site.
- Review regularly. Update the file whenever services, URLs or contact details change.
Frequently asked questions
Is llms.txt an official standard?
No. It is a community proposal that some tools support. There is no universal requirement for AI providers to read it.
Will llms.txt improve my Google rankings?
There is no evidence from search engines that it affects ordinary rankings. Its purpose is to help AI systems that choose to read it.
Can llms.txt block AI crawlers?
No. Blocking and permission rules belong in robots.txt and server settings. llms.txt only describes content.
How long should the file be?
Short and selective. List the most important pages with brief descriptions, not an exhaustive inventory of every URL.
Do I need a developer?
Not necessarily. It is a plain text file that can be uploaded through hosting tools, though a developer can automate updates.
Need help with this? See our GEO — Generative Engine Optimisation service or talk to Yash Parikh.