Quick answer
An llms.txt file is a plain-text, markdown-style file at the root of your website that lists and summarises your most important pages for large language models. It is a proposal, not an official standard, and there is little evidence that major AI search tools rely on it, so treat it as optional, low-cost housekeeping rather than a ranking tactic.
Last updated: November 2026
Every few months a new file or tag is declared the secret to ranking in AI search. In 2024 the llms.txt proposal became the latest. Many SEO tools now generate one in a click, and some businesses have been told they will vanish from AI answers without it.
The reality is more measured. The idea is sensible, the file is harmless and quick to create, but nobody has shown that it changes whether ChatGPT, Gemini or Perplexity cite you. Knowing that lets you spend your effort where it counts.
This guide explains what llms.txt is, how it differs from robots.txt and sitemaps, what the evidence says, how to create one properly, and what to prioritise first if your real goal is AI visibility.
In this guide
- What llms.txt is
- How it differs from robots.txt and sitemaps
- What the evidence says
- Why you might still create one
- How to create an llms.txt file
- A short example structure
- How to check whether AI crawlers can reach your site
- What matters more for AI visibility
- Common mistakes with llms.txt
- A realistic view of the future
- FAQs
What llms.txt is
The llms.txt proposal, published in 2024, suggests that websites add a file at /llms.txt written in markdown. It contains a short description of the site followed by curated links to key pages, ideally with brief notes. The goal is to help language models quickly understand a site without wading through complex HTML, menus and scripts.
Some sites also publish an expanded version, often called llms-full.txt, which includes the full text of important documentation. It is mostly used by software and developer sites that want AI coding assistants to read their docs reliably.
How it differs from robots.txt and sitemaps
- robots.txt tells crawlers which parts of a site they may or may not access. It is a long-established convention that major crawlers obey.
- An XML sitemap lists the URLs you want search engines to discover and index, with optional metadata.
- llms.txt is a human-readable summary and reading list for language models. It does not block or allow anything, and no major search engine has said it uses it for ranking.
What the evidence says
Google has said it does not use llms.txt for search, and its representatives have compared it to the old keywords meta tag. OpenAI, Anthropic and others have not committed to using it in their consumer search products. Some AI crawlers appear to request the file in server logs, but that is not the same as influencing answers.
Adoption is mainly among documentation and developer sites. Studies looking at large numbers of websites have not found a link between having the file and being cited more often. For a typical local or service business, it is unlikely to make a measurable difference on its own.
Why you might still create one
- It takes under an hour and costs nothing.
- It forces you to decide which pages best represent your business, which is a useful exercise.
- It may help AI agents and developer tools that do look for it.
- It is harmless if kept accurate, and may become more useful if adoption grows.
How to create an llms.txt file
A good file is short and specific. It starts with a heading naming your business and a one- or two-sentence summary, then groups links under clear headings, each with a short description. A typical service business might list its core services, pricing page, location pages, about page, key guides and contact page.
Create it as a plain text file, name it llms.txt, and place it in the root of your domain so it is available at yourdomain.co.uk/llms.txt. On WordPress you can upload it through your hosting file manager, or use an SEO plugin that offers a generator. Keep the descriptions factual and avoid marketing claims.
- Choose ten to twenty pages that best explain what you do.
- Write one line for each describing what the visitor will learn.
- Link only to pages that are live, canonical and indexable.
- Update it when pages move or services change.
A short example structure
Imagine a Leeds accountancy firm. The file would open with its name and a sentence such as “Chartered accountants in Leeds helping contractors, landlords and small limited companies”. Under a heading for services it would list pages such as contractor accounting, landlord tax returns and VAT, each with a short description. Under another heading it would list the fees page, about page and contact page.
The point is clarity. If a person read only this file, would they understand who you are, what you offer and where to learn more? If yes, the file is doing its job.
How to check whether AI crawlers can reach your site
Before worrying about llms.txt, confirm that the basics are in place. Open your robots.txt file and look for rules that block crawlers such as GPTBot, ClaudeBot, PerplexityBot or Google-Extended. Some security plugins and hosting providers add blanket blocks without telling you, and many site owners are surprised to find they have been invisible to these tools for months.
Next, check your server logs or use a log analysis tool to see whether AI crawlers actually visit. If you see regular visits, the content they receive matters, so make sure your key pages return clean HTML with real text rather than content loaded entirely by scripts. Test a few important pages with JavaScript disabled to see what a simple crawler sees.
Finally, make sure your pages carry accurate titles, headings and structured data, and that your sitemap lists the pages you want found. These are the signals both search engines and AI tools use repeatedly, and they remain far more important than any optional summary file.
What matters more for AI visibility
If your aim is to be cited by AI tools, focus first on things that are proven to matter: pages that rank well, content that answers questions clearly and directly, accurate and consistent business information, reviews and mentions on trusted sites, and technical health so that pages can be crawled and read.
Make sure you are not accidentally blocking AI crawlers you want, check your robots.txt, and keep your pages fast and readable without heavy scripts. Our technical SEO service reviews all of this, and a website SEO audit will show whether your site is ready for both search and AI tools.
Common mistakes with llms.txt
- Treating it as a ranking factor or a guarantee of AI citations.
- Linking to redirected, blocked or noindex pages.
- Stuffing it with keywords or promotional text.
- Forgetting to update it when the site changes.
- Paying for a service whose only deliverable is generating the file.
A realistic view of the future
Standards for communicating with AI systems are evolving quickly. It is possible that a version of llms.txt or a similar convention will gain wider support, in which case early adopters lose nothing. It is also possible that AI tools will rely on better signals, such as structured data and on-page content, and the file will fade.
The sensible approach is to be pragmatic. Create a simple file if you like, keep it accurate, and put your main energy into content and trust signals, which have a proven track record in both search and AI results. That way you benefit whichever way the standards evolve.
Technical SEO Services
Crawling, indexing, speed and schema fixed properly.
Frequently asked questions
Does Google use llms.txt?
Google has said it does not use llms.txt for search ranking. Other AI providers have not confirmed that they rely on it either.
Where should the llms.txt file go?
At the root of your domain, so that it loads at yourdomain.co.uk/llms.txt, in plain text or markdown format.
Is llms.txt the same as robots.txt?
No. robots.txt controls crawler access. llms.txt is a summary and reading list intended to help language models understand your site.
Can llms.txt hurt my SEO?
No, as long as it is accurate and does not link to broken or blocked pages. It simply has no confirmed effect on rankings.
