What Is llms.txt and Does It Actually Help AI Search?

This article explains what is llms.txt, how to create one, and, most importantly, what it will and will not do for your visibility in AI answers.

What is llms.txt in simple words?

llms.txt is a plain text file written in Markdown that sits at the root of a website and tells a language model, in a few lines, what the site is about and where the best content lives. The llms.txt proposal was published by Jeremy Howard in September 2024, mainly with developer documentation in mind.

The idea is that an AI agent helping a user can read a short, clean summary instead of parsing menus, scripts and ads. The proposal says the file is mainly meant to be used on demand, when an agent needs information while assisting a user, rather than as a training signal.

The format is simple. Only an H1 with the site or project name is required. After that, the proposal suggests a short blockquote summary, optional notes, and H2 sections that list links with a short description. A section titled “Optional” holds links an agent can skip when it needs less context.

Does Google use llms.txt?

No, Google does not use llms.txt for its AI search features. Google’s guide to AI features and your website says you do not need to create new machine readable files, AI text files or markup to appear in AI Overviews or AI Mode. Google’s John Mueller has also publicly compared llms.txt to the old keywords meta tag, which Google ignores.

That matters because many tool vendors sell llms.txt as a ranking factor for AI search. It is not one for Google. For Google, what decides AI Overview links is the same thing that decides regular results: indexable, helpful pages. My guide on AEO vs SEO explains why those two are mostly the same work.

Which AI tools actually read llms.txt?

Very few AI providers document that they read llms.txt, so you should assume it is not used unless a provider says otherwise in its own documentation. The proposal has been adopted mostly by software documentation sites and by some coding assistants and developer tools that can fetch the file when you point them at it.

The bigger AI search products publish their own guidance, and it is about crawler access rather than llms.txt. OpenAI’s crawler documentation is about allowing OAI-SearchBot. Perplexity documents PerplexityBot. Anthropic documents ClaudeBot, Claude-User and Claude-SearchBot. Google, as covered above, says AI text files are not needed at all. None of those guides tells site owners that llms.txt is required to be cited.

There is also a practical check you can run yourself. Look at your server access logs for requests to /llms.txt and note the user agents. On most small business sites I have checked, the file gets occasional hits from SEO tools and scanners and very little from the AI assistants people actually use. Your logs are the honest answer for your own site, and they are worth more than any vendor claim.

So the realistic view in 2026 is this: llms.txt is a reasonable convention for documentation heavy sites, a harmless extra for everyone else, and not a lever for Google rankings or AI Overview links.

How do I create an llms.txt file?

You create an llms.txt file by writing a short Markdown document and uploading it to your site root so it loads at yourdomain.com/llms.txt. On a WordPress site you can upload it by FTP or through your host’s file manager, or use a plugin that generates it.

1. Write an H1 with your business or site name.
2. Add a one or two sentence blockquote describing what you do and for whom.
3. Add an H2 such as “Services” and list 5 to 10 key pages as Markdown links, each with a short note.
4. Add an H2 “Guides” for your best evergreen articles.
5. Add an H2 “Optional” for secondary pages like your about page or case studies.
6. Upload the file to the site root and open the URL in a browser to confirm it returns plain text with a 200 status.
7. Update it when you add or remove important pages, the same way you would update a sitemap.

Keep it factual. Do not paste keyword lists or marketing claims into it; an agent reading it is looking for orientation, not persuasion.

Two common mistakes I see when reviewing these files: linking to pages that redirect or return 404, and listing 200 URLs so the file becomes a second sitemap. Both defeat the purpose. Link only to final, indexable URLs, and keep the main sections to the pages a new customer or a developer would genuinely need first. If your site is on WordPress, also confirm that a caching or security plugin is not serving the file as HTML or blocking it for unknown user agents.

Is llms.txt the same as robots.txt?

No, llms.txt and robots.txt do different jobs: robots.txt tells crawlers which URLs they may or may not request, while llms.txt only offers a summary and links. An llms.txt file cannot block or allow any bot. If you want to control AI crawlers such as GPTBot or ClaudeBot, that still happens in robots.txt, which you can check with my robots.txt tester.

Should I add llms.txt to my website?

Add llms.txt only if it takes you less than 30 minutes and you treat it as a convenience for AI agents and developer tools, not as an SEO tactic. For a documentation site or a software product with an API, it can genuinely help coding assistants find the right pages. For a local business site, the benefit is speculative.

What I tell clients is to finish the work that is known to matter first: fix indexing problems, allow the search crawlers you want, write answer-first pages and build real authority. That is the core of my GEO services, and llms.txt is a small optional extra at the end of the list, never the headline.

What does a good llms.txt file look like?

Frequently asked questions