Is llms.txt the same as robots.txt?

No. robots.txt is a long-standing convention that tells crawlers which paths they may or may not fetch, and most crawlers respect it. llms.txt is a newer proposal that summarizes your key pages for language models. It grants or blocks nothing. If you want to control AI crawler access, robots.txt and your CDN's bot settings are the tools to use, not llms.txt.

Does Google read llms.txt?

Google hasn't said it uses llms.txt for Search, and its AI features documentation says no special files are needed to appear in AI Overviews or AI Mode. Those features draw on Google's normal index, built by Googlebot from your regular pages. Having the file won't hurt you, but don't expect it to change your visibility in Google's AI results.

Where should the llms.txt file go?

Put it at the root of your domain, so it loads at yourdomain.com/llms.txt, the same place robots.txt lives. Serve it as plain text. On Next.js you can place it in the public folder. On Framer and Webflow, check whether the platform supports custom root files on your plan, since that varies, or host it through your CDN or a redirect.

What is llms-full.txt?

llms-full.txt is an optional companion file that contains the full text of the pages listed in llms.txt, joined into one Markdown document. It's mainly useful for documentation sites, because a developer can hand the whole file to a coding assistant in one go. For a marketing site it's usually overkill, and it can go stale quickly when pages change.

Can llms.txt hurt my SEO?

There's no known way for llms.txt to hurt your search rankings. It's a static file that search engines don't treat as a ranking signal. The real risk is indirect: a team spends time on it and believes the AI visibility job is done, while the pages themselves stay hard to crawl or never answer the question buyers ask.

All posts

Web design & development

What Is llms.txt and Does It Actually Help AI Visibility?

What llms.txt is, how the file is structured, what 2026 studies found about AI citations, and the few cases where it's still worth adding to your site.

By Mehdi HassanOct 29, 20267 min read

The short answer

llms.txt is a plain Markdown file at the root of a website that lists its most useful pages for large language models. It's a proposal, not a standard. 2026 studies of hundreds of thousands of domains found no measurable lift in AI citations from having one, so treat it as optional, cheap hygiene, mainly useful for developer docs.

Key takeaways

llms.txt is a community proposal, not a web standard that any major AI engine has committed to.
SE Ranking's 2026 study of 300,000 domains found no measurable citation lift from having the file.
Google says its AI features need no special files or markup beyond normal SEO.
It earns its keep on developer docs, where coding tools and agents read it on purpose.
Rendering, crawler access and answer-first pages matter far more for AI visibility.

Part of How to Build an AI-Search-Ready Website: Structure, Schema and Crawlability

What is llms.txt?

llms.txt is a Markdown file served at yoursite.com/llms.txt that gives language models a short, curated map of a website: what it is, and which pages hold the useful information. The idea is that a model with a limited context window shouldn't have to wade through navigation, scripts and cookie banners to find your docs or pricing.

It was proposed by Jeremy Howard of Answer.AI as an open community convention. It sits alongside robots.txt and sitemap.xml in spirit, but it does something different: robots.txt says what crawlers may fetch, the sitemap lists every URL, and llms.txt tries to say what matters. It's one small piece of an AI-search-ready website, and as you'll see below, a minor one.

File	Purpose	Who reads it
robots.txt	Rules for which bots may crawl which paths	Every well-behaved crawler, by long convention
sitemap.xml	A complete list of URLs and their last-modified dates	Search engines, as a discovery hint
llms.txt	A curated, human-written summary of key pages	Some AI tools and agents, by choice; no major engine has committed to it

What does an llms.txt example look like?

An llms.txt file follows a simple Markdown layout: a top-level heading with the site or project name, a one-paragraph summary in a blockquote, then sections of links, each with a short note on what the page covers. There's an optional companion, llms-full.txt, that puts the full text of those pages into one file.

First line: a level-one heading with your company or product name.
Summary: one blockquote sentence that says what you do and for whom, written like a definition.
Sections: level-two headings such as Docs, Pricing, Guides, each followed by a list of links with a one-line description.
Optional section: links a model can skip if it's short on space, such as the changelog or press page.

Hypothetical example: what a B2B SaaS llms.txt might contain

For example, suppose a payroll startup called Northpay writes one. The heading is Northpay. The summary says Northpay is payroll software for US companies with 10 to 200 employees. A Product section links the features page, the pricing page and the security page, each with a single descriptive line. A Docs section links the API reference and the getting-started guide. An Optional section links the blog index and the careers page.

That's the whole file. It takes under an hour to write, and the hard part isn't the syntax, it's deciding which ten pages actually describe the business. If you can't pick them, that's a sign your site's structure needs work before any file will help.

Does llms.txt work for AI search visibility?

On the evidence available in 2026, llms.txt does not measurably improve how often a site is cited by AI search engines. SE Ranking's 2026 study of 300,000 domains found no measurable citation lift for sites that had the file, and a separate 2026 analysis from MaxAEO reached the same conclusion.

Google has been direct about it too. Its documentation on AI features in Search says there are no extra requirements to appear in AI Overviews or AI Mode, and no special markup or files are needed beyond normal SEO. AI Overviews pull from Google's regular index, which is built by Googlebot, not by reading llms.txt.

Why doesn't llms.txt help AI citations?

llms.txt doesn't help citations because the engines that cite websites don't need it to find or understand your pages. Answer engines retrieve pages from their own indexes or live fetches, then quote passages that answer the user's question. A summary file doesn't change what's on those pages.

Discovery is already solved. Crawlers find pages through links and sitemaps. A second list adds little.
Citation is passage-level. Engines quote a specific paragraph or table row, and that has to exist on the page itself. See how to structure pages so AI engines quote them.
No engine has committed to it. Without a major engine reading the file as part of ranking or retrieval, there's no mechanism for it to help.
Trust comes from elsewhere. Engines weigh the source's authority and how well a passage answers the query, not whether the site published a guide for bots.

When is llms.txt worth adding?

llms.txt is worth adding when people use AI tools to read your content on purpose, which mostly means developer documentation. Coding assistants and agents that a developer points at your docs can use a clean index and a full-text file to answer questions about your API without scraping a JavaScript-heavy docs site.

Pros

Cheap: an hour or two to write and a static file to host.
Genuinely useful for API and SDK docs that developers feed into coding tools.
Forces a useful exercise: naming the ten pages that define your business.
No known downside for search rankings.

Cons

No measured lift in AI citations in the 2026 studies.
Another file to keep in sync when pages change, or it goes stale and misleads.
Easy to mistake for real AEO work and skip the fixes that matter.
Not a standard, so tools may ignore it or change how they treat it.

For a typical startup marketing site with a homepage, product pages, pricing and a blog, we'd file it under nice-to-have. If you're debating it, the full pros and cons are in our piece on whether your website should have an llms.txt.

What should you fix before llms.txt?

Fix the things that decide whether AI engines can read and quote your site at all. These have clear mechanisms behind them, and each one can block citations on its own, which llms.txt can't do.

Rendering. Make sure your main content is in the HTML the server sends. Many AI crawlers don't run JavaScript, so a client-rendered React site can look empty to them. Read why server-side rendering matters for AI crawlers.
Crawler access. Check robots.txt and your CDN's bot settings, so OAI-SearchBot, PerplexityBot and Googlebot can reach your public pages. Our guide to allowing or blocking AI crawlers has the rules.
Answer-first pages. Open each key page and section with a direct, quotable answer, and use real tables for comparisons and prices.
Clear entities. Use one consistent company name, a clear about page and descriptive titles, so engines know who you are.
Measurement. Track AI referrals so you can see whether any of this moves the needle.

If you want a structured way to check all of this, run through an AI visibility audit first. Then add llms.txt at the end if you have docs, or if it costs you nothing to keep current.

Frequently asked questions

Is llms.txt the same as robots.txt?: No. robots.txt is a long-standing convention that tells crawlers which paths they may or may not fetch, and most crawlers respect it. llms.txt is a newer proposal that summarizes your key pages for language models. It grants or blocks nothing. If you want to control AI crawler access, robots.txt and your CDN's bot settings are the tools to use, not llms.txt.
Does Google read llms.txt?: Google hasn't said it uses llms.txt for Search, and its AI features documentation says no special files are needed to appear in AI Overviews or AI Mode. Those features draw on Google's normal index, built by Googlebot from your regular pages. Having the file won't hurt you, but don't expect it to change your visibility in Google's AI results.
Where should the llms.txt file go?: Put it at the root of your domain, so it loads at yourdomain.com/llms.txt, the same place robots.txt lives. Serve it as plain text. On Next.js you can place it in the public folder. On Framer and Webflow, check whether the platform supports custom root files on your plan, since that varies, or host it through your CDN or a redirect.
What is llms-full.txt?: llms-full.txt is an optional companion file that contains the full text of the pages listed in llms.txt, joined into one Markdown document. It's mainly useful for documentation sites, because a developer can hand the whole file to a coding assistant in one go. For a marketing site it's usually overkill, and it can go stale quickly when pages change.
Can llms.txt hurt my SEO?: There's no known way for llms.txt to hurt your search rankings. It's a static file that search engines don't treat as a ranking signal. The real risk is indirect: a team spends time on it and believes the AI visibility job is done, while the pages themselves stay hard to crawl or never answer the question buyers ask.

Mehdi Hassan · CEO & Design Director

Copywriter, designer and developer, and founder of Designify Studio. Over five years, 60+ projects across 14+ industries, for startups that have raised $300M+.

Want us to do this for you?

We take craft seriously over at Designify Studio. We do deep research and do what we do best - pay attention to the smallest details and make sure we speak to the right audience.

Schedule Call