Are llms.txt and llm-info Pages Worth It? What the Data Actually Shows

An llms.txt is worth a few minutes of thought and not much more.

We checked 300+ of our clients' websites for one, then compared their visits from AI assistants such as ChatGPT, Gemini and Perplexity over twelve months.

Sites with the file got the same AI traffic as sites without it.

Is llms.txt worth the effort?

llms.txt is worth the effort only as a short housekeeping job, never as a strategy. It costs little to write and nothing to host. But no search engine or AI company has said it uses the file to decide what to show or cite, and our own client data shows no difference in AI visits between sites that have one and sites that don't.

Make one if it takes you minutes, skip it if it's a project, and spend the time you save on the checks in the last section. The client data is where that answer comes from.

Where did llms.txt come from?

llms.txt came from Jeremy Howard, who proposed it on 3 September 2024 as a way "to provide information to help agents use a website." The idea was a short Markdown map of a site, written for AI tools to read.

Two more terms grew up around it:

• llms-full.txt holds the full text of a site's key pages in one Markdown file. Mintlify, a documentation platform, says it built the format with Anthropic for Anthropic's developer docs.
• llm-info page, also called an AI info page, is an ordinary web page of plain facts about a business for AI assistants to quote. It has no spec at all; we cover it further down.

The file itself is small, as the next section shows.

What is llms.txt?

llms.txt is a Markdown file at the root of a website that lists the pages an AI tool should read first. Jeremy Howard proposed it on 3 September 2024, describing it as a way "to provide information to help agents use a website." It remains a proposal: no search engine has adopted it as a standard. Our llms.txt glossary entry has the short version.

The format is simple. Only the first line is required:

# Hawk Academy                         <- H1 with the site name (the only required part)

> Free SEO and AI search education:    <- a one-line summary in a blockquote
> Claude skills, tools and guides.

## Guides                              <- H2 sections holding lists of links
- [Is llms.txt worth it?](https://hawkacademy.co/seo-guides/is-llms-txt-worth-it): our client data

## Optional                            <- links an AI tool can skip when short on space
- [Glossary](https://hawkacademy.co/glossary)

The proposal also suggests a plain Markdown copy of each important page at the same address with .md on the end. None of that tells Google anything, as its own guide makes clear.

Does Google use llms.txt?

Google does not use llms.txt. Its AI optimisation guide says Search ignores the file, including Google's AI features. In Google's words, creating one "will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them."

The same guide says it is "completely fine" to keep one "for other services or systems that use these files", and lists "creating unnecessary AI text files (like llms.txt)" among the tactics you can ignore.

That leaves the question of how everyone else responded.

How did the industry respond to llms.txt?

The industry split between developer tools that adopted it and search people who doubted it.

• Documentation sites took it up first. OpenAI and Perplexity publish an llms.txt for their own developer docs, so coding tools can read those docs quickly.
• Platforms switched it on by default. Shopify adds an llms.txt to its stores, which is why most of our clients' files sit on Shopify.
• Google stayed sceptical. In 2025 John Mueller called it "comparable to the keywords meta tag", a tag Google stopped using years ago. Google's guide made the same point formally in 2026.

So after a couple of years of debate, we looked at our own data.

What happened across 300+ client sites?

Across 300+ sites, the sites with an llms.txt got the same AI assistant visits as the sites without one.

We measured AI assistant sessions per 1,000 non-brand Google clicks, so a big site and a small site can be compared fairly across the last 12 months (October 2025 to September 2026).

Three details matter more than the headline:

• Most Shopify stores already have one, 121 of the 125 Shopify stores in the sample serve an llms.txt, because Shopify adds a default file. Their owners never chose it, and it made no visible difference.
• Hand-made files lean slightly higher, on other platforms, the 48 sites with a file had a median of 177.7 against 125.1 for the 221 without, a gap within chance (p = 0.45).
• Size and platform don't explain it away. We also compared sites only against others on the same platform and of a similar size. Still no difference (p = 0.81).

The limits, stated plainly: we checked for the file on 2 November 2026, after the twelve-month window, so some sites added theirs late.

AI sessions are a floor, because assistants often hide where a visit came from. And these are our own clients' sites, one sample of the web. What do the AI companies themselves say?

Do ChatGPT, Claude and Perplexity read llms.txt?

ChatGPT, Claude and Perplexity have not said they use your llms.txt to choose what to read or cite. Their crawler documentation, checked on 2 November 2026, tells site owners to control their bots with robots.txt. OpenAI and Perplexity each publish an llms.txt for their own developer docs, which helps coding tools read those docs. That is a different job from deciding which websites an answer cites.

Larger studies point the same way.

What does other research say about llms.txt?

Other research finds no link between llms.txt and AI citations. SE Ranking looked at roughly 300,000 domains, found an llms.txt on 10.13% of them, and concluded the file "doesn't seem to directly impact AI citation frequency. At least not yet." Search Engine Journal reported the study on 20 November 2025.

Their study measured citations and ours measured visits, and both reach the same answer. If you still want one, it's a quick build.

How do you make an llms.txt?

You make an llms.txt by writing a short Markdown file and serving it as plain text at /llms.txt:

1. Start with # Your site name and a one-line summary in a blockquote.
2. Add a few ## sections, each a list of links to your most useful pages, with a short note after each.
3. Put secondary pages under ## Optional.
4. Upload it to the root, so it loads at yourdomain.com/llms.txt.
5. Check it the way we did: open the address and confirm it shows plain Markdown text with your site name at the top.

On Shopify, you probably already have one. Open yourstore.com/llms.txt before building anything. If you edit it, keep the edit small: our data gives you no reason to spend an afternoon on it. The other idea people ask about is the llm-info page.

What is an llm-info page, and is it worth making?

An llm-info page is an ordinary web page of plain facts about a business, written so an AI assistant can quote it accurately: what you do, who runs it, where you operate and what it costs. You will also see it called an AI info page. There is no spec and no standard name. See the llm-info page glossary entry for the short version.

It differs from llms.txt in one way that matters. It is a normal HTML page, so the AI crawlers that already read your site can read it too, under the same robots.txt rules. Whether it changes what AI tools say about you is unproven: the reports so far are individual sites' experiences, and we know of no large study. Our client data does not cover these pages, because there is no reliable way to spot one across 300+ sites.

Our verdict: build one if AI assistants describe your business wrongly. Otherwise put the same facts on your About page, where people read them as well. Hawk Academy does a version of this with its EntityMap, a page listing the facts about Hawk Academy, its people and its free resources, each tied to the page it comes from. Both ideas come after the checks below.

Which AI search checks come before llms.txt?

The checks that come before llms.txt are whether AI crawlers can reach your pages and read them. robots.txt controls that access, and it is where we find real problems on client sites. Three checks, all free:

• See which AI bots your robots.txt lets in with the AI Crawler Access Checker.
• Check that your content sits in the raw HTML with Can AI See Your Page. Most AI crawlers, including GPTBot, ClaudeBot and PerplexityBot, read raw HTML and never run JavaScript.
• See whether AI assistants already send you visits with our guide to tracking Claude traffic in GA4.