Lighthouse audit llms-txt · Agentic Browsing

"llms.txt does not follow recommendations": how to fix it in Lighthouse

Quick answer: Lighthouse fails "llms.txt does not follow recommendations" when /llms.txt cannot be fetched, returns a 5xx or 429, or is not Markdown with an H1 title and at least one link. A missing file that returns 404 is not a failure. Fix it by serving a short Markdown file: "# Site name", a one-line summary, and links to your key pages.
View raw .md for LLMs / your notes

This audit is part of Agentic Browsing, the newest Lighthouse category, which checks how well AI agents can read and use a page. Google marks the category as under development, so its audits may change. The llms-txt audit fetches /llms.txt from the root of your site and checks that it looks like the format proposed at llmstxt.org: a Markdown file with a title and links to your important pages. It fails with the title "llms.txt does not follow recommendations". On lighthouse-md.com, 75 of 698 analyses in the last 30 days failed it. This page covers exactly what the audit checks, why most failures come from sites that never wrote an llms.txt at all, and a minimal file that passes.

TL;DR

What does "llms.txt does not follow recommendations" mean?

Lighthouse runs one request for /llms.txt on the origin of the page you audited, and then applies these checks in order:

Result of the requestAudit resultWhat the report shows
Network error or timeoutFailFetch of llms.txt failed: Timed out fetching resource (or another error)
No status at allFailFetch of llms.txt failed
HTTP 500 to 599, or 429FailFailed with HTTP status 503
HTTP 400 to 499 (other than 429), including 404Not applicableNothing counts against you
HTTP 200, file checkedPass or fail on contentOne row per problem, below

When the file comes back, Lighthouse checks the content for three things:

CheckRule in the auditError row when it fails
TitleA line that starts with #, a space, then textFile is missing a required H1 header (e.g., "# Title").
LinksAt least one Markdown link, <a href="url">text</a>File does not appear to contain any links.
Length50 characters or moreFile is suspiciously short.

Any one row fails the audit. The checks are format checks only. Lighthouse does not check whether the links work, whether the content is accurate, or which content type the server sends.

Why does Lighthouse say my llms.txt is missing an H1 when I never made one?

Because your server answered /llms.txt with something else and a 200 status. This is the most common failure, and the fix is not to write an llms.txt. Typical causes:

Check with:

curl -s -o /dev/null -w "%{http_code} %{content_type}\n" https://example.com/llms.txt

200 text/html on a site that has no llms.txt is this problem. Make the path return a real 404, or add a real file. Either one clears the failure.

What does "Fetch of llms.txt failed: Timed out fetching resource" mean?

Lighthouse asked for /llms.txt and did not get a complete answer in time. The audit does not report a status in this case, only the error text. The usual causes:

Request the URL with curl -i from outside your network and time it. Serve the file as a static asset (or return a fast 404), exclude the path from bot challenges, and cache it at the CDN.

How do I write an llms.txt that passes?

A minimal valid file:

# Example Store

> Handmade ceramics shipped from Lisbon. Product pages, care guides and shipping policy.

## Key pages

- [Shop all products](https://example.com/collections/all): the full catalog
- [Care guide](https://example.com/pages/care): washing and storing ceramics
- [Shipping and returns](https://example.com/policies/shipping-policy): rates, times and returns

That file passes all three checks: an H1, Markdown links, and well over 50 characters. The llmstxt.org proposal adds more structure:

  1. One H1 with the site or project name. It is the only required part of the proposal.
  2. A blockquote with a short summary of what the site is.
  3. Optional paragraphs with details a reader needs to use the links.
  4. H2 sections, each a list of Markdown links with a short note after each.
  5. An optional ## Optional section for links an agent can skip when its context is short.

Serve it at the root, https://your-site/llms.txt, as a static file. text/plain or text/markdown both work; the audit does not check the content type. For a live example, see lighthouse-md.com/llms.txt.

Should I add an llms.txt file at all?

It is optional. llms.txt is a proposed convention, not a standard, and Google Search representatives have said publicly that Search does not use it. Some AI tools and coding agents do read it, and it is cheap to maintain for documentation sites, developer tools and stores with many pages. If you do not want one, a clean 404 passes this audit. What you should not leave in place is a path that returns your HTML page with a 200, or one that times out.

How do I fix it on Shopify, WordPress, or Next.js?

How do I verify the fix?

  1. Run curl -i https://your-site/llms.txt. You want either a 404, or a 200 with your Markdown in the body.
  2. Re-run Lighthouse with the Agentic Browsing category. On lighthouse-md.com it runs on every analysis; with the PageSpeed Insights API, add category=agentic-browsing. The audit should read "llms.txt follows recommendations" or show as not applicable.

What mistakes should I avoid?


Audit your URL at https://lighthouse-md.com.

Audit your page now

Paste your URL, get scores plus a CLAUDE.md plan for Claude Code.

Run audit →
Add us as a preferred source on Google

One click, and Google shows more of lighthouse-md.com in your search results.