llms.txt on my Next.js site, and now Lighthouse grades it
vennee.dev has a hand-written llms.txt and an llms-full.txt built from the blog. Notes on the spec, .txt routes in Next.js, and Lighthouse 13's new audit.
llms.txt is a Markdown file at /llms.txt that summarizes a site for language models, so they don't have to dig the content out from between navigation, footer and scripts. vennee.dev already had one. I rewrote it, added an llms-full.txt generated from the blog, and ran the new Lighthouse, which now has an audit for exactly this file.
The spec requires one H1
The original proposal is by Jeremy Howard, published in 2024 on llmstxt.org. The only required part is an H1 with the site's name. After that you can add a blockquote summary and a few paragraphs without headings. Then come H2 sections, each a list of links written as [name](url): notes. A section named Optional holds secondary links that an agent short on context can skip.
The proposal doesn't require llms-full.txt. I use it to serve all the English posts in one response.
The index is written by hand
The first version was 40 lines, opening with VENN — Creative Web Developer Portfolio and a description that would fit any portfolio. The current one is written against what the site actually says:
# Anh Tuan (Venn): freelance Next.js and WebGL developer
> Venn (vennee.dev) is the portfolio of Lê Anh Tuấn (Le Anh Tuan), a freelance
> web developer in Ho Chi Minh City, Vietnam (GMT+7). ...
Use the projects page and the contact page as the source of truth for
selected work and availability. Contact: vennee.dev@gmail.com.
## Primary pages
- [Home](https://vennee.dev/): Who he is, what he builds, how a project runs, ...
## Selected work
## What he works on
## Blog
## Site language
## Profiles
I did try generating it from the SEO copy, the projects table and the post list. The data was right. It also read like a sitemap, and it lost the services section, the note on languages and the line telling a model to treat the projects and contact pages as the source of truth. A person has to write those.
The cost is remembering to update it. The file still called my R3F cube post "no WebGL on phones" after phones got the cube back, and the newest post was missing. I fixed both alongside this post.
llms-full.txt is built at build time
The blog is stored in Markdown files at content/blog/<slug>/<locale>.md, which the full-text route combines. The route handler is at src/app/llms-full.txt/route.ts:
import { buildLlmsFullTxt } from "@/lib/llms";
export const dynamic = "force-static";
export function GET() {
return new Response(buildLlmsFullTxt(), {
headers: { "Content-Type": "text/markdown; charset=utf-8" },
});
}
In Next.js 16, route handlers aren't cached by default. export const dynamic = "force-static" renders this one once at build time, like a static file, which is enough since the blog only changes on deploy.
Each post gets its own ## title. The headings inside it move down a level. YAML and shell comments in code blocks also start with #, and shifting them blindly adds stray # signs to your code. The function has to skip every line between a pair of ``` fences:
function nestHeadings(markdown: string) {
let inFence = false;
return markdown
.split("\n")
.map((line) => {
if (/^\s*```/.test(line)) inFence = !inFence;
if (inFence || !/^#{1,5} /.test(line)) return line;
return `#${line}`;
})
.join("\n");
}
The output is 266 KB with 29 English posts, and lines like # deploy/docker-compose.yml inside code blocks come through untouched.
Two Next.js snags
I created src/app/llms.txt/route.ts while public/llms.txt was still there, and the dev server answered with a 500:
A conflicting public file and page file was found for path /llms.txt
Each path gets one source. Next.js won't choose between a static file and a route for you.
The site's proxy (middleware) rewrites every path without a locale prefix to /en/..., so /llms-full.txt could have become /en/llms-full.txt and a 404. The matcher already skips any path with a dot in it:
export const config = {
matcher: ["/((?!api|_next/static|_next/image|favicon.ico|.*\\..*).*)"],
};
If your matcher lacks that .*\\..* part, open /llms.txt before you deploy, because nothing on the site looks broken when it 404s.
Google ignores it, Lighthouse grades it
According to Search Engine Land, Gary Illyes said Google won't crawl or use llms.txt. John Mueller has also compared it to the keywords meta tag. Be skeptical of anyone selling "llms.txt optimization" for Google rankings.
Lighthouse 13.5.0 grades it anyway, in a new Agentic Browsing category with seven audits. On vennee.dev only the accessibility tree, CLS and llms-txt count toward the score. The WebMCP and ai-catalog.json audits come back Not Applicable, and Lighthouse sets an N/A audit's weight to 0. Per the audit's documentation, a site with no file is "Not Applicable" and a server error while fetching it fails the audit. The audit code in 13.5.0 also requires an H1, at least one Markdown link and at least 50 characters. vennee.dev passes the whole category.
I don't know whether ChatGPT, Claude or Perplexity fetch the file, since Vercel serves it straight from the CDN and no function runs that could log the user agent. It's one file, so add it if you like, just don't expect traffic from it.
Left for later
The spec also suggests a .md version of each page, linked with <link rel="alternate" type="text/markdown">. I haven't done it, since the blog already has llms-full.txt and the other pages are mostly interface and motion. A Vietnamese version at /vi/llms-full.txt, built from the vi.md files, is more likely.
More posts
- 01
Deploying Next.js and NestJS to AWS EC2 with Docker Compose
How I deploy a pnpm monorepo to one EC2 box. Images get built in GitHub Actions and Docker Compose pulls them from GHCR. The server has no code.
- 02
How I store OAuth tokens for social media APIs in NestJS
OAuth for Facebook, TikTok and YouTube in a NestJS API. State and PKCE, encrypted tokens, refreshing before each publish, and when to ask for a reconnect.
- 03
PageSpeed gave my site 25 on mobile. My own Lighthouse said 90+
My own Lighthouse gave vennee.dev 90+. PageSpeed mobile said 25. A loader hid the LCP, Lighthouse simulated a slow network, and WebGL ran on the CPU.