Insights · 2026-04-22
The llms.txt file: what it is, and why your site probably needs one
A tiny Markdown file at your root could be the single highest-leverage thing you do for AI-agent visibility this quarter. Here's how it works and why.
Get your free Agent Readiness Score Upgrade to Pro — $49/year
If you've ever opened robots.txt you already understand the vibe: it's a plain-text file at your site's root that speaks to a machine audience. llms.txt is its younger sibling — a Markdown file designed specifically for large-language-model agents.
The minimal version
Put a file at https://yourdomain.com/llms.txt that looks like this:
That's it. Three sections, a title, a description. No JSON, no schema, no config.
Why it matters more than you think
Large-language-model agents are increasingly using Markdown as their preferred ingestion format — not HTML, not JSON-LD, not OpenAPI. Markdown is cheap to tokenize, explicit about structure, and human-writable.
When an agent runs a task on your behalf ("find three B2B invoicing tools that integrate with Xero"), the more prep work it has to do — crawling, stripping nav, running JS — the less likely you are to be on the shortlist.
Handing the agent a pre-digested Markdown map of your site is the digital equivalent of having a well-lit shop window. It doesn't guarantee a visit, but it changes the odds.
What to put in it
Three principles:
1. Start with intent, not technology. The first two lines should tell an agent *what the business does and who it's for*. Don't lead with "React + Rails SaaS platform." 2. Link to the pages that answer task-shaped questions. "Pricing," "API docs," "Support," "Contact Sales" — these are the pages agents actually route users to. 3. Keep it short. Under 4KB is ideal. If you can't fit your whole site, link to a sitemap. llms.txt is a table of contents, not the book.
How to validate it
Run the Agent Readiness Score™ on your site — the llms.txt / Markdown-for-agents check will tell you whether a valid file is discoverable at your root and in the correct format.
If it's missing, that single fix can move your Technical Readiness score by 5–10 points.