llms.txt implementation guide

llms.txt is a voluntary plain-text file proposed at llmstxt.org as a human-readable summary of your site. Some AI agents read it in-session when given a URL as context; some do not. It is nearly free to create and easy to keep current.

What llms.txt is

The llmstxt.org proposal describes a plain Markdown file placed at /llms.txt on your domain. Its purpose is to give LLM tools — and human readers — a curated one-page summary of what your site is, what it covers, and which pages matter most. Some tools that accept a URL as context will fetch and read it. Most will not, and none are required to.

The value is narrow but real: it documents your canonical self-description in a machine-legible and human-legible format, it takes minutes to write, and a well-maintained file costs nothing to keep current. Prompt Goblin's own live file is at promptgoblin.io/llms.txt and serves as a worked example of the format described below.

What llms.txt is not

Not a crawl directive — it does not tell any crawler what to index or skip.
Not an index submission — it does not register pages with any search engine.
Not enforced by anyone — no tool, crawler, or engine is required to read or obey it.
Not robots.txt — they share a naming pattern but serve entirely different purposes (see the comparison table below).

llms.txt vs robots.txt

Property	llms.txt	robots.txt
Purpose	Human-readable site summary for LLM sessions and curious readers	Crawl access directives for web crawlers
Format	Plain Markdown (h1, blockquote, section headings, link lines)	Custom directive syntax (User-agent, Disallow, Allow, Sitemap)
Enforcement	None — voluntary, no tool required to read it	Respected by compliant crawlers (Googlebot, Bingbot, etc.)
Who reads it	LLM tools that accept a URL as context; human developers	Web crawlers from all major search engines
What happens without it	Nothing — no penalty, no crawl change	Crawlers assume full access (all paths allowed)
SEO/AEO impact	Not a crawl or ranking signal; read or not read by individual tools	Direct impact on what gets crawled and indexed

How to write an llms.txt file

The llmstxt.org specification defines a lightweight structure. The file uses standard Markdown: one h1 for the site or brand name, a blockquote for the one-paragraph summary, and one or more h2sections each containing link-plus-description lines. That's the full structure — nothing proprietary.

The four elements

h1 — site or brand name. One line. This is the name an LLM session would use to refer to your site.
Blockquote — summary paragraph. Two to four sentences. Who you are, what you do, who you serve. Written for a reader who has never seen your site.
h2 sections — topic or resource groups.Each section groups related pages. Common sections: “Services”, “Learn”, “Methodology”, “Pricing”. Use whatever matches your actual site structure.
Link lines — URL plus description. Each line in a section is a Markdown link followed by a colon and one sentence describing what that page covers. Be accurate — this is what a tool will use to decide whether to fetch the page.

Worked template

# Your Brand Name
> One-paragraph summary of what your site is, what it covers, and who
> it is for. Write this for a reader — human or LLM — who has never
> seen your site. Two to four sentences is enough.

## Services
- [Service overview](https://yourdomain.com/services): What you offer, who it is for, and the delivery model.
- [Pricing](https://yourdomain.com/pricing): Current tier names and prices.

## Learn
- [How to show up in ChatGPT](https://yourdomain.com/learn/how-to-show-up-in-chatgpt): The three mechanical inputs that raise citation probability.
- [Why schema markup is not enough](https://yourdomain.com/learn/why-schema-not-enough): Post-implementation diagnostic for the schema-done-still-uncited gap.

## Methodology
- [How the scan works](https://yourdomain.com/methodology): What the audit measures, how findings are scored, and what the refund covers.

Place the finished file at /public/llms.txt in your project (so it serves at yourdomain.com/llms.txt) and deploy. That is the entire implementation — no build step, no plugin, no configuration.

Keeping it current

A stale llms.txt that contradicts your actual site is worse than no file at all: any tool that does read it will receive an inaccurate summary and may pass that summary to users. Update the file whenever:

Pricing tiers or prices change.
A service is added, renamed, or retired.
A significantly new page ships that you want included in the summary.
Your canonical self-description changes (new ICP, repositioned offering).

A monthly review takes under five minutes. Treat it the same way you treat updating your LinkedIn or directory profiles: a quick check whenever something meaningful changes.

Common mistakes

Keyword-stuffing the summary

The blockquote summary is for human and LLM readers, not for search-engine keyword density. A summary that reads as a keyword list is less useful and no more likely to cause citations. Write a plain English description of what you actually do.

Treating it as a ranking or retrieval signal

llms.txt is a site-summary file, not a crawl or ranking signal. The real levers for AI citation probability are covered in How to show up in ChatGPT.

Letting it contradict the actual site

Listing a retired service, a changed price, or a page that no longer exists creates an inaccurate record. If a tool passes that summary to a user, the user receives bad information. The honest-broker standard: the file must match what the site actually offers right now.

Writing it for crawlers instead of readers

Robots.txt is for crawlers. Sitemaps are for crawlers. llms.txt is explicitly for humans and LLM sessions. Structured in the way the specification describes — readable prose, clear descriptions, logical sections — it serves both audiences. Structured like a sitemap or a robots directive, it serves neither.

Frequently asked questions

Does adding llms.txt improve my chances of being cited by ChatGPT or Perplexity?

llms.txt is a site-summary file, not a ranking or retrieval signal. Some AI agents read it when given a URL as context; many tools do not. The real levers for citation probability are brand mentions on authoritative sources, Bing rank, and crawlable answer-shaped content.

Is llms.txt similar to robots.txt?

They share a naming pattern but serve entirely different purposes. robots.txt gives directives to web crawlers — block this path, allow that one — and is enforced by crawlers that respect it. llms.txt is a voluntary plain-text guide for LLM sessions and human readers; it carries no crawl directive and no tool is required to read or obey it.

How often should I update llms.txt?

Update it whenever your canonical self-description changes: new pricing tiers, retired services, significantly new pages, or a renamed product. A stale llms.txt that contradicts your actual site misinforms the tools that do read it. Monthly review takes minutes.

What happens if I never create a llms.txt?

Nothing consequential. No penalty from any search engine, no crawl block. It is a low-effort hygiene step worth doing because it is nearly free — not because anything breaks without it.

Sources cited on this page

llmstxt.org — the llms.txt proposal — The original specification and rationale for the llms.txt file format.

All other claims on this page are qualitative observations about the llms.txt specification. No statistics are cited because none are asserted.

What this does not guarantee

Nothing on this page promises a citation by any answer engine. Where Prompt Goblin engagement is mentioned: the refund covers the delivered work — audits, technical fixes, copy, measurement loop — never a citation number or ranking position.

Want to know which levers actually move citation probability for your site? Run a free scan and get a ranked list of the gaps that matter — not a hygiene checklist.