What is LLMs.txt?
llms.txt is a simple text file that helps AI models discover, understand, and index the content on your website just like robots.txt and sitemap.xml does for search engines.
How to Add LLMs.txt to Your Site
After downloading the generated file, upload it to the root of your domain - e.g., yourwebsite.com/llms.txt. Most AI models will look for it there.
File Tree
Selected: src/llms.txt
Why You Need an LLMs.txt Generator
With the rise of AI-powered tools and agents, it is essential to make your content easily accessible to Large Language Models (LLMs) for better indexing, summarization, and contextual understanding.
How the Generator Works
Just paste your website URL above. We’ll fetch your sitemap or crawl links directly from your homepage, filter the results, and generate a downloadable llms.txt file you can add to your site.
Benefits of Using Our Generator
- No signup or login needed
- Unlimited free usage
- Improves AI discoverability and accessibility
- Helps ensure your important content isn’t missed by AI bots
Use Cases for LLMs.txt
- Blog owners and content creators wanting AI visibility
- Startup documentation and help centers
- Online course and tutorial websites
- eCommerce product pages and FAQs
llms.txt vs robots.txt vs sitemap.xml — What's the Difference?
Traditional web standards like robots.txt and sitemap.xml were designed for search engine crawlers. They tell bots which pages to index and how to navigate your site. But AI language models like ChatGPT, Claude, Perplexity, and Gemini process information differently — they need structured, contextual content they can reason about, not just a list of URLs.
| File | Audience | Purpose | Format |
|---|---|---|---|
| robots.txt | Search crawlers | Allow/disallow crawling rules | Plain text directives |
| sitemap.xml | Search crawlers | URL discovery + metadata | XML |
| llms.txt | AI language models | Content structure + context for AI reasoning | Markdown |
| llms-full.txt | AI models (large context) | Expanded page index with titles, URLs, and descriptions | Markdown |
You need all three. robots.txt controls crawl access, sitemap.xml helps search engines discover pages, and llms.txt helps AI models understand your content. Together, they form a complete discoverability strategy for both traditional search and AI-powered search.
What is GEO (Generative Engine Optimization)?
GEO is the practice of optimizing your website content to appear in AI-generated answers from tools like ChatGPT, Perplexity, Google AI Overviews, and Claude. While traditional SEO focuses on ranking in search engine result pages (SERPs), GEO focuses on being cited in AI responses.
An llms.txt file is a foundational element of any GEO strategy. It provides AI models with a structured, authoritative summary of your site — making it easier for them to understand what you do, reference your content accurately, and cite your pages in responses.
Key GEO strategies that llms.txt supports:
- •Structured content hierarchy for AI comprehension
- •Answer nuggets — concise factual blocks AI models extract and cite
- •Schema markup alignment for entity recognition
- •Citation-friendly formatting that AI models prefer to reference
The llms.txt Specification Explained
The llms.txt specification (v1.1.0, January 2026) defines a Markdown file placed at /llms.txt in your site root. The format is intentionally simple — it uses standard Markdown that any LLM can parse without special tooling.
Required structure
# Your Project Name > A concise summary of what your site/product does. ## Documentation - [Getting Started](/docs/start): Quick setup guide - [API Reference](/docs/api): Complete API documentation ## Optional - [Blog](/blog): Engineering articles and updates - [Changelog](/changelog): Release notes
Required: An H1 heading with your project/site name is the only mandatory element.
Recommended: A blockquote summary, H2 sections grouping related pages, and links with descriptions. The ## Optional section marks lower-priority resources that AI models can skip under context constraints.
llms-full.txt: A companion file with expanded entries — each page listed as an H2 section with URL and description — for AI models with large context windows. Companies like Anthropic, Cloudflare, and Vercel publish both files.
Who Is Using llms.txt Today?
Thousands of sites now serve llms.txt, including major developer platforms and AI companies. Adoption accelerated in 2025-2026 as AI-powered search became mainstream.
Anthropic
Cloudflare
Vercel
Mintlify
GitBook
Supabase
Stripe
Cursor
Fern
ReadMe
Documentation platforms like Mintlify, GitBook, and ReadMe now auto-generate llms.txt files. If your docs platform doesn't support it yet, this generator creates one from your existing sitemap — no platform migration needed.
Best Practices for llms.txt
1
Keep it concise
AI models have context limits. Your llms.txt should be a curated index, not a content dump. Save the full content for llms-full.txt.
2
Use descriptive link text
Instead of "Click here", use descriptive titles like "API Authentication Guide" — AI models use link text to understand page relevance.
3
Add meta descriptions
Each link entry should include a colon-separated description explaining what the page covers. This helps AI models decide which pages to reference.
4
Group by topic with H2 headers
Use ## Documentation, ## API, ## Tutorials, ## Blog to help AI models navigate your content hierarchy.
5
Mark optional content
Use an ## Optional section for supplementary resources. AI models with limited context will skip these and focus on core content.
6
Update regularly
Regenerate your llms.txt when you add new pages, update documentation, or change your site structure. Stale files lead to inaccurate AI citations.
7
Validate with AI models
After creating your llms.txt, paste it into ChatGPT or Claude and ask "Based on this llms.txt, what does this company do?" — the answer reveals how well-structured your file is.
How This Generator Works
This tool processes everything in your browser via CORS proxies. Submitted domains are logged for analytics. No signup required. Here's the process:
1
Sitemap Discovery
Fetches your sitemap.xml (tries multiple standard locations including WordPress wp-sitemap.xml). Falls back to crawling your homepage for links if no sitemap is found.
2
Page Processing
Extracts title and meta description from each page using client-side HTML parsing. Processes pages concurrently for speed.
3
Format Selection
Choose Standard (index with descriptions), Full (complete content dump), Categorized (grouped by URL path), or Minimal (titles only).
4
Download
Copy or download the generated file. Upload it to your site root at /llms.txt (or /llms-full.txt for the full version).
Client-side processing · Domains logged for analytics · Unlimited free usage · Built by Keploy