# llms.txt Generator

> Free llms.txt generator. Paste a site URL and get both files the llmstxt.org convention describes: llms.txt, an index of your pages, and llms-full.txt with the page content inlined. Deterministic, no AI model involved. No login, no ads.

This is the Markdown copy of https://usesuperflow.ai/tools/llms-txt-generator, published for AI agents and scripts.

## What it does

Generate a spec-correct llms.txt and llms-full.txt for any site, then publish them at your site root.

## How to use it

1. **Paste your site URL.** We read the homepage, then robots.txt, then the sitemaps it points at. Sites without a sitemap fall back to the links on the homepage.
2. **We build the inventory.** Discovered pages are grouped into sections from their URL paths, and titled from their slugs. Top level pages sit in a section called Core.
3. **Copy or download both files.** llms.txt is the index. llms-full.txt is the same site with page content converted to Markdown and inlined. Each has its own copy and download button.

## Call it directly

**HTTP:** `POST https://usesuperflow.ai/api/tools/llms-txt-generator`. No API key, no account.

```bash
curl -sS https://usesuperflow.ai/api/tools/llms-txt-generator \
  -H 'Content-Type: application/json' \
  -d '{"url":"example.com"}'
```

Returns `{ ok, report: { siteName, llmsTxt, llmsFullTxt, pagesDiscovered, pagesIncluded, truncated }, cached, ageSeconds }`.

**MCP:** this tool is `generate_llms_txt` on Superflow's MCP server at `https://usesuperflow.ai/api/mcp` (Streamable HTTP, no authentication). Setup for every client: https://usesuperflow.ai/tools/mcp

Limits: 10 runs per hour per IP. Allow up to 75 seconds for a response. Results are cached for 24 hours per URL; pass `"refresh": true` to run again.

## Facts

| | |
| --- | --- |
| Cost | Free. No login, no email, no ads. |
| Where it runs | On our servers. Guarded fetches of your homepage, robots.txt, your sitemaps, and the pages selected for inlining. |
| AI involvement | None. Generation is a deterministic transform of your site's own inventory, so it costs nothing to run and produces the same output every time for an unchanged site. |
| API | POST /api/tools/llms-txt-generator with a JSON body of {"url": "example.com"}. Returns {"ok": true, "report": {...}} where report carries llmsTxt, llmsFullTxt, siteName, pagesDiscovered, pagesIncluded, and truncated. Failures return {"ok": false, "code": "...", "message": "..."}, never a bare 500. |
| Rate limit | 10 runs per hour per IP. |
| Stored data | The two generated files, cached for 24 hours keyed on the URL. Nothing beyond that cache. |
| Discovery caps | Up to 200 same-origin URLs, from up to 5 sitemap candidates and 5 child sitemaps of a sitemap index. |
| Inlining caps | Up to 15 pages inlined into llms-full.txt, an 800 KB total budget, and 200 KB per page. Runs stop starting new fetches after about 22 seconds. |
| Where the files go | Your site root: https://yoursite.com/llms.txt and https://yoursite.com/llms-full.txt, served as plain text. |

## Questions

### What is llms.txt?

A proposed convention, published at llmstxt.org, for a plain text file at your site root that tells a language model what your site contains. The format is small: one H1 with the site name, a blockquote summary, then H2 sections of Markdown links. A companion file, llms-full.txt, inlines the page content so a model can read the site without fetching each page.

### Is llms.txt a standard? Does anything actually read it?

No, and honestly, not much yet. It is a proposal by one author, not a specification any company has agreed to follow, and no major AI provider has publicly committed to reading it. Some documentation platforms and AI coding tools do look for it. Treat this the way you would treat any low cost bet: publishing the file takes a few minutes and cannot hurt you, and if adoption grows you already have one. Anyone telling you it guarantees anything is guessing.

### Where do I put the files?

At your site root, so they answer at https://yoursite.com/llms.txt and https://yoursite.com/llms-full.txt. Serve them as plain text. On most static hosts that means dropping them in the public folder. On a CMS you may need a route or a redirect. Nothing else about your site needs to change.

### Does this use an AI model?

No. Despite the name, no model runs at any point. The name refers to the output file, not the method. Generation is a deterministic transform of your own site inventory: titles come from your page titles and URL slugs, sections come from your URL paths, and content comes from a mechanical HTML to Markdown conversion. Two runs over an unchanged site produce the same bytes, which also means nothing is invented about your site.

### Why does it say fewer pages were included than found?

Because they are different jobs. Discovery lists up to 200 same-origin URLs, and every one of those appears in llms.txt as a link. Inlining content is far more expensive, so llms-full.txt includes up to 15 pages and stops at an 800 KB budget. On a site larger than that you get a real, valid, partial file. The page shows both numbers so you know which you have.

### Should I edit the output before publishing it?

Usually yes. The generator works from your URL structure, which is a decent proxy for importance but not the same thing. Read the summary line, reorder sections so the pages you most want understood come first, and delete anything that is noise. The file is yours. This tool gets you to a correct starting point in seconds instead of an hour.

### Do you store my site data?

We cache each result for 24 hours, keyed on the URL, so repeat runs and shared links load instantly. Nothing is stored beyond that cache. No account, no email, and no history of what you generated.

## Related tools

- [AI Visibility Checker](https://usesuperflow.ai/tools/ai-visibility-checker) — See whether ChatGPT, Claude, and Perplexity can actually read your site. Markdown copy: https://usesuperflow.ai/tools/ai-visibility-checker.md
- [Markdown for Agents](https://usesuperflow.ai/tools/markdown-for-agents) — Turn your pages into clean Markdown you can host for AI agents to read. Markdown copy: https://usesuperflow.ai/tools/markdown-for-agents.md
- [robots.txt AI Checker](https://usesuperflow.ai/tools/robots-txt-ai-checker) — Test your robots.txt against every AI crawler that matters. Markdown copy: https://usesuperflow.ai/tools/robots-txt-ai-checker.md
- [JSON-LD Validator](https://usesuperflow.ai/tools/json-ld-validator) — Check your structured data against Schema.org and what search engines accept. Markdown copy: https://usesuperflow.ai/tools/json-ld-validator.md

## About

llms.txt Generator is one of a set of free tools published by Superflow at https://usesuperflow.ai/tools. None of them require a login, an email address, or payment.

Category: ai-visibility.

Superflow is a website and creative-asset review tool. Its agents watch every page of a site and report what changed. See https://usesuperflow.ai.