01What is llms.txt
llms.txt is a proposed standard (originated by Jeremy Howard in 2024) for AI-readable site summaries. The file is a markdown document at /llms.txt with sections describing the brand, primary pages, capabilities, and (optionally) full document context. It is now supported by major AI crawlers.
02Why llms.txt matters
AI crawlers and LLM training pipelines parse llms.txt to understand site structure quickly. Sites without llms.txt rely on the crawler inferring structure from HTML, which is slower and lower fidelity. llms.txt accelerates accurate citation and reduces the chance of misattribution.
03Recommended llms.txt structure
1) Brand summary (name, domain, founder, service area). 2) Primary pages with one-line descriptions. 3) Capabilities list. 4) Contact information. 5) Optional llms-full.txt with deeper context. Keep llms.txt under 200 lines; offload depth to llms-full.txt.
04llms.txt vs robots.txt
robots.txt tells crawlers which URLs to crawl or skip. llms.txt tells AI crawlers what the brand is and which pages matter. They are complementary: deploy both. Most sites in 2026 still deploy neither for AI optimization.