What is llms.txt?
llms.txt is a proposed Markdown file that gives language models a concise introduction to a website and links to its most useful pages. The format is designed to make important documentation and context easier to find than a full site crawl.
Where do I put my llms.txt file?
The common location is your domain root: https://example.com/llms.txt. Some tools may also support a file in a subpath, but the root location is the clearest default for a public site.
Is llms.txt required for AI crawlers?
No. It is a voluntary convention, not a requirement for indexing or crawling. It can complement strong site structure, descriptive content, and accessible documentation.
How is llms.txt different from robots.txt?
robots.txt tells participating crawlers which paths they may access. llms.txt is a curated content guide that points a model or agent toward useful information after it can access your site.