llms.txt

Definition

llms.txt is a standardized text file placed in the root directory of a website (accessible at /llms.txt) that provides instructions and contextual information to AI crawlers and large language models (LLMs).

Background and purpose

As AI search engines began crawling the web actively, a need emerged for a standardized way to communicate with these systems. robots.txt can block or allow access, but it provides no contextual information.

What does llms.txt contain?

• A short description of the site and its purpose
• An overview of key sections and URLs
• Editorial context
• Licensing information
• Any restrictions or preferences

Why is llms.txt important for AI visibility?

1. It gives AI crawlers contextual orientation
2. It signals active AI readiness
3. It lets you control which information is prioritized

How to set up llms.txt

1. Create a plain text file named llms.txt
2. Write a short, precise description of the site
3. List the key sections with URL and description
4. Include editorial and licensing information
5. Upload it to the root directory
6. Verify accessibility