feat: add llms.txt endpoint for LLM-optimized documentation by quantizor · Pull Request #2388 · tailwindlabs/tailwindcss.com

3 min read Original article ↗

I totally see the value in the feature and I would like to find a way to add it.

But the reality is that 75% of the people on our engineering team lost their jobs here yesterday because of the brutal impact AI has had on our business. And every second I spend trying to do fun free things for the community like this is a second I'm not spending trying to turn the business around and make sure the people who are still here are getting their paychecks every month.

Traffic to our docs is down about 40% from early 2023 despite Tailwind being more popular than ever. The docs are the only way people find out about our commercial products, and without customers we can't afford to maintain the framework. I really want to figure out a way to offer LLM-optimized docs that don't make that situation even worse (again we literally had to lay off 75% of the team yesterday), but I can't prioritize it right now unfortunately, and I'm nervous to offer them without solving that problem first.

@PaulRBerg I don't see the AGENTS.md stuff we offer as part of the sponsorship program as anything similar to this at all — that's just a short markdown file with a bunch of my own personal opinions and what I consider best practices to nudge LLMs into writing their Tailwind stuff in a specific way. It's not the docs at all, and I resent the accusation that I am not disclosing my "true intentions" here or something.

This feature is so that people can build MORE things with Tailwind in a FASTER and more EFFICIENT capacity.

@mtsears4 Tailwind is growing faster than it ever has and is bigger than it ever has been, and our revenue is down close to 80%. Right now there's just no correlation between making Tailwind easier to use and making development of the framework more sustainable. I need to fix that before making Tailwind easier to use benefits anyone, because if I can't fix that this project is going to become unmaintained abandonware when there is no one left employed to work on it. I appreciate the sentiment and agree in spirit, it's just more complicated than that in reality right now.

@adamwathan I totally sympathize with you.

Here’s my unbiased suggestion: you’re blocking llms.txt, which prevents LLM bots from crawling your site. Since your content is already included, they will continue serving outdated content.

They still crawl all open-source projects, so finding a “how to” isn’t a problem for any LLMs.

You're not blocking robots.txt; otherwise, you'd be removed from search engines. Many search engines provide index data as augmented data. The "llms.txt" file is for LLM agents and cloud AI agents (a small subset), which are mostly excluded from training data.

Users can still share Tailwind CSS links in Chat Agents, and they’ll be accessed from the local machine without any restrictions.

But making that move could cost you valuable traffic from LLM chat references. SEO is on the brink of becoming outdated, with major traffic sources now coming from LLM mentions on authority sites you already own, and chat app agents featuring ads.

Your decision might be counterproductive. I have the best intentions for Tailwind CSS to thrive, and I’m a proud user for both local and private projects.

Best,
Nilay
Disclaimer: I’m sharing my idea in a personal capacity and as-is, without any warranty. Please make sure to verify and fully understand your actions if you choose to use my insight.