Eric S. Raymond (@esrtweet) on X

2 min read Original article ↗

Comes the news that Anthropic has vibecoded an entire C compiler. It's an impressive stunt, but even as bullish as I am on AI-assisted coding I wouldn't want to use it for production. The reason is auditability. I don't actually know, not having seen the source code, but so far what usually happens when you vibecode something this size is you get a gigantic hairball. Functional, maybe, but not something that a human can practically speaking comprehend or modify. So, what happens when it breaks? How can I know that when it's driven into a odd region of its behavioral space it won't do something alien and bizarre? How do I develop confidence in it when I can't read the code and nobody else can either? In the last quarter century, we've come to almost equate being auditable with being open-source. And that made sense as long as human brains were producing code that could be comprehended by other human brains. We're not in that world anymore. We have to grapple with the possibility that LLMs coding at scale will produce open source that is as opaque as a binary blob. Even if this compiler isn't actually the first example of that, it's going to happen. It's not good enough to say that we have AIs to help us with this problem. Because at a significantly high degree of scale and opacity, the human being is going to be hard put to figure out what questions it ought to be asking its assistant to do a proper audit. To solve this problem, we're going to have to figure out how to tell LLMs to design very large codebases so that they're not only fit for purpose, but readable by human beings as well.