The Saga Begins (Part 1 of 4)
It all started 13 years ago.
This programming saga started the way many of mine do: I got nerd-sniped by a Hacker News post. The post announced the C++ Grandmaster Certification course. Building a compiler had been on my list for a while, and this looked like the perfect opportunity to start.
I spent nights and weekends on it for a month or so and completed assignments one through five. I would like to say that I stopped because of other demands on my time. While that was technically true, it was not the real reason. Like many side projects, this one was murdered by tedium.
Maybe this is a familiar story: I retire from side projects when the next problem needs more willpower than I have to spare. In this case I had started a complicated refactor to switch the parsing pipeline from push to pull. The macro processing state-machine had to be rewritten from scratch. I knew the conversion would be a tedious half-dozen hours, and I couldn’t muster-up the mental fortitude, so I found something else to work on.
Aside: what was the C++ Grandmaster Certification?
The C++ Grandmaster Certification (cppgm), was a staged compiler-construction course. Each student was supposed to build an original self-hosted C++11 toolchain piece by piece: preprocessor, parser, backend, assembler, linker, etc. Each Programming Assignment (PA) came with a starter kit, design docs, a reference implementation, and a test suite. You finished an assignment, submitted it for automated grading, and then unlocked the next one. The original course stopped at PA9 and the site is no longer live, but the archived version is still worth a read: C++ Grandmaster Certification on the Internet Archive. I refer to the original milestones as PA1 through PA9 throughout this series.
Revisiting the Challenge
Eventually the certification course disappeared without producing part two, but I kept my old code in an archive. It is a common story: plenty of engineers I know have unfinished projects like this sitting around, waiting on some future when spare time is once-again abundant.
That future has arrived.
Agents are good at tedious work, so I fired up Codex, explained where the unfinished rewrite was, and asked it to finish the refactor. Within an hour it was done. Curious whether it could go further on its own, I asked it to take the existing code and work through the remaining assignments until all the tests passed. It finished in a couple of hours with minimal supervision.
Letting an Agent Loose
That led to the obvious next question: how much was I needed at all? I packaged the old course into a single repository with an agents.md file (github.com/vishvananda/agent-cppgm) and pointed an agent at it. I have since run many agents through the certification gauntlet using nothing more than an initial prompt and “please continue” if one got stuck.
A couple of behaviors during these runs surprised me. One agent reached for gdb in batch mode to work through a hard-to-localize failure and sorted the problem like a human with an interactive debugger. Another wrote a custom Python program to read the lesson grammar and generate the C++ parser code from it. Both made perfect sense in retrospect given the training data for the current crop of LLMs.
Lessons From the Experiments
Given a reasonable harness, all the frontier models make it through. Even Sonnet 4.6 passed every test. Anthropic’s Building a C compiler with a team of parallel Claudes had already shown agents can write compilers, so the success itself was not the surprise. The surprise was how little steering it took.
Two ingredients did most of the work:
- Incremental acceptance tests: Bounded steps with sharp tests let an agent make monotonic progress and know when each step is done.
- An execution harness: An agent that can run the compiler it just wrote and read its own errors will outwork an agent that can only reason about code.
The harness side is the recent change. A model that writes reasonable C++ on the first pass has been around for a while. A model that can run the compiler, read the error, and try again for hours without losing the thread is much newer. That difference is also what kept me coming back to the problem: short cycles meant I could try a direction, watch what the agent did, and adapt how I was directing it without burning a day on each experiment. The step change is the agent’s ability to sustain that loop long enough that I iterate on direction rather than on every individual fix.
The Harder “Half”
After PA9 you do not have a full C++ compiler. You have a useful collection of pieces and an end-to-end backend through a toy target. The challenging parts of C++ are still to come, and the original course never got there. If you want to give your own agent a run at the original assignments, head over to github and give it a shot.
But here is the exciting part: over the past couple of months I used 2 billion tokens to design and implement the remaining 28 assignments and the full self-hosting compiler that goes with them. Part 2 covers the assignment set build out and links to a version your agent can use. Part 3 covers what went wrong building the reference compiler and the two large rewrites that followed. Part 4 covers the tooling I ended up building around the agent: lessons that can be applied to your own greenfield and brownfield agentic builds.