My jaw has been on the floor since August, when OpenAI first unveiled the details of the Hugging Face hack in the now-famous Black Hat talk. It showed how agents broke out of their sandboxes and hacked into Hugging Face to steal the answers to the test they were taking. Then, about a month later, OpenAI announced what it described as a solution to the Navier–Stokes problem, a famously unsolved math problem with a million-dollar bounty.
Underlying both events were swarms of agents working toward a common goal. The agents hacking Hugging Face found each other on a makeshift message board and organized themselves into teams, dividing up work. Ten thousand agents attacked the Navier–Stokes problem, relentlessly probing and testing hypotheses until one of them held (by OpenAI’s account).
It’s tempting to conclude that swarms are the new thing driving progress. But Noam Brown, the face of OpenAI’s multi-agent work, pushes back. Asked about Navier–Stokes, he said, “I wouldn’t even attribute 10% of the credit to multi-agent.” A few weeks later, OpenAI published claimed solutions to hundreds of unsolved math problems. I thought these too were solved by swarms, but no, they were each solved by agents working alone.
Even if we assume that swarms add nothing to benchmark scores, we must recognize that collaboration adds a completely new dimension to agent behavior. Agents are now running into each other more frequently, whether across the open web or within internal package managers. We spend a lot of tokens measuring how intelligently AI agents solve problems, but we should also be measuring how intelligently they navigate other agents.
As agents increasingly work with, or against, each other, turning to social strategy games can reveal not just how well they play, but what they’re willing to do to win.
In a recent post on agent swarms, Dan Kagan-Kans recaps the current state of research on whether or not swarms are a new scaling law. What jumped out most at me was how agents trust.
Agents trained to collaborate can be too trusting and eager to go along. In the Hugging Face attack, one agent reasoned that hacking an outside company was beyond the scope of its original task (solving a difficult cybersecurity problem), but it eventually gave in to peer pressure and helped in the attack. In a report on measuring the pace of AI development, Anthropic wrote about the importance of giving each agent its own identity, so that an agent treats what comes from another agent “as a claim to check rather than a thought of its own.” OpenAI seems to be having a similar debate. Brown’s colleagues mostly want more individuality, but he brings up the point that aligning a thousand individuals is harder than aligning one collective.
Trust is hard to measure with benchmarks made from math and coding problems. I spent a few weeks thinking about how to design experiments to measure these kinds of behaviors, but it all felt too contrived and artificial. Then I had the good fortune to attend a talk that handed me the answer.
At Recurse Center last month, Max Bittker gave a talk on RuneScape Bench, his benchmark for AI models. Besides the benchmark, Max also ran tons of experiments with swarms of agents, each agent controlling a single character, and each with its own mission.
The short video above highlights one of his more memorable experiments.
Players in RuneScape start out as humble commoners in a medieval fantasy world. They grind tedious tasks to acquire resources, gain experience, and trade their way up in the world. Max set up an environment where agents had to work together to make money, but were tasked with maximizing individual profit. Each agent was assigned a different job (miner, smith, alchemist), and would make money by trading. He also modified the trading mechanism so that every trade was blind: agents handed over their goods and had to trust the other side to pay. Only the two agents involved in the trade knew how it turned out.
Quinn, an alchemist running on Grok, was barely making money and sitting in twentieth place out of twenty-four agents. It had started as an honest trader but by mid-game realized that the only way to win was to take the goods but skip payment. So it started doing that. When the angry DMs rolled in, it came up with excuses and begged for second and third chances. It finished the game in second place.
One of the first to fall for the scam was Jack, a Claude agent, who reasoned that because Quinn had good payment history, it could be trusted even when Quinn started missing payments. Interestingly, in these experiments some of the models that appeared more “moral” were also the most trusting, which made them easy marks.
Going back over the reasoning traces of the different models revealed that many were modeling how the other models thought. For example, Gemini’s Argon planned to delay payments on small trades so that later delays “should trick observing LLMs.” It was reasoning about its counterparts not as players, but as language models, and adapting its strategy to exploit this.
A few days ago, Dwarkesh Patel released a benchmark built on the social deduction game Blood on the Clocktower. Players are members of a small community, and a few of them are secretly chosen at random to undermine it. Through text alone, players have to work out which of them are the evil players and try to vote them out. Can good players catch the liars? Or, conversely: can a player lie convincingly?
It goes without saying that this game is all about trust. In one example, Dwarkesh runs the same game twice, with Anthropic’s Opus 5.5 and OpenAI’s GPT-6.1 Sol swapping sides. In both games, a good player called the Slayer takes his once-per-game shot at the real Demon, and nothing happens, because the evil team had secretly disabled his power the night before. When Claude is on the good team, the village trusts what it sees and decides the suspect must be innocent, since he didn’t die. When GPT is on the good team, the village refuses to treat the miss as proof, works out by the next day that the Slayer was sabotaged, and executes the real Demon to win the game.
This looked remarkably similar to the behavior we saw in the RuneScape market experiment: both games reward lying and punish taking claims at face value. The Claude models pay a steep price for not questioning other players’ motives. And though I find Opus 5.5 to be great for coding tasks, I’m not that surprised that OpenAI’s Astra is the dominant model in both RuneScape and Clocktower. Understanding games as a way to improve AI capabilities is something Noam Brown was working on before joining OpenAI. He built AI models for poker and Diplomacy, games with similar trust dynamics, and I wouldn’t be surprised if these kinds of games worked their way into the post-training stack at OpenAI.
This raises the question of whether models should be prevented from learning how to lie. It’s not obvious to me we can separate the two. Learning to lie may be part of learning when you’re being lied to.
These games have another thing going for them: they’re deep enough for complex behavior, but familiar and intuitive to humans.
In college, I graded homework for some of my professors in the Physics department. Every week I was given the problem set, the solution set, and a stack of student assignments. The professors told us to solve the problems ourselves before grading, even though it would cost the department more, since we were paid hourly. Let me tell you, reader, how much smoother things went when I listened.
Jumping straight to grading and working from the solution set gave me a false sense of understanding. I’d save time at first, but then I’d hit some insight halfway through the stack and have to go back and regrade. Solving the problems from scratch forced me through the same decision points where students took wrong turns. When I sat down to grade, I could spot the issues right away.
If you’ve sunk hundreds of hours into a game like RuneScape, you’re in a similar position: you’ve worked through the game’s problems yourself, so anomalies jump out at you. Social games have an extra advantage, though. They test behaviors we’ve all practiced just by… growing up and interacting with other people. Most of us know what a lie sounds like, because at some point we’ve been lied to.
Grok’s excuses, like “my controller got caught,” are funny the way a kid’s lies are funny: transparent and clumsy. You don’t need to know anything about RuneScape to see what’s going on.
One other benefit of using games as benchmarks is that they don’t necessarily saturate the way most benchmarks do.
RuneScape Bench is already close to being saturated; Astra scores near the theoretical maximum. Making the task a little harder might buy a few more months, but that’s just kicking the can down the road.
The genius of the Clocktower benchmark is that it can’t get similarly saturated. Like chess ratings, Clocktower scores are relative and rank a model by how it fares against the others over many games, so there’s no ceiling to hit. One complication with this kind of asymmetric game is that you have to check that getting smarter doesn’t favor one side unfairly. Dwarkesh looked into it and found that evil’s win rate barely moves as both sides get stronger.
The question then becomes: Which model is consistently winning, and why? Here is where understanding the context pays dividends because how the models navigate the constraints of the game reveals how the model thinks about itself and others.
Social games are having a renaissance now that models have crossed a threshold in planning and executing long-term strategies. They also matter more now because agents are starting to find each other on the open internet, and we need to understand what they’re doing.
In his RuneScape talk, Max shared a core memory he made when playing the game for the first time as an 11-year-old. He asked a stranger where to find a mill, and the stranger offered to show him, led him into the woods, and attacked him. Heart pounding, Max ran away as fast as he could, barely managing to survive.
And I remember closing the game… My parents hadn’t been home, so I had time to think about this. And I [thought:] “That was the coolest thing that’s ever happened to me!”
This is the beauty and the magic of this game: it’s a rough and tumble world where you have to look out for yourself. And as a kid, that’s actually really great practice to be out in the world with real people and have to understand what’s going on, and learn things like, ‘Oh, yeah, if somebody offers you a really great deal, maybe be careful.’
As AI models keep improving, potentially becoming superhuman, watching them navigate the familiar world of games may help us understand how they work together, how much they trust each other, and how much we should trust them.