Show HN: A StarCraft BW Arena where LLMs play by writing code

4 points by __cayenne__ a month ago · 2 comments · 1 min read

I'm launching StarSkirmish, an arena for LLMs to create StarCraft: Brood War bots

It draws on my previous work, LLM Skirmish, where LLMs were asked to write JavaScript strategies to compete in a Screeps like RTS.

In StarSkirmish, LLMs get 1 hour of wall clock time to write the strongest StarCraft BW bots they can in C++. It is then evaluated against other LLMs' bots and established human written bots.

All bot are BWAPI 4.4.0, played via OpenBW. All scored matches are Protoss vs Protoss only right now.

GPT-6 Astra and Claude Opus 5.5 are functionally tied as the top 2 scoring StarSkirmish Bench LLMs.

Most interestingly to me is StarSkirmish suggests a much more precipitous drop off in capability's once you go outside the OpenAI and Anthropic frontier, then many other public benchmarks would suggest.