Third Batch Benchmark
The Third Batch Benchmark will test AI systems on ten naturally occurring research math problems which have been solved by human mathematicians but whose solutions have not been made public.
Timeline
Problem selection
August — September 2026
Mathematicians across fields submit unpublished problems with proofs of at most 12 pages. All submissions undergo a first round of refereeing, and 10 problems are selected for the benchmark.
Benchmark testing
Early October 2026
The editorial board tests AI systems via API. Each system is an open source harness which calls publicly available models, and gets one shot per question with no additional interaction.
Benchmark grading
October 8-14, 2026
Roughly 30 expert referees review the AI solutions double blind, scoring correctness, exposition, and attribution separately. Results — including the problems, human solutions, referee reports, and editorial board scores — will be published on October 14, 2026.
AI Systems Tested
DeveloperOpenAI
Base modelGPT-6 Astra (reasoning effort max)
DeveloperAnthropic
Base modelClaude Opus 5.5 (high effort)
TeamTim Gehrunger (lead), Johannes Schmitt, David Holmes, Jasper Dekoninck, Yonggang Jiang, Gergely Bérczi, Uri Kreitner, Liam Price
Base modelsGPT-6 Astra Pro, GPT-6 Astra, Claude Fable 5.1, Claude Sonnet 5, Gemini 3.1 Pro (preview)
TeamYuanjie Ren (lead), Yidi Qi, Jinzheng Li, Zeru Zhu, Kai Chau (Matthew) Kung
Base modelsGPT-6 Astra, GPT-6.1 Sol
TeamZhengyang Zhang (lead), Jing Xiong, Jui-Hui Chung, Hongzhou Lin, Xingyang Li, Junxian Guo, Enrico Shippole, Mark Kim, Ligeng Zhu
Base modelKimi K3 (open-weight, self-served)
Results
