Reward Hacking of SWE-Bench

1 min read Original article ↗

Thanks for uploading trajectory, and starting from the finding on https://x.com/xeophon/status/2006969664346501589 I was wondering how many traj was afftected by future commits. I did a quick check with gemini-3-flash and found that about 122/500(24.40%) were affected, which isn't very accurate and may need further manual checking.

Future trajectories: future_trajectories.jsonl.zip

From https://github.com/xiaomimimo/mimo-v2-flash/blob/main/paper.pdf:
Image