GitHub - JeanKaddour/sokoban_speedrun: RL models to play Sokoban. The fastest recipe wins.

Sokoban Speedrun

Fastest recipes to RL models to solve Sokoban to a held-out target on one node:

• LLM Track: RL-fine-tune Qwen3-4B-Instruct-2507 from 57% to >80% held-out pass@1 on one 8xH100.
• Non-LLM Track: train a from-scratch agent on a single H100.

Play Sokoban if the task is unfamiliar.

LLM Track

World record history

Rules

Fastest wall-clock run wins: one run on one 8xH100 node, from training step 1 through the final training update.

• Score: the lower 95% bootstrap CI of pass@1 on llm/datasets/sokoban_eval.jsonl — a record must score > 0.80.
• Eval: 8 completions/puzzle, 12,288 tokens, temperature 0.8, top-p 0.95, seed 12345.
• Fixed: model, train set, eval set, reward function, hardware.
• Open: RL algorithm, loss, schedules, engine, parallelism, domain-agnostic rewards, prompt.
• Not allowed: Sokoban-specific hints, heuristics, or few-shot examples.
• Verification: Rerun with a second seed; both runs must score above the target. The score column reports the worse of the two runs.

Running

cd llm
uv sync
NODE_GPUS=8 uv run torchrun --standalone --nproc_per_node=3 -m speedrun
uv run python -m eval_speedrun --eval-checkpoint outputs/<run>/step_000051

Non-LLM Track

This track uses PufferLib's Boxoban environment; the initial PPO implementation was forked from pufferlib/torch_pufferl.py.

World record history

Rules

Fastest wall-clock run wins: one run on a single H100, from training step 1 through the final training update.

• Score: the lower 95% CI of the held-out solve rate — a record must score > 0.70.
• Eval: official DeepMind Boxoban test split unfiltered/test.
• Open: policy architecture, RL algorithm, optimizer, schedules, implementation.
• Verification: Rerun with a second seed; both runs must score above the target. The score column reports the worse of the two runs.

Running

cd non_llm
uv sync
uv run python speedrun.py

Submitting a record

Each track's assemble_record.sh (llm/, non_llm/) turns a finished run into a record dir: it collects the log, eval JSON, and source snapshot, builds the report, pins the top-level speedrun.py, runs verify_record.py, and adds or refreshes the record's row + redraws the leaderboard and rolling recent-training figures. Configure record runs by editing RECIPE in speedrun.py and launch them flag-free — the pinned speedrun.py then is the recipe (assembly rejects flag-configured runs). It reads a local outputs/<RUN>/ by default; pass SOURCE=modal to pull off the volume.

1. Train + eval with your track's Running commands.

2. Assemble the record:

   cd llm        # or: cd non_llm
   RUN=<RUN> DEST=records/<date>_01_<name> ./assemble_record.sh

   The record's README.md is scaffolded with a placeholder ## Idea section. Review the pinned speedrun.py diff, then fill in by hand: the record's ## Idea, and the new leaderboard row's Description + Contributors in this top-level README.md.

3. Open a PR with the record dir + new row.

Optional: verify it yourself with a second seed (otherwise the maintainers do; either way both seeds must clear the target), assembled into the record's verification/ subdir:

RUN=<VRUN> VERIFY_OF=records/<date>_01_<name> ./assemble_record.sh

The top-level speedrun.py files always hold the current record's recipe.

Credits

@joshua-a-harris's nanoRL speedrun, nanochat, modded-nanoGPT, ScaleRL, ReasoningGym for the LLM-track Sokoban env, DeepMind for Boxoban and PufferLib for the efficient boxoban implementation.