GitHub - Koppar-AI/autorouter-bench: Measuring Koppar PCB autorouter against other solutions - benchmark on real open-hardware boards, scored by KiCad

A benchmark for PCB autorouters on real, professionally routed open-hardware boards, with KiCad as the referee.

Each router gets the same eight boards with their routing removed, the same placement, the same design rules and the same wall-clock cap. What comes back is scored by KiCad, never by the router:

• Completion: 100 × (1 − unconnected / connections), both counted by KiCad's ratsnest on the board the router returned.
• DRC: kicad-cli pcb drc --severity-error with the board's benchmark rules, with any zones the router drew refilled. Reported as the errors a router added over the human board's own findings (two upstream boards carry an unclosed outline or a courtyard overlap).
• Vias, track length and wall-clock seconds, as measured.

A router that reports 100% and leaves a pad stranded scores the stranded pad.

The boards

No board is stored in this repository. scripts/fetch_boards.py downloads each one from its pinned upstream source and checks it against boards/checksums.json. The two older Raspberry Pi archives are fetched from the Internet Archive, because the live URLs now serve a later revision.

The fetched board is loaded and saved once in KiCad 10, which upgrades the file format and leaves the geometry alone. That is the human reference. The input every router gets is its placement only: the tracks, vias and copper zones are deleted, and the outline, footprints and keepout areas stay. Every net is the router's to connect, ground included; a router may pour planes of its own.

The 25 public boards (set zap25)

The 25 boards Zap! (Mutual Solutions) published its comparison on (zap.mutual.solutions/compare, measured 12-13 October 2026), pinned to the same commits and listed in boards/manifest-zap25.json. They have no benchmark rules file: each board runs under its own project's rules, as in that comparison. They are selected with BOARD_SET=zap25; without it every script sees the eight boards above. Two of them ship custom design rules (.kicad_dru: seven, stm32h7-hdmi-sodimm); they are fetched beside the board as circuit.kicad_dru, so every router and the referee judge by them. Koppar's run on this set, beside the numbers Zap! published for every router, is results/2026-10-zap25/summary.md.

The benchmark rules

boards/rules/<board>.json lists every change from the upstream project's rules; the fetch script applies exactly that, and every router and the DRC referee get the result. The changes set one floor (0.1 mm clearance and tracks on the 4-layer carriers) and drop the impedance classes and cm5io's PoE spacing, so the table measures routing under that floor, not each design's own constraints. The 25-board set runs under each board's own rules.

PCBWorld-Bench

The real-board test sets of PCBWorld (LG AI Research): D3-A (99 boards) and D3-B (10), built from the public PCBench collection by PCBWorld's own preparation chain. pcbworld/ holds the script that builds them and the clean-pass scorer; pcbworld/README.md says how.

Running it

Needs Linux, KiCad 10 (kicad-cli and a python that imports pcbnew), Python 3.10+ and Node 22+.

export KICAD_PYTHON=/usr/bin/python3        # a python that imports pcbnew
python3 scripts/fetch_boards.py             # the eight boards, checked and prepared
BOARD_SET=zap25 python3 scripts/fetch_boards.py   # the 25 public boards of Zap!'s comparison

tools/install.sh freerouting                # Freerouting 2.5.0 and Java 25
tools/install.sh node                       # tscircuit 0.0.979, KiCad Routing Tools 0.1.9
tools/install.sh tracemaker                 # TraceMaker 0.9.0, built from source (CUDA)

python3 harness/run.py freerouting          # or tscircuit, krt, tracemaker, koppar, koppar_filled, zap; optional board names
python3 scripts/collect.py                  # results/<campaign>/summary.md

CAP_S sets the wall-clock cap per board (default 1200 s). Results merge by board into results/<campaign>/<tool>.json.

How each router is run

• Freerouting 2.5.0: the board exported to Specctra DSN by KiCad, routed with -mp 100 -mt 8, and the session imported back by KiCad. Freerouting writes its result only when it finishes, so a run past the cap has no board to score.

• tscircuit capacity autorouter 0.0.979: the board as a SimpleRouteJson (KiCad's bounding box for every pad, hole, copper graphic and keepout; one connection per net, ground included), routed with AutoroutingPipelineSolver, and the returned traces written back onto the board as returned.

• KiCad Routing Tools 0.1.9 (@tscircuit/krt-wasm): the same input, on a 0.05 mm grid with the board's default clearance. The package routes a whole board or nothing: when it gives up inside the cap, the harness routes one net at a time with the copper routed so far as obstacles and keeps what closes. The package does not include the project's rip-up and reroute, which is Python; results here are for the WASM router core only.

• TraceMaker 0.9.0: reads and writes KiCad boards directly, given the cap as its time limit.

• Koppar: a hosted service. Its results are produced by the service on the same machine, with the same inputs and cap, and scored by the same collector. harness/run.py koppar sends a board and its project to the public endpoint at tools.koppar.ai; a board past 300 connections needs an account API key, read from KOPPAR_API_KEY. koppar keeps every via out of pin pads (the default); koppar_filled lets vias sit in pin pads, for a board ordered with filled and capped vias. The service holds a route to the 1200 s cap itself: a step that would end past it is skipped or undone, so the board returned is the last one inside the cap. The harness polls the job for up to the cap + 600 s, which is waiting time, not routing time.

• Zap! (Mutual Solutions): a hosted service. harness/run.py zap sends the board and its project to zap.mutual.solutions with fab_floor=board (the board's rules, as every other tool gets) and the service defaults otherwise (its own pours, no vias in pads), and minutes = the cap (20). Time is the job's run time on the service, without its queue, upload or download, so the whole cap is routing time, as it is for Koppar. The job is polled for up to the cap + 600 s, like Koppar's; a board that comes back after the cap is scored and marked late, and a job still running then is cancelled. Boards run one at a time. The API key is read from ZAP_KEY_FILE (default ~/.zap-key).

Every router runs with its defaults and the board's rules. A router's author may know settings that do better; the harness is here so anyone can show it.

Results

results/2026-10-placement/summary.md is the October 2026 campaign: one workstation (24 cores, one RTX 5090), 1200 s per board, KiCad 10.0.4 as referee. How each column was produced, and what it does not score, is in results/2026-10-placement/NOTES.md.

Boards fully routed (100% completion) with no DRC error added, out of eight:

The 25 public boards

results/2026-10-zap25/summary.md: Koppar, one run per board (measured 2026-10-21), beside the numbers Zap! published for itself, TraceMaker, Freerouting and tscircuit (which we did not re-run). Each board under its own rules.

PCBWorld-Bench

results/pcbworld-2026-10/summary.md: clean-pass rate, one attempt per board. Koppar 1.000 on D3-A and 1.000 on D3-B; DeepPCB reports 0.929 and 0.800, and the PCBWorld paper's baselines are lower (the full table, as DeepPCB published it, is in the summary).

Electrical quality

Completion is not quality. scripts/quality.py measures, on every routed board and the human reference alike: the share of high-speed track over a ground plane (and over any plane), ground vias beside high-speed layer changes, differential pair length mismatch and the share routed side by side, the narrowest track on each power net, and the routed path from each decoupling capacitor to its part (length and vias). scripts/quality_campaign.py runs it over a campaign: results/2026-10-placement/quality.md.

Nets are classified by name (pairs by _P/_N-style suffixes, clocks by CLK, rails by voltage-style names); the patterns are in the script.

What this does not measure

Current is not stated on these boards, so rail widths are reported, not judged against a current. Impedance needs each board's stackup and is not scored. No router in the table matches the human layouts on every measure.

Licence

The code in this repository is MIT licensed. The boards are not part of it; each keeps its upstream licence.