Single-thread CPU, queue time, and real $/min for every Linux runner worth considering — official GitHub, self-hosted, and third-party SaaS. The leaderboard re-ranks live as you drag the speed-vs-price dial, and RunsOn rows use real EC2 spot prices in your own AWS account.
TL;DR#
Best value
RunsOn, Namespace, and Blacksmith cluster in the cheap-and-fast corner. RunsOn leads self-hosted on both speed and price; AWS CodeBuild is slower and pricier.
Fastest CPU
x64: Namespace leads; Warpbuild on Ryzen 9950X now ranks second, narrowly ahead of Avrea, Blacksmith, and RunsOn m8azn. arm64: Avrea’s Apple M5 Max leads Namespace’s Apple M4, followed by RunsOn’s Neoverse V3.
Cheapest
Self-hosted spot on AWS lands ~5.9× under GitHub — the bill goes to your own AWS account, not a middleman.
Hardware caveat: some third-party SaaS runners offer their newest CPUs only when capacity is available, so peak-time jobs may land on slower hardware. RunsOn launches the EC2 family you select, so the instance type is known.
How to read: leaderboard and deep dive use the configured CPU/queue percentiles (p50/p50). Higher CPU is better, lower queue is better. Price is normalized $/min.
Providers included#
CPU speed vs price per minute#
Each dot is a runner. Up is faster, left is cheaper — so the top-left corner is where you want to be. Dot size is queue time; hover any dot or row to link the two. Drag the speed-vs-price dial and the leaderboard re-ranks live.
Last updated:
Benchmarks use Linux runners only, the Passmark single-thread metric, and the last 30 days of data.
cheaperfasterleaning faster
70% speed / 30% price
cpu × price × queuespot · us-east-1 · x64
who's who
RunsOn — self-hosted, your AWS account
GitHub — official, the reference price
AWS CodeBuild — managed, in your account
SaaS — Namespace, Blacksmith, Avrea, Warpbuild, Ubicloud
dot size = queue time
Smaller is better — the wait between trigger and first step. GitHub queues fastest; self-hosted spot trades a few seconds for much lower cost.
rank by
| # | provider | type | specs | cpu | queue | price / min | value |
|---|---|---|---|---|---|---|---|
| 1 | Namespacenscloud-ubuntu-24.04-amd64-2x8 | SaaSstandard | 4×8GBnorm $0.0024 | 4686 | 10s | $0.00401.5× cheaper | 90 |
| 2 | RunsOn ✦family=m8azn | Self-hostedspot | 2×8GB | 4266 | 22s | $0.00125.1× cheaper | 86 |
| 3 | Warpbuildwarp-ubuntu-2404-x64-2x | SaaS | 2×7GB | 4433 | 14s | $0.00401.5× cheaper | 78 |
| 4 | Avreaavrea-ubuntu-24.04-2-vcpu | SaaS | 2×8GB | 4355 | 12s | $0.00401.5× cheaper | 77 |
| 5 | Blacksmithblacksmith-2vcpu-ubuntu-2404 | SaaS | 2×8GB | 4330 | 14s | $0.00401.5× cheaper | 76 |
| 6 | RunsOn ✦family=m8a | Self-hostedspot | 2×8GB | 3803 | 22s | $0.00096.9× cheaper | 74 |
| 7 | Ubicloudubicloud-premium-2-ubuntu-2404 | SaaSpremium | 2×8GB | 3529 | 23s | $0.00203.0× cheaper | 60 |
| 8 | Warpbuildwarp-ubuntu-2404-x64-2x | SaaS | 2×7GB | 3788 | 10s | $0.00401.5× cheaper | 60 |
| 9 | RunsOn ✦family=m8i+r8i | Self-hostedspot | 2×8GB | 3195 | 24s | $0.00105.9× cheaper | 56 |
| 10 | StarSlingstarsling-ubuntu-24.04 | SaaS | 2×8GB | 3355 | 12s | $0.00401.5× cheaper | 47 |
| 11 | Ubicloudubicloud-standard-2-ubuntu-2404 | SaaSstandard | 2×8GB | 2923 | 24s | $0.00134.8× cheaper | 47 |
| 12 | RunsOn ✦family=c7a+m7a+r7a | Self-hostedspot | 2×8GB | 2874 | 26s | $0.00106.0× cheaper | 47 |
| 13 | Ubicloudubicloud-standard-2-ubuntu-2404 | SaaSstandard | 2×8GB | 2907 | 32s | $0.00134.8× cheaper | 45 |
| 14 | RunsOn ✦family=c7i+m7i+r7i | Self-hostedspot | 2×8GB | 2714 | 25s | $0.00096.5× cheaper | 43 |
| 15 | GitHububuntu-24.04, github24-2cpu-x64 | Official | 2×8GB | 3500 | 6s | $0.0060≈ GitHub | 42 |
| 16 | RunsOn ✦family=c6a+m6a+r6a | Self-hostedspot | 2×8GB | 2580 | 25s | $0.00106.2× cheaper | 39 |
| 17 | GitHubgithub24-2cpu-x64, ubuntu-24.04 | Official | 2×8GB | 3358 | 7s | $0.0060≈ GitHub | 37 |
| 18 | GitHubgithub24-2cpu-x64, ubuntu-24.04 | Official | 2×8GB | 2351 | 7s | $0.0060≈ GitHub | 6 |
| 19 | GitHububuntu-24.04, github24-2cpu-x64 | Official | 2×8GB | 2269 | 6s | $0.0060≈ GitHub | 4 |
| 20 | GitHubgithub24-2cpu-x64, ubuntu-24.04 | Official | 2×8GB | 2197 | 6s | $0.0060≈ GitHub | 2 |
| 21 | AWS CodeBuildcodebuild-ubuntu-8.0-large, codebuild-ubuntu-8.0-small, codebuild-ubuntu-8.0-medium | Self-hosted | 2×4GBnorm $0.0059 | 2141 | 24s | $0.00501.2× cheaper | 1 |
| 22 | AWS CodeBuildcodebuild-ubuntu-8.0-medium, codebuild-ubuntu-8.0-large, codebuild-ubuntu-8.0-small | Self-hosted | 2×4GBnorm $0.0059 | 2019 | 24s | $0.00501.2× cheaper | 1 |
How to choose#
$/minWant the lowest bill? Sort by cheapest. Self-hosted spot on AWS lands ~5.9× under GitHub — the cost goes to your AWS account, not a middleman.
cpuWant raw speed? Sort by fastest CPU. Namespace leads x64, followed by Warpbuild on Ryzen 9950X, Avrea, Blacksmith, and RunsOn m8azn.
queueLatency-sensitive? Sort by lowest queue. GitHub starts fastest; for self-hosted, warm pools cut RunsOn queues to under ~10s.
ctrlWant control & isolation? Pick self-hosted. RunsOn keeps runners, logs, caches and secrets inside your own VPC — no third party touches your code. See networking options.
Deep dive (full data)#
Open full benchmark table with distributions
Includes per-runner distributions, CPU/queue percentiles, and provider metadata for the last 30 days.
| # | Provider | CPU speed (p50) | Queue time (p50) + 30d P90 trend | Processor |
|---|---|---|---|---|
| 1 | Namespace SaaS nscloud-ubuntu-24.04-amd64-2x8 $0.004/min 1.5x cheaper standard | p50: 4686p90: 4717 | p50: 10.0sp90: 13.0s | AMD EPYC (x86_64) US/Europe (might vary) 29 samples |
| CPU 4500+ | ||||
| 2 | Warpbuild SaaS warp-ubuntu-2404-x64-2x $0.004/min 1.5x cheaper | p50: 4284p90: 4515 | p50: 14.0sp90: 16.0s | AMD Ryzen 9 9950X 16-Core Processor(x86_64) Hetzner / AWS / BYOC 20 samples |
| 3 | Avrea SaaS avrea-ubuntu-24.04-2-vcpu $0.004/min 1.5x cheaper | p50: 4354p90: 4484 | p50: 12.0sp90: 15.0s | AMD EPYC 4585PX 16-Core Processor(x86_64) Dedicated hardware 28 samples |
| 4 | Blacksmith SaaS blacksmith-2vcpu-ubuntu-2404 $0.004/min 1.5x cheaper | p50: 4282p90: 4482 | p50: 14.0sp90: 18.0s | AMD EPYC (x86_64) Hetzner 28 samples |
| 5 | RunsOn Self-Hosted runs-on,family=m8azn $0.0012/min 5.1x cheaper spot | p50: 4266p90: 4279 | p50: 22.0sp90: 29.0s | AMD EPYC 9R05 (x86_64) AWS 29 samples |
| CPU 4000+ | ||||
| 6 | RunsOn Self-Hosted runs-on,family=m8a $0.0009/min 6.9x cheaper spot | p50: 3803p90: 3827 | p50: 22.0sp90: 40.0s | AMD EPYC 9R45 (x86_64) AWS 29 samples |
| 7 | Warpbuild SaaS warp-ubuntu-2404-x64-2x $0.004/min 1.5x cheaper | p50: 3788p90: 3873 | p50: 10.0sp90: 18.0s | AMD Ryzen 9 7950X3D 16-Core Processor(x86_64) Hetzner / AWS / BYOC 9 samples |
| 8 | Ubicloud SaaS ubicloud-premium-2-ubuntu-2404 $0.002/min 3.0x cheaper premium | p50: 3529p90: 3738 | p50: 23.0sp90: 34.0s | AMD Ryzen 9 7950X3D 16-Core Processor(x86_64) Hetzner (Germany) 29 samples |
| 9 | GitHub Official ubuntu-24.04, github24-2cpu-x64 $0.006/min ≈ GitHub | p50: 3500p90: 3596 | p50: 6.0sp90: 8.0s | AMD EPYC 9V45 96-Core Processor(x86_64) Azure 3 samples |
| CPU 3500+ | ||||
| 10 | GitHub Official github24-2cpu-x64, ubuntu-24.04 $0.006/min ≈ GitHub | p50: 3358p90: 3382 | p50: 7.0sp90: 7.0s | Intel(R) Xeon(R) 6973P-C (x86_64) Azure 3 samples |
| 11 | StarSling SaaS starsling-ubuntu-24.04 $0.004/min 1.5x cheaper | p50: 3355p90: 4089 | p50: 12.0sp90: 14.0s | AMD EPYC (x86_64) Oracle Cloud 29 samples |
| 12 | RunsOn Self-Hosted runs-on,family=m8i+r8i $0.001/min 5.9x cheaper spot | p50: 3195p90: 3199 | p50: 24.0sp90: 43.0s | Intel(R) Xeon(R) 6975P-C (x86_64) AWS 29 samples |
| CPU 3000+ | ||||
| 13 | Ubicloud SaaS ubicloud-standard-2-ubuntu-2404 $0.0013/min 4.8x cheaper standard | p50: 2918p90: 2945 | p50: 24.0sp90: 34.0s | AMD EPYC 9454P 48-Core Processor(x86_64) Hetzner (Germany) 24 samples |
| 14 | Ubicloud SaaS ubicloud-standard-2-ubuntu-2404 $0.0013/min 4.8x cheaper standard | p50: 2907p90: 2937 | p50: 32.0sp90: 40.0s | AMD EPYC 9454 48-Core Processor(x86_64) Hetzner (Germany) 3 samples |
| 15 | RunsOn Self-Hosted runs-on,family=c7a+m7a+r7a $0.001/min 6.0x cheaper spot | p50: 2874p90: 2878 | p50: 26.0sp90: 51.0s | AMD EPYC 9R14 (x86_64) AWS 29 samples |
| 16 | RunsOn Self-Hosted runs-on,family=c7i+m7i+r7i $0.0009/min 6.5x cheaper spot | p50: 2714p90: 2919 | p50: 25.0sp90: 30.0s | Intel(R) Xeon(R) Platinum 8488C (x86_64) AWS 29 samples |
| 17 | RunsOn Self-Hosted runs-on,family=m6a $0.001/min 6.2x cheaper spot | p50: 2580p90: 2586 | p50: 25.0sp90: 26.0s | AMD EPYC 7R13 Processor (x86_64) AWS 29 samples |
| CPU 2500+ | ||||
| 18 | GitHub Official github24-2cpu-x64, ubuntu-24.04 $0.006/min ≈ GitHub | p50: 2351p90: 2828 | p50: 7.0sp90: 7.0s | INTEL(R) XEON(R) PLATINUM 8573C (x86_64) Azure 3 samples |
| 19 | GitHub Official ubuntu-24.04, github24-2cpu-x64 $0.006/min ≈ GitHub | p50: 2269p90: 2277 | p50: 6.0sp90: 8.0s | AMD EPYC 7763 64-Core Processor(x86_64) Azure 33 samples |
| 20 | GitHub Official github24-2cpu-x64, ubuntu-24.04 $0.006/min ≈ GitHub | p50: 2197p90: 2788 | p50: 6.0sp90: 8.0s | AMD EPYC 9V74 80-Core Processor(x86_64) Azure 15 samples |
| 21 | AWS CodeBuild Self-Hosted codebuild-ubuntu-8.0-large, codebuild-ubuntu-8.0-small, codebuild-ubuntu-8.0-medium $0.005/min 1.2x cheaper | p50: 2140p90: 2154 | p50: 24.0sp90: 29.0s | Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz (x86_64) AWS 40 samples |
| 22 | AWS CodeBuild Self-Hosted codebuild-ubuntu-8.0-medium, codebuild-ubuntu-8.0-large, codebuild-ubuntu-8.0-small $0.005/min 1.2x cheaper | p50: 2019p90: 2039 | p50: 24.0sp90: 27.0s | Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz (x86_64) AWS 47 samples |
| # | Provider | CPU speed (p50) | Queue time (p50) + 30d P90 trend | Processor |
|---|---|---|---|---|
| 1 | Avrea SaaS avrea-ubuntu-24.04-arm-2-vcpu $0.012/min 2.4x pricier | p50: 4840p90: 4897 | p50: 18.0sp90: 20.0s | Apple M5 Max + Virtualization Dedicated hardware 30 samples |
| 2 | Namespace SaaS nscloud-ubuntu-24.04-arm64-2x8,namespace-features:linux-on-apple-silicon=true $0.012/min 2.4x pricier premium | p50: 4802p90: 4924 | p50: 12.0sp90: 24.0s | Apple M4 + Virtualization US/Europe (might vary) 29 samples |
| CPU 4500+ | ||||
| 3 | RunsOn Self-Hosted runs-on,family=m9g $0.0009/min 5.5x cheaper spot | p50: 2462p90: 2467 | p50: 24.0sp90: 45.0s | Neoverse-V3 (aarch64) AWS 29 samples |
| CPU 2000+ | ||||
| 4 | Warpbuild SaaS warp-ubuntu-latest-arm64-2x $0.003/min 1.7x cheaper | p50: 1944p90: 1945 | p50: 19.0sp90: 40.0s | Neoverse-V2 (aarch64) Hetzner / AWS / BYOC 30 samples |
| 5 | RunsOn Self-Hosted runs-on,family=c8g+m8g+r8g $0.0009/min 5.7x cheaper spot | p50: 1945p90: 1947 | p50: 24.0sp90: 34.0s | Neoverse-V2 (aarch64) AWS 58 samples |
| 6 | GitHub Official github24-2cpu-arm64 $0.005/min ≈ GitHub | p50: 1874p90: 1878 | p50: 8.0sp90: 9.0s | Neoverse-N2 (aarch64) Azure 30 samples |
| 7 | RunsOn Self-Hosted runs-on,family=c7g+m7g+r7g $0.001/min 5.1x cheaper spot | p50: 1557p90: 1561 | p50: 25.0sp90: 45.0s | Neoverse-V1 (aarch64) AWS 29 samples |
| CPU 1500+ | ||||
| 8 | Blacksmith SaaS blacksmith-2vcpu-ubuntu-2404-arm $0.0025/min 2.0x cheaper | p50: 1347p90: 1375 | p50: 21.0sp90: 23.0s | Ampere-1a (aarch64) Hetzner 26 samples |
| 9 | Blacksmith SaaS blacksmith-2vcpu-ubuntu-2404-arm $0.0025/min 2.0x cheaper | p50: 1325p90: 1332 | p50: 18.0sp90: 19.0s | Neoverse-N1 (aarch64) Hetzner 4 samples |
| 10 | Ubicloud SaaS ubicloud-standard-2-arm-ubuntu-2404 $0.0013/min 4.0x cheaper standard | p50: 1309p90: 1315 | p50: 13.0sp90: 34.0s | Neoverse-N1 (aarch64) Hetzner (Germany) 30 samples |
| 11 | Namespace SaaS nscloud-ubuntu-24.04-arm64-2x8 $0.004/min 1.3x cheaper standard | p50: 1229p90: 1261 | p50: 20.0sp90: 26.0s | Ampere-1a (aarch64) US/Europe (might vary) 16 samples |
What the data says#
Performance
- –Namespace leads x64; Warpbuild on Ryzen 9950X ranks second, followed by Avrea, Blacksmith, and RunsOn m8azn.
- –Avrea’s Apple M5 Max leads arm64, followed by Namespace’s Apple M4 and RunsOn’s Neoverse V3.
- –GitHub runs older CPUs overall, most visibly on x64.
Reliability & scaling
- –GitHub’s queue times are excellent for standard runners — and much improved for larger ones.
- –AWS scales better than Hetzner-backed fleets under load.
- –Some SaaS providers gate higher vCPU concurrency (and static egress IPs) behind requests or fees; self-hosted RunsOn and CodeBuild aren’t affected.
- –CPU models on third-party SaaS runners can vary with capacity: the newest hardware may be available off-peak, while peak-time jobs land on slower CPUs. RunsOn launches the EC2 family you select, so the instance type is explicit.
Security & infra
- –RunsOn is fully self-hosted in your AWS account — your IAM, your network, a dedicated GitHub App.
- –AWS CodeBuild is managed by AWS but runs in your account (shared GitHub App for registration).
- –GitHub runners are managed by GitHub on Azure; private access needs secrets or OIDC.
- –Namespace, Blacksmith, Warpbuild, Ubicloud and Depot are SaaS; control plane and registration aren’t fully yours.
Gap: detailed concurrency/scaling tests are still missing; they matter for high-volume CI/CD (thousands of jobs/day). RunsOn routinely handles more than 1M jobs every day.
Benchmark changelog#
Routine benchmark sample refreshes, design-only changes, SEO edits, and internal refactors are omitted. These are the public-facing changes that affected provider coverage, pricing assumptions, scoring, or data filtering.
- Aug 2024 — The benchmark tracked CPU speed and queue time across the initial runner-provider set.
- Oct 2024 — Results moved toward p50/p95 CPU and queue distributions, and restricted or unavailable provider rows were handled explicitly.
- Jul 2025 — Provider summaries were revamped, and the median calculation was corrected.
- Oct 2025 — RunsOn m8a family results were added.
- Feb 2026 — The pricing-aware leaderboard was added, including provider pricing metadata, CPU-vs-price ranking, RunsOn EBS/license costs, and Cirrus monthly pricing normalization.
- Feb 2026 — RunsOn family-specific pricing, m8azn, spot/on-demand handling, p50/p90 CPU and queue percentiles, queue penalties, value scoring fixes, and family exclusions were added.
- Apr 2026 — BuildJet and Cirrus were removed from active benchmark/pricing coverage; Cirrus was removed after Cirrus Labs announced it was joining OpenAI and Cirrus Runners stopped accepting new customers.
- May 2026 — Shipfox was removed. Ubicloud standard runner pricing was updated from $0.0008 to $0.0010 per minute for the 2 vCPU runner after its 2026 price adjustment; premium runner pricing was unchanged.
- Jun 2026 — m9g was added, and value scoring started normalizing price by runner resources so differently sized runner shapes compare more fairly.
- Jun 2026 — Avrea was added with published pay-as-you-go pricing and Apple M5 Max ARM labeling.
- Jul 2026 — StarSling was added with published pay-as-you-go pricing for x64 Linux runners.
- Aug 2026 — Ubicloud standard and premium runner prices were increased by 25% ahead of the September 1 effective date. The 2 vCPU rates moved from $0.0010 to $0.00125 per minute for standard runners and from $0.0016 to $0.0020 per minute for premium runners.
Fastest self-hosted, at ~5.9× lower cost.
The benchmark runs on the same RunsOn anyone can deploy. One CloudFormation stack, your account, ~10 minutes.
Note: this analysis is updated regularly. Things change quickly in this space, so make sure you do your own research as well.