A recent ask about an experience of leaving RSpec / Factorybot in favor of Minitest / Fixtures recently reminded me I'd never really looked back at the data I'd left for myself. In an effort to address these asks, I'm sharing my experience with this on an internal (for now) Ruby on Rails project called Nebula.
The Problem
Our test suite used to take about 14 minutes to run. That was RSpec + FactoryBot doing what they do; creating a ton of overhead we didn't realize was there. This effectively capped the number of cycles we could have on a given item to 4 / hr, which became untenable.
Our first attempt to recapture lost time was with test-prof. We spent a good amount of time optimizing with test-prof and specifically, the let_it_be helper, to reduce factory overhead. This got our runtime down to about 2 minutes 30 seconds. Felt like a huge win at the time. Until @dhh posted his results and I realized we had a lot more room for improvement.
At the time of this, Nebula's codebase and test suite were over 5x smaller and 2x as slow. That's broken.
The Migration
It started with a simple test, and before long it was clear that this was the path.
To stem the bleeding, the first step was to maintain 2 test suites. The rules were; nothing goes into RSpec. If you have to touch it and write a test, you write that test in Minitest.
This caused us to slowly convert Factories to Fixtures, sections of RSpec tests to Minitests, and eventually have so little left that it just made sense to pick off the remainder. This is the route I'd recommend to anyone going on this adventure.
During the transition period, we just ran both test suites to ensure coverage.
On migrating, I won't sugarcoat it...it's a pain in the ass. We tried having AI migrate everything, but it had a hell of a time understanding context inheritance and the expected test states from the RSpec tests. In the end, it was a combination of a little AI help once we established known patterns, and just a lot of manual effort to unwind the existing tests.
The Result
4 seconds.
Not a typo. Virtually the same suite, covering the same app, runs in 4 seconds. And here's the thing that makes it even crazier — the codebase actually grew significantly between when I ran the RSpec benchmarks and when we finished converting to Minitest.
The codebase is 2.6x larger. The test code is actually 26% smaller due to less ritual code. And it runs in 4 seconds vs 2+ minutes from the already-optimized RSpec setup.
RSpec is a nice DSL in theory. Writing specs in it feels great at first. But all that magic — lazy evaluation, implicit subjects, let chains, shared contexts add up in ways you don't see until it feels too late. FactoryBot on top of that just compounds the problem.
Minitest is just Ruby. There's no DSL to learn, nothing to debug that isn't your own code. Tests read like the code they're testing. And it's fast because it's multi-threaded by default.
Before this transition, we had to be choosy about when we ran tests, and would groan any time an AI agent fired off an unscoped test run or needed to run multiple times. Now, it's not even a thought.
Honestly, it's not even worth typing a command to scope the test suite when you can just bin/rails test in the same amount of time it'll take to type the path.
Test Details
All tests (RSpec and Minitest) were ran on the same system, a custom-built system running a Ryzen 9 9950X3D with parallelize(workers: :number_of_processors) on Minitest. With that removed, the runtime is 20 seconds.