How We Configure Our Rails Local CI - Good Enough

5 min read Original article ↗

Continuous integration is a great thing, and having tests and security checks run before every deploy is also a great thing. But if you’re a developer who has been shipping production code for more than a week, you definitely understand how much it can all feel like a house of cards that tumbles down nearly every day.

The Good Enough suite of products have been using GitHub Actions to make sure our automated test suites run before each deployment. The (mostly free) servers GitHub offers are predictably slow, with the Pika test suite generally taking close to ten minutes to run. (To that you say, “Delete most of your system tests!” Alas, due to Pika’s lovely editor, we unfortunately have to maintain quite a few system tests for the service.) Even when upgrading, and paying for, a higher-strength GitHub Action server we were seeing runs approaching eight minutes for Pika.

That’s already no fun, but even worse is the fact that our system tests were a bit flaky in the GitHub Actions environment. We eventually got the hint that running system tests in parallel just isn’t possible, but even running them one test at a time would lead to odd failures in part because of how slow things  move in the Actions environment. So imagine the cycle of trying to deploy a Pika update and needing to run continuous integration two, three, or four times. Frustration!

There’s got to be a better way!

There is. Hopefully. With the arrival of Rails 8.1 came the option to set up local CI. As a team of two wanting to move a little more quickly and with a little less frustration, this seems like a perfect fit. Here’s how I’ve set it up for Pika…

ci.rb:

# Run using bin/ci

CI.run do
  step "Setup", "bin/setup --skip-server"

  step "Security: Gem audit", "bin/bundler-audit"
  step "Security: Brakeman code analysis", "bin/brakeman --quiet --no-pager --exit-on-warn --exit-on-error --confidence-level 2"
  step "Security: Importmap vulnerability audit", "bin/importmap audit"

  step "Tests: Rails", "bin/rails test"
  step "Tests: System", "bin/rails test:system"
  step "Tests: Seeds", "env RAILS_ENV=test bin/rails db:seed:replant"

  # Set a green GitHub commit status to unblock PR merge.
  # Requires the `gh` CLI and `gh extension install basecamp/gh-signoff`.
  if success?
    step "Signoff: All systems go. Ready for merge and deploy.", "gh signoff"
  else
    failure "Signoff: CI failed. Do not merge or deploy.", "Fix the issues and try again."
  end
end

In order for the importmap vulnerability audit to run successfully, I needed to update our gemfile with openssl:

group :development, :test do
  gem "openssl"
end

Here’s an excerpt of Pika’s application_system_test_case.rb:

ENV["PARALLEL_WORKERS"] ||= "1"  # System tests seem less flakey when not run in parallel
require "test_helper"

class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
  browser_options = Selenium::WebDriver::Chrome::Options.new.tap do |opts|
    opts.add_argument("--window-size=1200,800")
    opts.add_argument("--disable-extensions")
    # Disable non-foreground tabs from getting a lower process priority
    opts.add_argument("--disable-renderer-backgrounding")
    # Normally, Chrome will treat a 'foreground' tab instead as backgrounded if the surrounding
    # window is occluded (aka visually covered) by another window. This flag disables that.
    opts.add_argument("--disable-backgrounding-occluded-windows")
    # Suppress all permission prompts by automatically denying them.
    opts.add_argument("--deny-permission-prompts")
    opts.add_argument("--enable-automation")
  end

  Capybara.register_driver :chrome_headless do |app|
    browser_options.add_argument("--headless")
    Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
  end

  Capybara.register_driver :chrome do |app|
    Capybara::Selenium::Driver.new(app, browser: :chrome, options: browser_options)
  end

  if ENV["SYSTEM_TESTS_BROWSER"]
    driven_by :chrome, screen_size: [ 1200, 1000 ]
  else
    driven_by :chrome_headless, screen_size: [ 1200, 1000 ]
  end
end

Prerequisites to run local CI:

  • brew install gh

  • gh auth login

  • gh extension install basecamp/gh-signoff

  • Run: gh signoff install

This installs the GitHub command-line interface, installs the signoff extension for GitHub command-line, and turns on the signoff requirement in your repo.

Here’s the process:

  • Get all your changes pushed to a branch and make a PR

  • Make sure your local environment doesn’t have any lingering file changes or CI will fail

  • Run bin/ci

Running our Pika CI locally completes in under three minutes. That’s a big improvement! Upon successful completion of local CI, signoff will land on your branch, and you can merge and push to main.

If you ever need to move quickly, say in an emergency situation:

> gh signoff create -f
> git push

Since Lettini and I are both super-duper admins in our GitHub account, we needed one more update to protect us from willy-nilly pushing to main. I had to update a setting on GitHub in each repository. I clicked on Do not allow bypassing the above settings in repo > branches > Branch protection rules > main > edit:

It’s not all rainbows and unicorns

In an ideal world, hands-off CI is a really great thing. It will take a bit for these steps to become muscle memory. I hope they do! System tests are still notoriously flaky, but running tests only in our local environments means we shouldn’t have to account for both general flakiness and super-slow-test-running flakiness.

GitHub has a useful feature called Dependabot, which can apply security updates to your dependencies and create a pull request that’s often ready to merge. Sometimes we’ve just clicked that merge button in the past, feeling confident because our test suite had already run in GitHub Actions. Now we’ll have to pull down those branches to go through a local CI and signoff step in order to merge things.

If local CI doesn’t end up fitting us, I’ve also discovered there are faster, GitHub-Action-based alternatives for automating CI, such as Blacksmith. These services also have historically been cheaper than increasing server power at GitHub, though recent policy changes at GitHub have changed that math.

And a thank you

I’d be remiss if I didn’t thank 37signals for opening up their Fizzy repository. This helped me to really streamline our application_system_test_case.rb, which had become a Frankenstein’s monster of a thing as I troubleshot system test issues over the years.