Rapid local development of custom Docker GitHub Actions

Date: 2023-03-11

Developing a custom GitHub Action that uses a Docker image is slow. Documentation for how to build an Action on top of closed-source code is hard to find. This post aims to fill the documentation gap and provide a solution to the slow development problem.

I'll lay out a simple "private" Go project to use as a basis for a custom Action, explain how to build a public Action for the project, and finally construct a fast, local toolchain for development.

The initial project

Let's say we have a Go project in a private repository that looks like this:

custom-action-demo-code
├── cmd
│   ├── myproject
│   │   └── main.go
├── go.mod
└── pkg
    └── mylib
        └── mylib.go

We normally release this project by building cmd/myproject/main.go and distributing the binary (possibly with Docker).

Turning the project into a GitHub Action

We now want to build a variant of this project to be distributed as a publicly-available GitHub Action.

First, let's update the project a bit for this use-case.

custom-action-demo-code
├── cmd
│   ├── myproject
│   │   └── main.go
│   └── myprojectvariant
│       ├── Dockerfile
│       └── main.go
├── go.mod
└── pkg
    └── mylib
        └── mylib.go

The program we want to run as our custom Action is cmd/myprojectvariant/main.go, packaged via the Dockerfile. All we need to do is follow GitHub's guide, right? Not quite.

Problem: Our source is closed

For our Action to be available to the public, its definition file "action.yaml" must be in a public GitHub repository. However, our project is closed-source and we don't want to open source it just for the sake of making our Action. This means the following type of Docker Action isn't available:

Solution: Public container references

Fortunately, GitHub has an answer. Docker Actions can use a pre-built container:

This means we can build a Docker image from our closed-source code and reference that image in the open-source action.yaml.

Let's make a new public repo for our action.yaml and add a testing workflow.

custom-action-demo
├── .github
│   └── workflows
│       └── test.yaml
├── README.md
└── action.yaml

This addresses the closed-source problem. Now we can push a public image, update our action.yaml with that image, and have a functional Action. However, there are problems with the testing experience.

Problem: Remote testing is a bad development experience

Imagine a full test cycle involving a code change in our closed-source repository:

Update mylib.go
Build a new Docker image
Push the new Docker image to a public registry
Update action.yaml with the new image tag
Push the action.yaml update to the public GitHub repository
Trigger a test run of action.yaml on the public GitHub repository

This is slow:

Step 3 has a lot of network overhead
Step 5 has a small amount of network overhead
Step 6 has overhead from spinning up a GitHub Actions runner and pulling the new image over the network

And there are other problems beyond speed:

Steps 3, 4, and 5 publish functionality that may not be ready for public consumption
Step 3 could incur cost if you are billed for registry space
Step 6 could incur cost because GitHub Actions runners aren't free

Solution: Local testing with `act`

There is a wonderful project called act which is designed to run Actions workflows locally. Our test is a workflow, so we can use act to run it.

In custom-action-demo:

$ act -l
Stage  Job ID  Job name  Workflow name  Workflow file  Events
0      test    test      Test           test.yaml      workflow_dispatch

We can run our test with act -j test, eliminating most problems with steps 4, 5, and 6.

We can fix the rest of our problems by taking advantage of the local Docker registry, which act can use to "pull" the image for our custom Action. Instead of a remote image, we can set the image: field of action.yaml to an image in our local registry.

Putting all of these ideas together, here's a new testing workflow:

Update mylib.go
Build a new Docker image
Update action.yaml with the new (local) image tag
Test with act -j test

All together now

Finally, we can package this flow into a single command. I'm going to wrap steps 2, 3, and 4 up using just; feel free to use your favorite tool instead, like make or a Bash script.

custom-action-demo
├── .github
│   └── workflows
│       └── test.yaml
├── README.md
├── action.yaml
└── justfile

Now a simple just test will build the image, update the Action, and run our test job locally! No network overhead, no Actions runner overhead, no losing focus.

Bonus: Building a Go project with Docker is slow

If you want to save even more time in local development, the go build step can be done outside of Docker to take advantage of the Go toolchain's caching. The resulting binary is then copied into a Docker container. Here's the new build definition in justfile and new Dockerfile:

Source code

Source code for this blog post can be found on GitHub: