Is it possible to require all GitHub Actions tasks to pass without enumerating them? · community · Discussion #26733

The Question

Is it possible to configure required status checks in a repo to say “all GitHub Actions tasks must pass”, instead of needing to select individual GitHub Actions tasks by name?

This is how Travis, CircleCI, AppVeyor, and Azure DevOps integrations with GitHub all work…the default behavior is that all tasks must pass, and you can selectively mark tasks as optional.

I’d like to be able to have an administrator set this one time and not have to change it every time we change our GitHub Actions flow.

Background Context

Consider the screenshot below:

This is taken from a pull request in a project I’m working on. The master branch of that repository has a GitHub Actions workflow with 4 tasks. A repo administrator has manually marked each of them as required, following the steps in “About required status checks”. You can see, for example, that a task r-package (macOS-latest, clang) is required.

The pull request I’m proposing changes that GitHub Actions workflow, such that there is no longer a task called r-package (macOS-latest, clang). I am now in a situation where the pull request can’t be merged by maintainers because it looks like the required tasks never ran, and there a bunch of new non-required tasks that passed. See microsoft/LightGBM#3065 (comment) for more background.

Thanks for the time and consideration!

@graingert it was discovered that when dependencies don’t succeed, that check job is marked as skipped which auto-merge treats the same way as success. I solved this problem with https://github.com/re-actors/alls-green.

Here's a short example:

---

on:
  pull_request:
  push:

jobs:
  tests:
    ...
    strategy:
      matrix:
        python-version:
        # - ~3.12.0-0  # doesn't yet exist as of Oct 04, 2022
        - ~3.11.0-0
        - >-
          3.10
        - 3.9
    ...
    continue-on-error: >-  # this allows "unstable" versions to fail, except for 3.11, meaning 3.12 alpha when it's added
      ${{
        (
          startsWith(matrix.python-version, '~')
          && !startsW…

View full answer

Unfortunately there isn’t a way to do this. I’ve had this issue in the past as well where I had a path exclusion filter set up to not run build workflows on non-code changes and they similarly blocked merges. I had to end up doing this as a workaround.

1 reply

oh sad. Yeah I guess it is just something we’ll have to deal with. Not a big deal! Overall I’m still really liking GitHub Actions.

Thanks for the help.

0 replies

oooo cool idea! That’s very clever, I like it. I think that would solve what I’m looking for, thank you.

0 replies

Here's a short example:

---

on:
  pull_request:
  push:

jobs:
  tests:
    ...
    strategy:
      matrix:
        python-version:
        # - ~3.12.0-0  # doesn't yet exist as of Oct 04, 2022
        - ~3.11.0-0
        - >-
          3.10
        - 3.9
    ...
    continue-on-error: >-  # this allows "unstable" versions to fail, except for 3.11, meaning 3.12 alpha when it's added
      ${{
        (
          startsWith(matrix.python-version, '~')
          && !startsWith(matrix.python-version, '~3.11')
        )
        && true
        || false
      }}
    ...
    steps:
    - ...

  check:  # This job does nothing and is only used for the branch protection
    if: always()

    needs:
    - tests

    runs-on: ubuntu-latest

    steps:
    - name: Decide whether the needed jobs succeeded or failed
      uses: re-actors/alls-green@release/v1
      with:
        jobs: ${{ toJSON(needs) }}

...

See https://github.com/marketplace/actions/alls-green#why.

12 replies

This is a very cool Action, thanks for that. But it works only for the jobs of one workflow. In the case where multiple workflows are triggered, and you want all of them to pass, currently there is no solution.

@nikaro The solution is to use reusable workflows or have the action integrated into several workflows under different names and all added to the branch protection. I should probably document that.
I've also seen actions that allow polling to wait other checks so that's another way.
My preference is using the reusable workflows:

It allows to modularize different job definition, separating them logically even within the same repo
I came up with an approach of having such workflows prefixed with reusable- for visual distinction in the file list. They also only have workflow_call: trigger, no others.
I tend to call the main workflow definition ci-cd.yml and that's where I add alls-green. If it only consists of the other workflow calls, it's easier to reason about structuring the job dependencies on the high level. It also lets me avoid having a million lines of YAML in a single file.
Bonus: it's possible to make a separate workflow that runs on cron and include that ci-cd.yml (this is a good workaround for low-activity repos where the workflows with cron tend to become disabled periodically, so it wouldn't affect the PRs)

@nikaro The solution is to use reusable workflows or have the action integrated into several workflows under different names and all added to the branch protection. I should probably document that. I've also seen actions that allow polling to wait other checks so that's another way. My preference is using the reusable workflows:

It allows to modularize different job definition, separating them logically even within the same repo

I came up with an approach of having such workflows prefixed with reusable- for visual distinction in the file list. They also only have workflow_call: trigger, no others.

I tend to call the main workflow definition ci-cd.yml and that's where I add alls-green. If it only consists of the other workflow calls, it's easier to reason about structuring the job dependencies on the high level. It also lets me avoid having a million lines of YAML in a single file.

Bonus: it's possible to make a separate workflow that runs on cron and include that ci-cd.yml (this is a good workaround for low-activity repos where the workflows with cron tend to become disabled periodically, so it wouldn't affect the PRs)

Doing it the way you suggested would make the workflows file harder to read, and it actually seems more troublesome than manually adding the required checks one by one.

I disagree. Having been using the pattern for long enough. It actually makes things structured and easier to maintain due to separation of the abstraction layers. I've heard positive feedback from other projects where I implemented this too.

How can we add/remove Details link ?
How can we re-direct to some url using Details link ?

1 reply

That would require a full GitHub App implementation, that's beyond what GitHub Actions CI/CD can provide you with. Though, https://github.com/marketplace/actions/alls-green does print out some rich output via a Job Summary interface (it shows up on the workflow summary page, below the job relationships graph).

I run an app, Mergery, that does this.

4 replies

Yes, some apps handle this properly. I think, Mergify does that too. However, this question seems to be more about GHA rather than external apps.

Not at all. In fact if you use Mergery it requires no code to be committed to set up, and because it's an app it avoids the catch-22 of wanting all actions to pass, but doing that check itself in an action.

Your statements seem to contradict each other, though.

I don't criticize, I pointed out above that such an app exists. And it's closed source too. Perhaps your wording confused me there.
It'd be nice to have this feature in GH, of course. Since it's not, my style is to make configs as code — straightforward and transparent. I know that apps can go beyond that since they can poll statuses from the old Statuses API as well as Checks API, which I didn't implement in the action, but maybe at some point I will.

This is really a problem, especially for large organizations/enterprises. AFAIK it is currently only possible to manually select status checks via "Require status checks to pass" at the repository or organization level, as part of a ruleset. An additional checkbox "Require all status checks to pass" would help a lot.

4 replies

And regarding the accepted answer re-actors/alls-green: it doesn't seem maintained (last commit 2 years ago)? But even if, I believe GitHub should a provide a native solution.

@beatngu13 there's a difference between unmaintained and does exactly what it's supposed to do.

@webknjaz It wasn't my intention to offend you or your project. I think it's reasonable that people (myself included) are cautious at first, when they see a project that hasn't had any commits in a few years. This is also why I explicitly formulated "doesn't seem maintained" as a question. If you're still maintaining it, great!

Nevertheless, given the number of upvotes and comments, I still believe GitHub should offer this feature natively.

I find the commit date the most unreasonable metric for whether a project is dead. Most of the time it's used to put pressure on the maintainers. I can have a bot making empty commits every day. It would make the metric green, but that wouldn't be useful beyond that. Sure, there are improvements to be made, but the main feature works and big projects are still using it, even without extra bells and whistles. It's not that hard to check how many projects use it: https://github.com/re-actors/alls-green/network/dependents / https://github.com/search?q=re-actors%2Falls-green+path%3A.github%2Fworkflows%2F+language%3AYAML&type=Code&s=indexed&o=desc.

As for GH offering this feature, perhaps somebody should talk to the support instead of a community forum. I'm pretty sure GH doesn't monitor this and all you'll achieve would be sending notifications to 11 participants and however many people subscribed to this discussion.

I have some ideas why GH doesn't provide it, though. They would have a hard time designing the UI for it. But I believe it's doable if they were to extend the workflow schema. Still, this is likely to be a can of worms, even with such a simplistic implementation due to the disconnected semantics of the branch protection and Checks API.

yep, this is shocking you can't require all configured checks to pass. I'd also expect this feature to be available

0 replies

it's 2025 and we still need this feature. not that commenting on a already resolved discussion would do anything

0 replies

We're facing the same challenge with GitHub's requirement to manually enumerate checks. This creates security risks and administrative overhead, especially for organizations with strict compliance requirements.
I see workarounds like alls-green and Mergery, but I'm curious:
How are organizations handling this in production?
Are there any enterprise-scale solutions that work well?
Has GitHub made any progress on this since 2020?
This seems like a basic security feature that should be native to GitHub. Would love to hear current best practices from the community.

0 replies

I also feel that the philosophy used by GitHub to handle this feature is flawed. As a user:

I add sets of assertions to ensure that my code is correct. This means that, by default, these assertions must pass.
For some reason (mainly performance-related), I may decide not to run certain assertions because they're not relevant to the changes in the PR. This use case is correctly handled by the GHA syntax, using the if feature.
And lastly, in exceptional cases, a job may be flaky, and for agility reasons, I might want to be able to merge even if some jobs fail. But this set of jobs is limited, and I should take an explicit action to allow them to fail — and, in theory, this is temporary. So maintaining this list is actually desired.

As a consequence, the default should be that all jobs must pass (no need to maintain a long list of jobs, or to break run optionality), and the exception should be to list flaky jobs (which is fine, since it's a short list).

But currently, the philosophy is exactly the opposite: all jobs are allowed to fail by default (with no option to switch to the other behavior), unless we explicitly list them as required. As a result, we must either:

Maintain a long list of required jobs, which breaks the optionality (actually a no-go),
Or create a unique mandatory dedicated job — we call it all-green — that check all others jobs, and explicitly list the flaky jobs as exceptions.

It really seems to me that this has been designed the wrong way 😢. The consequences of such a bad default are significant, especially considering that GitHub is by far the most widely used centralized repository. I'm genuinely surprised that there has been no reaction from GitHub.

2 replies

Note that not only the GHA statuses end up in the checks widget. But things reported by any integration. And even with GHA, GH has no way of knowing the comprehensive list of things that might get reported. Imagine a GitHub App (or an old-style integration) you installed starting to report a new status or behaving differently. In the fail-by-default scenario, it'd block merges seemingly arbitrarily. Another bit is how different things interpret same statuses. Neutral or skipped statuses are interpreted as successful, for example.

As for why GH doesn't explicitly state they don't want backwards incompatible changes — I think this is because this is a community rant forum and isn't an official support channel 🤷‍♂️

cc @karasowles would you be able to let somebody at GH know that this feature request concerns a lot of people?

Yeap, I also see many use case that make this feature not as obvious to do as it may looks like at the first place. But to me, they are only implementations difficulties.

My point was not "it's easy to do". My point is : it's how a PR engine should work by default.

And to be more precise, those "difficulties" are here because the PR engines has been designed with the "allow to fail by default" mindset. If by default your jobs are allowed to fail, the API model doesn't need a robust job started/finished abstraction.

So yes, it may be more complicated than just adding a checkbox in a UI.

But if you look at what would be the added value in term of quality for github users and enterprises, then maybe it's worthy to consider it.

As a user coming from years of using GitLab to using Github actions I must confess issues like these really makes a permanent palm on my face..
The logic you impose on much of your CI seems to be made by people who never use CI.
GitLab is in no ways perfect, but on the CI compared to Github actions years ahead.

0 replies