Settings

Theme

Show HN: Proactive Security Monitoring for GitHub Actions Workflows

32 points by fkjadoon94 2 years ago · 7 comments · 3 min read


Hey HN, it's Farrukh and Umar. We're building listen.dev–a tool for proactive security monitoring in GitHub Actions to secure software releases from supply chain threats.

Why we built this:

As friends and collaborators for over a decade, we've been working on various startup ideas in dev tools and infrastructure. In 2017, while building an ML ops toolkit on Kubernetes, we got hacked. During a pilot with a fintech customer, our cluster became victim to a crypto-jacking attack.

As it turned out, a dependency in our container base image contained malware (a Monero miner) which triggered inside the customer's environment. Needless to say, we lost the customer and racked up a massive cloud bill as a tiny startup. This first-hand experience introduced us to one of the biggest challenges in software security today.

The Problem:

Modern engineering teams rely heavily on 3rd parties—from open source packages, base images and 3rd-party tooling to build & deploy software quickly. But this creates security blind spots exploited in modern supply chain attacks. Some high-profile cases targeting developer environments include:

(1) event-stream: a malicious transitive dependency injected a wallet-drainer payload into the build process for CoPay’s bitcoin wallet (2) SolarWinds: a compromised build tool injected malicious code into downstream releases (3) Codecov: a bash uploader script inside the testing tool stole secrets when run in CI

While most teams today incorporate some form of security scanning, it typically focuses on known vulnerabilities. In contrast, we detect zero-day threats and harden your build & release processes against malicious activity. With a focus on developer experience.

Enter listen.dev: a tool to analyze the behavior of your GitHub Actions workflows. How it works:

- Native integration via a simple workflow step. You can instrument your build, test, and release processes in any language or stack. - Observes low-level behaviors using eBPF (network, file, process signals) over each run - Detects anomalies and malicious activity using threat intelligence and out-of-the-box detections for known bads (e.g., info stealers making unknown network connections, reverse shells, tampering of builds etc.) - Offers in-line PR feedback with context, plugging into existing toolchains via webhooks

Behind listen.dev is a team of builders and OSS maintainers with years of experience in security observability and developer tools—having previously worked on eBPF runtime security projects like Falco and Tracee. We're seeking feedback from DevOps and security folks to help us improve.

You can sign up for free at https://lstn.dev/hn, install our GitHub action in under a minute, and start monitoring your repos.

We'd love to hear from you–any feedback and questions are welcome. To learn more see https://docs.listen.dev and a video of how it works: https://lstn.dev/demo-video

rst13 2 years ago

Interesting take. I’ve been looking a lot into hardening build clusters for our dev teams. We’re a k8s shop and use a mix of egress controls, host isolation, and Falco for detections (although we’ve had stability issues running the sidecar architecture).

- How do you capture context from the builds beyond vanilla ebpf traces? is there a way to see the detection ruleset/customize it

- I’m generally wary of running ebpf stuff on prod infra. Whats the threat model for your tool?

I'd love to give this a spin, but browsing through docs it seems like listen.dev works on Github hosted runners right now. Is self-hosted support on the roadmap?

  • rafaeldavidtin 2 years ago

    1. Feature isn't public yet, but currently we're able to block ingress/egress by IP + MASK and/or DOMAIN resolutions (meaning blocking the resolution itself, or anything that is resolved from or to example.com, synchronously while the resolution happens). This would be given as a setting (yaml?) to our CI action to be configured by the end user. L7 firewall is in our roadmap.

    2. Leo is former falco creator (sysdig), I'm former tracee core developer (aqua). For CI/CD, performance isn't usually a concern, unless the "sidecar" is consuming all runner resources and causing latencies/jitterings/false-positives in functional tests (saying this because your comment about bad experiences with side cars, or eBPF agents).

    3. I don't want to FUD but argus is currently consuming less than 10% resources when compared with other eBPF security tools around, with the same workload, and it does not miss events (not a single one don't matter what you do, without causing latencies to your code). Argus has a complete different architecture. You don't even need an userland agent running if that is mandatory (although it improves some available data). Userland code basically aggregates and report, almost no resource consumption.

    4. Although argus performs so well, we're targeting it for CI/CD now, until something changes (never know), but all its infrastructure is ready for staging and production environments (same info and detection events).

    5. About the threat model:

    - Everything is a resource (namespace, user, group, task, socket, network flow, file).

    - Resources suffer actions (different depending on the resource, files actions are heavily influenced by VFS operations like unlink, read, write, mmap, for example).

    - All resources can be stored for further analysis (historical deduplicated data of all actions that have ever happened to each resource).

    - Detections are built on top of this model: events are deduplicated and generated (or re-generated) only if the resource data has changed (like another action was taken on it).

    Example: process opens a file that triggers a detection (under certain conditions). A detection event is generated with an unique deduplicated id. The process exits and know the "task" resource has a new info (exittime), so the event is sent again with updated info (but same unique key).

    Putting all this together, our detections vary among Evasion, Brute Force, Credentials, Discovery, Drift, Escalation, Execution, Exfiltration, Lateral Move, Mining, Persistence, Tamper OS, Tamper Kernel, Tamper Argus, App Hooking, Kernel Hooking, Denial-of-Service, Offensive Tool, Informational. We're not mapping 1:1 with MITRE but some of our labels are a mix of couple (or sometimes 3) of MITRE classes.

    I hope it answers your questions for now.

    • rst13 2 years ago

      Thanks for the detailed reply! The arch and perf gains sound compelling.

      It might be a bit early for benchmarks, but I'd be curious to see how argus compares vis-à-vis other ebpf-based introspection tools at scale, in terms of stability and coverage... Probably makes for a good blog post for some day.

belikebakr 2 years ago

Hey, a couple questions:

1. Can listen.dev handle the monitoring of dynamically generated or short-lived GitHub Actions, which may only exist for the duration of a pull request? 2. How does listen.dev differentiate between normal and anomalous behavior without extensive prior data on a new user’s workflows?

  • rafaeldavidtin 2 years ago

    About short lived, or dynamically generated, steps: yes (or it is a bug if it can't). About tasks duration: it does not matter if a task existed for 1 ms or 10 mins, we catch what we need/want (with almost no overhead, at all).

    About differentiating behaviors, we are going to provide more details in documentation very soon.

    Thanks!

ghiaadev 2 years ago

Tried a test run for my node repo. It worked fine when I used the semver version of the action, but not when I pin the action to the latest release hash as shown in the GitHub marketplace listing docs (Shouldn't the safer practice here work by default??)

  • leodido 2 years ago

    Hello, glad your test run went well!

    Would you mind telling us more at support@listen.dev regarding the pinning? Not sure I got what you mean, just want to be sure. Feel free to send us the details at that email.

    Thank you, ~leo

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection