GitHub - antithesishq/bombadil: Property-based testing for web UIs, autonomously exploring and validating correctness properties, finding harder bugs earlier

Bombadil

Property-based testing for web UIs, autonomously exploring and validating correctness properties, finding harder bugs earlier.

Runs in your local developer environment, in CI, and inside Antithesis.

Note

Bombadil is new and experimental. Stuff is going to change in the early days. Even so, we hope you'll try it out!

How it works

As a user, you:

Write a specification:

A specification is a TypeScript module that exports properties.

Properties are linear temporal logic formulas, describing what the system under test should and shouldn't do. The "bombadil/defaults" module provides a set of reasonable properties for web applications. You may also specify your own domain-specific requirements.
Run tests:

When you have a specification, you run tests against a URL using that specification. This can be done locally, or in something like GitHub Actions.

This is unlike Selenium, Cypress, or Playwright, where you write fixed test cases. Instead, you define actions and properties, and Bombadil explores and tests your web application for you. This is property-based testing or fuzzing for web applications.

Examples

Starter (only using default properties)

This specification doesn't specify any custom properties at all, it just reexports the default ones provided by Bombadil:

export * from "bombadil/defaults";

Invariant

An invariant is a very common type of property; something that should always be true. Here's one that checks that there's always an <h1> element with some text in it:

import { always, extract } from "bombadil";

const title = extract((state) => state.document.querySelector("h1")?.textContent ?? "");

export const has_title = always(() => title.current.trim() !== "");

Guarantee

A guarantee property is where something good should happen within some bounded amount of time. Here's one that checks that, when something is loading, it eventually finishes loading and you see a result:

import { now, eventually, extract } from "bombadil";

const is_loading = extract((state) => !!state.document.querySelector("progress"));

const result = extract((state) =>
  state.document.querySelector(".result")?.textContent ?? null
);

export const finishes_loading = 
    now(() => is_loading.current)
        .implies(
            eventually(() => 
            !is_loading.current && result.current !== null
            ).within(5, "seconds")
        );

Usage

Start a test:

$ bombadil test https://example.com

Or headless (useful in CI):

$ bombadil test https://example.com --headless

Check custom properties defined in a specification file:

$ bombadil test https://example.com my-spec.ts

These will log any property violations they find. If you want to immediately exit, for instance when running in CI, run with --exit-on-violation:

$ bombadil test --exit-on-violation https://example.com my-spec.ts

You can also store the trace (a JSONL log file) by providing --output-path:

$ bombadil test --exit-on-violation --output-path=/tmp/my-test https://example.com my-spec.ts
$ head -n1 /tmp/my-test/trace.jsonl | jq .
{
  "url": "https://example.com",
  "hash_previous": null,
  "hash_current": 15313187356000757162,
  "action": null,
  "screenshot": "/tmp/my-test/screenshots/1770487569266229.webp",
  "violations": [],
  ...
}

Note

The format of JSONL traces is currently under development and might change.

Install

So far there's not a lot options for installing Bombadil other than using Nix. That's going to change though! We want to supply:

statically linked executables, which you can just download
Docker images
a GitHub Action, ready to be used in your CI configuration

But for now, your best bet is either running it through Nix, like:

nix run github:antithesishq/bombadil

Or setting up the developer environment and compiling it with Cargo:

nix develop
cargo build --release

More Resources

Contributing: if you want to hack on it
Quickstrom: a predecessor to Bombadil

Old Tom Bombadil is a merry fellow,
Bright blue his jacket is, and his boots are yellow.
Bugs have never fooled him yet, for Tom, he is the Master:
His specs are stronger specs, and his fuzzer is faster.

Built by Antithesis.