Test analytics for GitHub and GitLab

Flakiness.io

2 min read Original article ↗

Frequently asked questions

Why is Flakiness.io cheaper than other platforms?

Flakiness.io charges for stored data, not per user or test run. The analytics engine is built on interval unions, which keeps large test histories efficient to pack and process at scale. Pricing follows that architecture: storage drives cost, not execution count.

How does Flakiness.io detect flakiness?

Every test result is tied to the commit and environment it ran on. A test that flips between pass and fail on the same commit is classified as a flake; a test whose outcome changes at a specific commit and stays that way is a regression. This separates noise from real breakage without rerunning anything.

The commit-aware analysis behind this is grounded in research published by Apple engineers.

Can Flakiness.io handle our massive monorepo?

Yes. Flakiness.io is built for large test histories, mixed stacks, and high test volume. We have tested the system on established projects with 500,000+ tests, and it handled the volume without issue.

Can I host Flakiness.io on my own servers?

Yes. A self-hosted deployment is one application container backed by PostgreSQL and an S3-compatible object store. The reporters and the Flakiness CLI work with custom deployments. See the self-hosting docs or contact [email protected] for licensing.

Who develops the test runner integrations?

More questions? Reach out at [email protected]