GitHub - chrissnell/crabby: A web page performance tester with support for Prometheus, InfluxDB, Graphite, Datadog, and Riemann

crabby is a website performance tester that measures page load times and reports the measurements to a collection endpoint for processing, monitoring, and viewing. Crabby can collect and report these metrics:

• DNS resolution time
• TCP connection time
• TLS negotiation time
• HTTP Response Code
• Remote server processing time
• Time to first byte (TTFB)
• Server response time
• DOM rendering time

Crabby currently supports these metrics delivery backends. You can enable any combination of them simultaneously and Crabby will send metrics to all of them:

• Prometheus - Time measurements as metrics, exposed via a Prometheus endpoint
• DogStatsD - Time measurements as metrics via Datadog's DogStatsD protocol
• InfluxDB - Time measurements as metrics using the InfluxDB v2 wire protocol over HTTP
• Splunk - Metrics and events via Splunk HTTP Event Collector
• PagerDuty - Incident generation based on failed jobs via PagerDuty V2 Events API
• Log - Configurable flat-file or stdout logging of metrics and events

Three Types of Performance Measuring

Crabby has three types of probes for measuring website performance:

• simple uses Go's built-in net/http client to conduct HTTP requests. These requests measure server performance metrics including TLS negotiation time for HTTPS. This probe fetches the full response for a URL but will not fetch objects referenced by that page. The simple probe is appropriate for measuring app/API availability, time-to-first-byte (TTFB), DNS lookups, and TCP connection time. Being headless, it cannot measure DOM rendering time.

• browser uses chromedp (Chrome DevTools Protocol) to conduct browser-based performance tests via a headless Chrome instance. The browser probe is appropriate when page render time measurement is the primary concern. These tests pull down the page along with all objects included in the page.

• api allows you to define multi-step API workflows with template-based response chaining. Each step can reference values from previous steps' responses using {{ step_name.key.nested_key }} syntax. This is useful for testing authenticated API flows where a token from one request must be passed to subsequent requests.

Metrics Delivery

Prometheus

Crabby has first-class support for Prometheus. Crabby exposes a Prometheus endpoint that provides all gathered metrics. The config.yaml in the examples directory will get you started. Crabby applies global and per-job tags (if configured) as labels.

DogStatsD

Crabby can send metrics via the DogStatsD protocol, compatible with the Datadog Agent and other DogStatsD-compatible collectors.

InfluxDB

Crabby can send metrics to InfluxDB over HTTP/HTTPS using the InfluxDB v2 client API. Crabby can talk to InfluxDB directly or you can stand up a Telegraf instance to relay the metrics to InfluxDB (or some other datastore).

Splunk

Crabby supports sending metrics and events to Splunk via HTTP Event Collector. The config.yaml in the examples directory will get you started. The Splunk storage backend supports specifying the host, source and sourceType values for events and also to which index to append them.

PagerDuty

Crabby can generate incidents based on failed jobs. Using PagerDuty's V2 Events API, Crabby will generate an incident for jobs that result in a 4xx or 5xx response code. See CONFIGURATION.md for details on how to configure this storage backend.

Log

Crabby includes a configurable logging backend that can write metrics and events to stdout, stderr, or a file with customizable format strings and timestamps.

Using Crabby

Crabby is configured by a YAML file that you pass via the -config flag. If you don't pass the -config flag, Crabby looks for a config.yaml by default. This config file defines the sites to be tested (called "jobs"), as well as the metric storage destination(s) for the metrics that are generated. Crabby supports multiple metric storage backends simultaneously so you could, for example, send metrics to InfluxDB while simultaneously making them available via the Prometheus endpoint.

For metrics storage backends that support them, Crabby supports tags, both global tags applied to all jobs and their metrics, and per-job tags that are applied to a single job's metrics. In the event of tag name conflicts, per-job tags will override global tags.

Crabby can load secrets (API tokens, routing keys) from files instead of inlining them in config, which is useful for Kubernetes Secrets or Docker Secrets. Use the token-file or routing-key-file config fields to point to a mounted secret file. Crabby also sets a configurable User-Agent header on all HTTP requests.

Helm Chart (Kubernetes)

The recommended way to deploy Crabby is with the included Helm chart. Crabby includes a Helm chart for deploying to Kubernetes clusters.

Installation

helm install crabby ./helm/crabby -f my-values.yaml

Values Reference

Docker

A multi-stage Dockerfile is included in the repo. There's also a docker-compose.yml in the example/ directory to get you started.

To use Crabby in Docker, mount your config.yaml configuration file into the container and set the CRABBY_CONFIG environment variable to the location where you mounted it. The Docker Compose example handles this for you.

Binaries

If you prefer, binaries are available on the Releases page for a variety of architectures. After downloading the release, you can snag the config.yaml from the example/ directory to get started.

Crabby Configuration

Crabby is configured by a YAML file that you pass via the -config flag (defaults to config.yaml). See CONFIGURATION.md for a detailed description of this file. There is also an example, if you need one.

Contributing

Interested in adding a new storage backend or job type? See DEVELOPING.md for architecture details, project layout, and step-by-step guides for extending Crabby.