cl-brotli

This library was inspired by cl-zstd by Guillaume Le Vaillant: the file structure, gray-stream class design, API naming, and test approach all follow cl-zstd as closely as possible.

This is intentional: cl-zstd is a clean, well-structured library and cl-brotli is meant to be readable alongside it, but even more relevant for this choice, cl-zstd was already supported for SSE streams, making it a good starting point: not being an expert in compression streams, having a reference implementation to follow was fundamental.

Where the brotli C API forces a deviation, the source code includes a short comment explaining why. Readers familiar with cl-zstd should be able to read cl-brotli with minimal friction. The actual implementation is completely different, being based on two different libraries (libbrotli vs libzstd).

The brotli shared libraries must be installed:

# Debian/Ubuntu
apt install libbrotli1 libbrotli-dev

etc.

This is a library so usage will be done by applications; the following snippets show some of the common entry points:

  (ql:quickload :cl-brotli)

  ;; One-shot
  (let* ((data #(1 2 3 4 5))
         (compressed   (brotli:compress-buffer data))
         (decompressed (brotli:decompress-buffer compressed)))
    (equalp data decompressed))  ; => T

  ;; Streaming (encoder)
  (with-open-file (out "/tmp/data.br" :direction :output :element-type '(unsigned-byte 8))
    (brotli:with-compressing-stream (stream out :level 4)
      (write-sequence (cl-octet-streams:string-to-octets "some-bytes") stream)))

  ;; Streaming (decoder)
  (with-open-file (in "/tmp/data.br" :element-type '(unsigned-byte 8))
    (brotli:with-decompressing-stream (stream in)
      (read-sequence buffer stream)))

An Huchentoot minimal example is in examples/server.lisp:

  $ sbcl --load examples/server.lisp 
  [...]
  Serving on http://localhost:4242/hello  (C-c to stop)
  [...]
  127.0.0.1 - [2026-06-03 20:25:45] "GET /hello HTTP/1.1" 200 - "-" "curl/8.20.0"

Using curl:

  $ curl  --compressed -v http://localhost:4242/hello
  [...]
  > GET /hello HTTP/1.1
  > Host: localhost:4242
  > User-Agent: curl/8.20.0
  > Accept: */*
  > Accept-Encoding: deflate, gzip, br, zstd
  [...]
  < Server: Hunchentoot 1.3.1
  < Transfer-Encoding: chunked
  < Vary: Accept-Encoding
  < Content-Encoding: br
  < Content-Type: text/plain; charset=utf-8
  [...] 
  Hello, Brotli!

The API mirrors cl-zstd closely; this table shows the initial mapping:

A simple benchmark test was added to the test system; load it and call run-benchmarks:

(ql:quickload :cl-brotli-tests)
(brotli-benchmarks:run-benchmarks)

Three workloads are measured across 256 B / 4 KiB / 64 KiB / 1 MB payloads of "realistic" SSE-style text (event: / data: lines with varied counters). The following is captured:

• round-trip: one-shot batch API, compress-buffer + decompress-buffer per iteration.
• stream-flush: streaming (like SSE), one long-lived encoder, one finish-output per event. This is the workload that matters for real-time SSE use.
• decomp-stream: streaming decode, a fresh make-decompressing-stream per iteration.

(brotli-benchmarks:run-benchmarks :level 0)   ; fastest encoder, lower ratio
(brotli-benchmarks:run-benchmarks :level 9)   ; slower encoder, better ratio