How We Made JSON 3x Faster on the BEAM | PastelAds

When a user loads a webpage or mobile app with an ad slot, an ad exchange runs an instantaneous auction to sell that impression. Within tens of milliseconds, the exchange broadcasts the opportunity to dozens of competing bidders, collects their price offers, picks the winner, and serves the winning ad, all before the page finishes rendering.

These Real-Time Bidding (RTB) auctions use JSON as their wire format, whether you like it or not. The OpenRTB specification mandates JSON over HTTP, and public ad exchanges are not going to switch to Protobuf or FlatBuffers anytime soon. Every incoming bid request is JSON, every bid response we return is JSON, and an active bidder processes millions of these documents every second.

Why JSON overhead hurts a bidder

A bidder operates under a hard deadline: you typically have just a few milliseconds of total CPU budget per request. If you respond late, the exchange times out and drops your bid on the floor. Every microsecond spent in a JSON parser is time stolen from auction valuation, feature extraction, and pricing logic. With standard JSON libraries, parsing and serialization can easily burn 20% of your total per-request CPU budget.

Memory allocation is the second bottleneck. The BEAM garbage collector works by copying live data between generational heaps. When you parse a full JSON document, the runtime allocates a fresh term tree on the calling process heap. An OpenRTB request might contain dozens of nested fields, but a bidding engine often only needs to inspect a subset of them to evaluate and price the impression. The remaining unread terms become instant garbage. At thousands of bids per scheduler per second, that continuous allocation churn creates GC pressure that directly inflates your p99 latency tail.

Before Torque, our bidders ran on simdjsone, a NIF wrapping the C++ simdjson library. It was substantially faster than pure-Erlang or Elixir alternatives, but under high bidder load, we still wanted better throughput and lower heap churn.

We wrote Torque to replace simdjsone in that critical path. At its first release in March, Torque decoded bid requests 1.8x faster than simdjsone. Since then, after roughly 300 commits and 20 releases, we have pushed performance much further.

What Torque is

Torque is a JSON library for the BEAM with first-class Elixir and Erlang APIs. Under the hood, it binds to ByteDance’s sonic-rs SIMD JSON library using Rustler NIFs. We vendor a fork of sonic-rs directly in the Torque repository with a handful of targeted changes:

1. We exposed the sonic-rs visitor interface. This allows Torque to instantiate Erlang terms directly while the SIMD tokenizer scans tokens, rather than building an intermediate Rust AST first.
2. We enforced a strict recursion depth limit to protect dirty scheduler stacks.
3. We preserved arbitrary-precision integer representations for values exceeding 64 bits.
4. We implemented a single-pass pointer extraction engine.

The basic Elixir API is straightforward:

{:ok, data} = Torque.decode(~s({"name":"Alice","age":30}))
json = Torque.encode!(%{id: "abc", price: 1.5})

Torque satisfies the :json_library contract for Phoenix and Postgrex, so switching an existing application takes a single configuration line. Struct encoding is handled via the Torque.Encoder protocol, which also supports @derive.

Erlang is fully supported as well. The Hex package builds cleanly with both Mix and rebar3, and the torque module follows standard Erlang conventions: JSON null decodes to null, while a missing key evaluates to undefined:

{ok, #{<<"a">> := null}} = torque:decode(<<"{\"a\":null}">>).

A bidding engine almost never needs an entire bid request document. It typically only inspects a small fraction of the fields to evaluate and price an impression. Materializing a full BEAM term tree for everything else is wasted work.

The standard two-step approach uses parse/2 to obtain a reference to the parsed document, followed by get_many/2 to extract multiple fields in a single NIF call via JSON Pointers:

{:ok, doc} = Torque.parse(bid_request, unique_keys: true)

[{:ok, imp_id}, {:ok, domain}, {:ok, device_ip}] =
  Torque.get_many(doc, ["/imp/0/id", "/site/domain", "/device/ip"])

Passing unique_keys: true tells the lookup to stop at the first matching key rather than scanning the rest of the object. We enable this whenever we know the exchange produces valid, non-duplicate JSON.

However, parse/2 still constructs an internal sonic-rs document tree for the entire payload just to extract a subset of fields. Because the target JSON paths are identical across every request, we can do much better.

At boot time, compile_pointers/2 parses the pointer paths once and organizes them into a search trie. At runtime, parse_get_many_nil/2 streams over the raw JSON in a single pass, matching against the compiled pointer trie. Torque materializes BEAM terms only at the terminal nodes of those paths, skipping all unselected data without ever allocating an intermediate document tree:

# Once, at startup:
pointers =
  Torque.compile_pointers(
    ["/imp/0/id", "/site/domain", "/device/ip"],
    unique_keys: true
  )

# Per request:
{:ok, [imp_id, domain, device_ip]} =
  Torque.parse_get_many_nil(bid_request, pointers)

Both missing paths and explicit JSON null values resolve to nil, which is exactly what a bidder expects and allows for clean list pattern matching.

When a pointer path targets an object or array, decode/1 constructs the nested term using zero-copy sub-binaries and shared key caching. Strings longer than 64 bytes can point directly into the input binary, but Torque only does this when the cost is favorable: if a tiny string is extracted from a massive request body, Torque copies the bytes so the small term does not pin a large refc binary in memory.

Compiled pointers also support validate: false. In this mode, Torque skips tokenization across payload regions that no pointer targets, scanning only for structural delimiters (brackets and quotes). While it will not catch syntax errors inside skipped sections, it still enforces valid UTF-8 on extracted slices, catches malformed target values, and rejects truncated payloads. This is a great optimization for payloads generated by trusted upstream encoders.

Here is how these extraction strategies compare when extracting 5 fields from a 1.2 KB OpenRTB request on a single core of an Intel Xeon E5-2630 v3:

The compiled pointer path is 2.2x faster than parse + get_many, and 6x faster than glazer (which must fully decode the document before searching for fields). The gap is even wider on x86 than on Apple Silicon, where document tree construction carries higher overhead.

How we made it fast

Our initial implementation ran in two passes: sonic-rs parsed JSON into its internal AST, and a second pass traversed that AST to build Erlang terms. Over the last six months, we systematically eliminated redundant passes, intermediate copies, heap allocations, and unnecessary ERTS bookkeeping.

Single-pass decoding via visitors. decode/1 hooks directly into the sonic-rs visitor interface, instantiating Erlang terms on the fly as the SIMD tokenizer identifies tokens. Removing the intermediate AST gave us our single largest decoding speedup, as the two-pass approach was roughly 3x slower than the parser itself.

Zero-copy strings. When a JSON string contains no escape sequences, Torque returns a sub-binary referencing the input slice. Bypassing string allocation and copying sped up decoding by roughly 15%.

Shared key caching. Tabular and record-oriented JSON payloads repeat identical object keys hundreds of times. The decoder maintains a per-call key cache, ensuring identical keys share a single BEAM term. On record-shaped payloads, this reduced the decoded term heap size by 37% and lowered p99 latency by 16%.

Offloading ERTS map construction. Profiling revealed that ERTS was spending nearly 60% of decode time constructing maps. Small Erlang maps (flat maps) require keys to be stored in sorted order. By sorting keys in Rust before handing them over to ERTS, and by constructing empty and single-element maps through dedicated fast paths, we cut ERTS overhead significantly.

Thread-local scratch buffers. Previously, the decoder allocated and zeroed internal stacks on every invocation. We moved these parser stacks into thread-local buffers with fixed upper bounds. The decoder now writes only into the slots it actually uses, speeding up small document decoding by roughly 10%.

SIMD escaping in the encoder. The encoder combines UTF-8 validation and JSON escape scanning into a single SIMD pass, leveraging NEON on ARM and runtime-selected AVX2 (falling back to SSE2) on x86_64. Because most strings in a bid response are under 32 bytes, the encoder reads short strings in 8-byte chunks without SIMD overhead and emits escape sequences with two fixed-width stores. This reduced instruction counts by 13% on escape-heavy payloads.

Atom key caching. Bidders generally construct outgoing responses as maps with atom keys. Resolving atom names through the ERTS API is notoriously slow. Torque maintains a small per-scheduler atom-to-string cache. Because atoms are never garbage collected on the BEAM, cached pointers remain valid for the life of the VM. This took atom key encoding from being 62% slower than binary keys to 15% faster.

Zero caller-side allocation on encode. The encoder writes directly into a thread-local buffer in Rust and allocates the final output binary inside the NIF. As a result, the calling BEAM process allocates only 64 bytes of heap space regardless of whether it is encoding a 1 KB payload or a 750 KB document.

Profile-Guided Optimization (PGO). Our release CI builds an instrumented NIF, runs a representative JSON workload through it, and recompiles the final binaries with PGO profiles. This yields a 5% to 15% speedup over standard -O3 builds. Precompiled binaries for Linux x86_64 and Apple Silicon include PGO, with x86_64 builds targeting modern microarchitecture levels (Haswell and newer receive AVX2-optimized builds).

Scheduler cooperation and dirty schedulers. decode and parse report execution progress back to the BEAM using enif_consume_timeslice, counting one reduction for every 20 bytes processed. This ensures NIF work is charged accurately against process reduction budgets. Payloads over 20 KB are automatically scheduled on dirty CPU schedulers. Because encoding output size cannot be predicted upfront, encoders must explicitly set dirty: true to run on dirty schedulers.

Numbers

All benchmarks were run on a single core of an Intel Xeon E5-2630 v3 (Haswell, 2.4 GHz) on a shared host running OTP 29 and Elixir 1.20. While this CPU is older than modern developer laptops, it closely mirrors the server hardware found across our bidder fleet. Results for Apple M1 Pro and commit-by-commit tracking are available on our benchmarks dashboard. Relative library rankings remain identical across both architectures.

glazer is the successor to simdjsone, written by the same author, and represents Torque’s closest competitor. Both glazer and Torque were built with PGO for these tests. Because glazer disables UTF-8 validation by default while Torque always validates UTF-8, we enabled UTF-8 validation on glazer for parity.

Decoding a 1.2 KB OpenRTB document

Decoding a 750 KB document

For full-document decoding, both NIF libraries are close: Torque is 9% faster than glazer on small payloads and 4% faster on large documents (where glazer shows a marginally better p99). Both NIFs outperform OTP’s built-in json module by 3x to 4x.

Note on memory metrics: Benchee only measures process heap delta and cannot track terms constructed directly within NIF allocations. As a result, reported memory numbers for NIF libraries do not scale linearly with document size in Benchee reports, whereas pure-Erlang/Elixir libraries reflect full heap growth.

Encoding a 1.2 KB bid response

The input is an atom-keyed map matching what our bidder produces in production. Each library was evaluated using its respective iodata encoding function.

Encoding is where Torque made its biggest gains this year. Early on, OTP’s json module edged out Torque in this benchmark; today, Torque encodes 2.3x faster than OTP json and 2.2x faster than glazer, while allocating 60x less heap memory than OTP json.

Under sustained concurrency, allocation volume matters just as much as median latency because it dictates how hard the garbage collector runs. On the 750 KB document benchmark, Torque achieves a 1.34 ms median encode time compared to 2.79 ms for glazer and 8.60 ms for OTP json, while holding caller process allocation to just 64 bytes.

Limitations and Trade-offs

• Recursion depth limit: Nesting depth is capped at 128 levels. Payloads over 20 KB execute on dirty schedulers, which have constrained stack sizes; unconstrained nesting caused stack overflows in early testing. When hitting this boundary, Torque safely returns {:error, :nesting_too_deep} without crashing the VM.
• Integer map keys: Because JSON object keys must be strings, integer map keys are converted on encode. A term like %{1 => "a"} round-trips as %{"1" => "a"} (matching Jason’s behavior).
• validate: false risks: Compiled pointers with validation disabled skip syntax checking across unselected payload regions. Only use this option on inputs produced by conformant upstream encoders.
• Panic behavior: Torque is compiled with panic = "abort". A panic in the underlying Rust code terminates the BEAM node rather than raising an Erlang exception. We benchmarked unwind support and found that catching panics slowed down large encoding runs by 7% to 8%, so we chose raw performance over fault isolation.