Fast JSON Processing in Real-Time Systems: Simdjson and Zero-Copy Design

At Estuary, we’re creating a real-time data streaming platform that doesn’t rely on Kafka and uses JSON as the primary data format stored in an object storage. Many people are interested in how we achieve millisecond-level latency in our data streams, so we will be publishing a series of articles on this topic!

One of the bottlenecks of NodeJS is the json encoding and decoding time. And the way isolates work don’t allow you to offload the parsing, and provide a bottleneck for sending data to other threads.

Anything that reduces that provides a good deal more headroom for Amdahl’s law.