Beating Go's encoding/json with Schema-Guided ProtoJSON

7 min read Original article ↗

In my previous article, benchmark results showed that Go’s official protojson package was surprisingly slow. Execution profiles consistently highlighted the same bottlenecks: reflection, pointer chasing, and descriptor traversal accounted for much of the latency and memory allocation.

ProtoJSON must preserve Protobuf semantics, including custom field names, enums, presence rules, and message structure, while producing standard JSON. The official implementation uses protoreflect to dynamically walk descriptors and inspect message structures on every call.

That raised a concrete question: how much of this cost is inherent to JSON formatting, and how much comes from repeatedly resolving schema mappings at runtime?

To answer this, I built protojsonx. Moving schema resolution out of the hot path brings ProtoJSON marshaling performance close to binary protobuf and ahead of both Go’s standard encoding/json and encoding/json/v2 in these benchmarks.

An experimental faster ProtoJSON encoder and decoder for Go.


Moving Schema Work Out of the Hot Path

The official implementation discovers how to map each protobuf field to JSON while processing the message. protojsonx resolves that mapping once and stores the result as a precomputed schema layout table.

It uses that precomputed schema information in two ways:

  • Runtime Table Mode: Builds flat schema layout tables once at startup initialization. This allows it to marshal and unmarshal messages using direct offset arithmetic instead of per-call descriptor traversal.
  • Generated Plugin Mode: Provides a protoc plugin (protoc-gen-go-protojsonx) that bakes type-specific marshaling and unmarshaling methods directly into generated .pb.go code, bypassing the runtime lookup layer entirely.

protojsonx is designed as a mostly API-compatible experimental alternative to Go’s official google.golang.org/protobuf/encoding/protojson package. It handles common rules including custom json_name, enums, field presence, unknown fields, oneofs, maps, and standard options (though experimental features like dynamic Any resolver callbacks are not yet supported).

Warning: protojsonx is experimental. It passes the official Protobuf conformance suite, but it has not yet seen enough production use for me to recommend it as a drop-in replacement.


Benchmark Setup

These benchmarks do not compare identical semantics. They answer a practical question: when an application needs ProtoJSON-compatible output, how expensive is that compared with Go’s general-purpose JSON packages?

Payloads and Sizes

I used the benchmark suite from my previous article, running on Go 1.26 on an Apple M1 Pro across three payload shapes:

  • Small: A flat object with 4 fields (string ID, status boolean, age integer, score float).
  • Medium: A nested user signup event containing an actor object, string tags, and a metadata map.
  • Large: An array repeating the Medium object 100 times.

Serialization output size matters, especially when comparing JSON and binary protobuf:

PayloadProtoJSON bytesgeneric JSON bytesbinary proto bytes
Small55 B55 B25 B
Medium293 B291 B162 B
Large29,412 B29,201 B16,500 B

The ProtoJSON and generic JSON sizes are nearly identical. Binary protobuf is significantly smaller, so binary numbers serve as a compact wire-format baseline rather than a direct format equivalent.

Compared Implementations

All tests measure end-to-end marshal and unmarshal execution times along with allocations.

  • The generic JSON cases (encoding/json and encoding/json/v2 from github.com/go-json-experiment/json) use native Go structs with equivalent fields.
  • The Protobuf cases use standard generated messages.
  • I also included hyperpb (a descriptor/layout-driven dynamic protobuf parser built around read-oriented offset decoding) and hyperpb.Shared (where the benchmark reuses an arena buffer). While hyperpb is not a ProtoJSON library, its numbers provide context on raw binary parsing overhead.
Show environment details and test command
  • Go Version: go version go1.26.3 darwin/arm64
  • Machine Details: Apple M1 Pro (10-core CPU, 16GB unified memory, macOS), GOMAXPROCS=8
  • Target Library Version: github.com/sudorandom/[email protected]
  • Benchmark Commit: b8fff78c
  • Benchmark Source: Available in benchmarks/
  • Test Execution Command:
    go test -bench=. -benchmem -benchtime=5s -count=5 > results.txt
    

The tables report arithmetic means for ns/op, B/op, and allocs/op computed from the raw five-run output.


Marshaling Performance

Marshaling starts with an already constructed Go message, so the encoder mainly needs to walk its fields and write JSON bytes.

Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson612 ns512 B121.0x (Baseline)
encoding/json183 ns64 B13.3x faster
encoding/json/v2289 ns112 B22.1x faster
protojsonx (Runtime Tables)142 ns64 B14.3x faster
protojsonx (Generated Plugin)102 ns64 B16.0x faster
proto.Marshal83 ns32 B17.4x faster
vtproto25 ns32 B124.5x faster
hyperpb + Shared296 ns144 B72.1x faster
Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson2,227 ns1,722 B341.0x (Baseline)
encoding/json521 ns464 B24.3x faster
encoding/json/v2803 ns608 B32.8x faster
protojsonx (Runtime Tables)404 ns320 B15.5x faster
protojsonx (Generated Plugin)318 ns320 B17.0x faster
proto.Marshal285 ns176 B17.8x faster
vtproto103 ns176 B121.6x faster
hyperpb + Shared1,062 ns744 B172.1x faster
Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson223,838 ns243,749 B27281.0x (Baseline)
encoding/json41,250 ns32,823 B25.4x faster
encoding/json/v262,036 ns32,856 B33.6x faster
protojsonx (Runtime Tables)31,134 ns32,797 B17.2x faster
protojsonx (Generated Plugin)27,601 ns32,784 B18.1x faster
proto.Marshal25,131 ns18,432 B18.9x faster
vtproto7,575 ns18,432 B129.5x faster
hyperpb + Shared103,495 ns107,216 B10222.2x faster

Interpreting Marshaling Results

Precomputing schema layout tables eliminates most of official protojson’s marshaling overhead. Both protojsonx modes reduce marshaling to one heap allocation in these benchmarks (64 B for small, 320 B for medium, and ~32 KB for large).

Across all payload sizes, runtime table mode gets surprisingly close to the generated plugin mode. The difference is 40 ns for the small message (142 ns vs 102 ns), and about 11% for the large case (31.1 µs vs 27.6 µs). With descriptor traversal gone, the remaining gap likely comes from interpreting the table at runtime instead of executing generated field-specific code. Projects with many generated message types will pay for that extra speed through additional generated code and a larger binary.

One result unrelated to protojsonx also stood out: encoding/json/v2 was consistently slower than v1 during marshaling (for example, 62 µs vs 41 µs on large payloads). That may reflect additional state tracking and the costs of its more flexible implementation, though the benchmark does not isolate the cause.

On the large payload, generated protojsonx took 27.6 µs, compared with 25.1 µs for proto.Marshal. I did not expect JSON encoding to get that close. For this payload, once descriptor traversal was removed, writing field names and formatted values added surprisingly little time over binary encoding.


Unmarshaling Performance

Marshaling showed that precomputed schema information eliminates most of the official implementation’s overhead. Decoding is a tougher test because avoiding reflection does not remove JSON tokenization, string parsing, or message construction.

Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson934 ns336 B141.0x (Baseline)
encoding/json752 ns280 B61.2x faster
encoding/json/v2339 ns48 B12.8x faster
protojsonx (Runtime Tables)267 ns96 B33.5x faster
protojsonx (Generated Plugin)136 ns96 B26.9x faster
proto.Unmarshal114 ns96 B28.2x faster
vtproto25 ns16 B137.4x faster
hyperpb360 ns798 B42.6x faster
hyperpb + Shared125 ns65 B17.5x faster
Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson3,703 ns1,304 B581.0x (Baseline)
encoding/json2,937 ns688 B191.3x faster
encoding/json/v21,133 ns256 B43.3x faster
protojsonx (Runtime Tables)1,108 ns576 B163.3x faster
protojsonx (Generated Plugin)720 ns528 B145.1x faster
proto.Unmarshal571 ns560 B156.5x faster
vtproto322 ns432 B1411.5x faster
hyperpb638 ns1,446 B55.8x faster
hyperpb + Shared290 ns357 B112.8x faster
Show complete data table
Format / Serializerns/opMemory (B/op)Allocations/opSpeed vs protojson
protojson377,892 ns119,256 B57131.0x (Baseline)
encoding/json272,103 ns70,584 B12161.4x faster
encoding/json/v2108,811 ns54,305 B3093.5x faster
protojsonx (Runtime Tables)111,141 ns62,232 B17093.4x faster
protojsonx (Generated Plugin)73,389 ns55,008 B14075.1x faster
proto.Unmarshal54,923 ns58,232 B15096.9x faster
vtproto36,002 ns58,168 B150810.5x faster
hyperpb24,835 ns60,053 B1215.2x faster
hyperpb + Shared18,163 ns21,863 B120.8x faster

Interpreting Unmarshaling Results

The generated decoder pulls further ahead than I expected. Runtime tables are almost twice as slow on the small payload (267 ns vs 136 ns) and remain about 34% slower on the large one (111.1 µs vs 73.3 µs). When parsing JSON dynamically, runtime table mode looks up field mappings in a table for every incoming key, whereas generated code emits direct message-specific key matching and field assignments.

The allocation count initially looks disappointing. Generated protojsonx still performs 1,407 allocations on the large payload. In this benchmark, decoding must allocate the nested messages, slices, strings, and maps that make up the result. Many of the remaining allocations therefore belong to constructing the output message graph rather than resolving its schema.

encoding/json/v2 performed much better than v1 during decoding, nearly matching runtime-table protojsonx on the large payload (108.8 µs vs 111.1 µs). Generated protojsonx remained faster (73.3 µs), which is consistent with its ability to emit message-specific field matching and assignment code without generic reflection overhead.


Conclusion

These benchmarks suggest that much of Go’s official protojson cost comes from resolving schemas at runtime rather than from JSON formatting alone. Precomputed tables recover most of the marshaling performance, while generated code has a larger advantage during decoding, where field matching happens for every JSON key.

That does not make JSON equivalent to binary protobuf. Tokenization, number parsing, and constructing the destination message still cost time and allocations. But the results show that ProtoJSON can be substantially faster than Go’s current general-purpose implementation.


What I Need Tested Next

If you are serving JSON APIs backed by Protobuf and protojson is showing up in your profiles, run the benchmarks against your own schemas and payloads before choosing an implementation.

I’m especially interested in benchmark results from real production schemas: large repeated fields, maps, oneofs, well-known types, custom json_name usage, and edge cases beyond standard benchmark structs.

Check out the project on GitHub: sudorandom/protojsonx.

The more weird schemas, the better.