Show HN: Using SIMD to decode and encode gigabytes of varints per second in Rust
github.comCan somebody explain what this is actually doing? I am quite confused
This is a Rust library for reading and writing the LEB128 integer compression format. LEB128 is a representation of arbitrary-size integers: https://en.wikipedia.org/wiki/LEB128
As I understand it, this library is optimized to avoid branching (which incur an overhead) and take advantage of SIMD instructions (which process data in parallel).