parallel-compression

1 min read Original article ↗

http://zlib.net/pigz/ is "A parallel implementation of gzip for
modern multi-processor, multi-core machines", and also a replacement
(file-compatible) for the standard gzip/gunzip tools. Use pigz
-p<threads> or pigs -p <threads> (both forms are supported).


"For gz files compression can be parallelized effectively, but
decompression can't be parallelized, at least not without specially
prepared deflate streams for that purpose. As a result, pigz uses a
single thread (the main thread) for decompression, but will create
three other threads for reading, writing, and check calculation,
which can speed up decompression under some circumstances."


Full details here: http://zlib.net/pigz/pigz.pdf