CompressARW

7 min read Original article ↗

compress-arw · wasm · 0.1.0

Lossless Sony ARW.
Pretends to be a JPEG.

Uncompressed 14-bit Sony raw is bulky because the sensor data sits in the file as 16-bit-aligned samples. CompressARW leaves the camera JPEG and EXIF at the front of the file, then packs the Bayer strip with same-channel deltas, a byte shuffle, and zstd. The result is a .ARWC.JPG. Finder, browsers, and exiftool stop at JPEG EOI. Decode splices the original ARW back together bit-for-bit.

macOS Apple Silicon · CLI macOS Intel · ARWC.app Windows x86_64 · ARWC.exe

  • ~½ typical size of the original ARW
  • 49% smaller on 10 A7R III files (zstd 19)
  • 817 → 414 MiB for that set · WASM level 9 is ~2 MiB larger per file

In this tab

The codec is the same Rust library compiled to WebAssembly. Nothing is uploaded. Use uncompressed .ARW from a body such as the α7R III (not Sony’s lossy “compressed RAW”). Default zstd level is 9, which the crate recommends for WASM.

Drop .ARW or .ARWC.JPG

or click to choose · several files run one after another

Which cameras

The codec does not look at the model name. It accepts Sony ARW whose raw strip is uncompressed 14-bit, 16-bit aligned, at EOF — the camera menu item RAW File Type → Uncompressed. Sony’s lossy “Compressed RAW” and lossless-compressed ARW 4.0 are refused.

Tested
Sony α7R III (ILCE-7RM3). The rates and fixtures on this page are from that body: 8000×5320, ~82 MiB per file.

α7R III
MENU → Camera Settings 1 → RAW File Type → Uncompressed. Single-shot and continuous stay 14-bit. BULB and Long Exposure NR drop to 12-bit and will be refused.

Same format
α7R II–VI, α7 II / III / IV, α7C / C II / CR, α7S II / III, α9 / II / III, α1 / II, α99 II, RX1R II / III, ILX-LR1 — when Uncompressed is selected and the file stays 14-bit.

Will not encode
Original α7 / α7R / α7S and most APS-C (compressed RAW only). α6700 (lossless compressed, no uncompressed option). 12-bit files from BULB, Long Exposure NR, or silent shooting on some older bodies.

Compression rates

Ten uncompressed Sony ILCE-7RM3 files, 8000×5320, 14-bit samples in 16-bit words (~82 MiB each). Same Bayer-delta + byte-shuffle + zstd pipeline this codec uses. Each file compressed on its own. Decode is bit-identical. Noisy or high-ISO frames shrink less; other bodies will differ.

817 → 414 MiB zstd 19 · 49% smaller · ~51% of original

817 → 429 MiB zstd 9 (WASM default) · ~2 MiB extra per file

File Original zstd 19 zstd 9 Saved at 19
OCT00527.ARW 81.9 MiB 46.0 MiB 47.6 MiB 43.9%
OCT01253.ARW 81.5 MiB 41.0 MiB 42.4 MiB 49.8%
OCT01840.ARW 81.6 MiB 39.2 MiB 40.8 MiB 52.0%
OCT02829.ARW 81.7 MiB 42.3 MiB 43.9 MiB 48.2%
OCT05023.ARW 81.7 MiB 38.9 MiB 40.6 MiB 52.3%
OCT05647.ARW 81.8 MiB 39.1 MiB 40.7 MiB 52.2%
OCT06077.ARW 81.9 MiB 40.6 MiB 42.2 MiB 50.4%
OCT08003.ARW 81.7 MiB 40.0 MiB 41.1 MiB 51.0%
OCT08389.ARW 81.8 MiB 43.9 MiB 45.4 MiB 46.4%
OCT08594.ARW 81.7 MiB 43.0 MiB 44.6 MiB 47.4%
Ten files 817 MiB 414 MiB 429 MiB 49%

Range on this set: 43.9%–52.3% smaller at level 19. Drop a file above to measure yours.

Compared with other lossless options

Same file: OCT00527.ARW, 81.90 MiB, uncompressed ILCE-7RM3. Generic tools were given the whole ARW. CompressARW first does a Bayer same-channel delta and a byte-shuffle of the 14-bit strip, then zstd — and still leaves a JPEG that Finder and exiftool can read. JPEG XL here is the mosaic strip only; the TIFF header would still need storing.

CompressARW · zstd 19 46.0 MiB

Shuffle + zstd 19 47.1 MiB

xz -9e on the ARW 50.9 MiB

JPEG XL lossless e7 51.2 MiB

xz -6 on the ARW 51.3 MiB

bzip2 -9 58.3 MiB

zstd -19 on the ARW 64.8 MiB

brotli -9 66.2 MiB

gzip -6 67.3 MiB

lz4 -9 80.6 MiB

Original ARW 81.9 MiB

Method Size Saved Native decode Notes
CompressARW, zstd 19 46.0 MiB 43.9% ~160 ms JPEG + EXIF in front; bit-identical ARW
Shuffle + zstd 19, no predictor 47.1 MiB 42.5% 76 ms Almost as small, slightly simpler
Same transform + xz -6 45.8 MiB 44.1% 990 ms 0.2 MiB smaller, ~10× slower
xz -6 on the raw ARW 51.3 MiB 37.4% 990 ms Best generic compressor; no JPEG face
xz -9e on the raw ARW 50.9 MiB 37.9% 3.0 s Too slow for WASM
JPEG XL lossless e7 51.2 MiB 37.5% slow Strip only; multi-MB decoder
zstd -19 on the raw ARW 64.8 MiB 20.8% 312 ms Leaves ~19 MiB on the table
brotli -9 66.2 MiB 19.2% 610 ms Browser-native, weak on this data
gzip -6 / zip 67.3 MiB 17.8% 267 ms Same class as a zip of the ARW
lz4 -9 80.6 MiB 1.6% 57 ms Fast, barely compresses

Not the same job

Option Original ARW bytes Opens as JPEG EXIF without unpacking
CompressARW .ARWC.JPG Yes, bit-identical Yes Yes
zip / gzip / xz / 7z of the ARW Yes No No
Sony “compressed RAW” (lossy) No No — still an ARW Yes
Sony lossless compressed (ARW 4.0) Yes, Sony’s codec No — still an ARW Yes
Lossless DNG of the file No — converted container No Usually

This tool only accepts uncompressed 14-bit ARW. Sony’s in-camera compressed and lossless-compressed modes are refused. zip is lossless but you cannot preview or grep EXIF until you unpack.

Why zip and zstd alone lose

Uncompressed Sony ARW stores 14-bit samples in little-endian 16-bit words. Every high byte is only 0–63, interleaved with the low bytes, so a generic compressor sees noise. Byte-shuffle splits those planes; zstd then crushes the high plane. A same-channel left delta (Bayer: subtract the pixel two to the left) saves another ~1 MiB. Bit-packing the 14 bits first mixed the planes and made zstd worse (66.9 MiB on this file).

Why not xz or JPEG XL

xz on the same transform is 0.2 MiB smaller and about 10× slower to decode. xz on the raw ARW with no transform is already 5 MiB behind. JPEG XL lossless was 51.2 MiB here with a multi-megabyte WASM decoder. gzip and brotli are convenient in the browser and still leave 20+ MiB on this file. Level 9 is the WASM encode default (~2 MiB larger than 19, much faster to compress).

How a .ARWC.JPG is laid out

The container is a normal JPEG with an ARWZ trailer after FF D9. You can range-request the leading preview. The raw is not required to view the file or to read EXIF.

Input
Uncompressed Sony ARW, 14-bit, 16-bit aligned, raw strip at EOF. Sony lossy compressed RAW and lossless-compressed ARW 4.0 are refused.

Rates
About half the original on uncompressed 42 MP A7R III files. Ten-file set: 817 → 414 MiB at zstd 19 (49% smaller); level 9 is ~2 MiB larger per file.

Preview / EXIF
Bytes 0 .. EOI only. Typically under 1 MiB. HTTP Range works.

Transform
Per-row same-channel left deltas on the CFA, then low/high byte planes, then zstd (level 19 on the CLI, 9 in WASM).

Roundtrip
Encode then decode must match the original file hash. Fixtures live in CompressARWTestData.

CLI, library, WASM, desktop

One crate. Default build is the CLI. WASM is a feature flag. The Electron viewer opens a folder of .ARW / .ARWC.JPG, shows the camera JPEG, and compresses or decompresses with the arrow keys.

Rust

use compress_arw::{decode, encode, extract_preview};

let jpg = encode(&arw_bytes)?;
let jpeg = extract_preview(&jpg)?;
let arw = decode(&jpg)?;

WASM

wasm-pack build --release --target web \
  -- --no-default-features --features wasm

encode(arw, level)
decode(file)
extract_preview(file)
preview_bytes(file)

Electron viewer

cd electron
npm install
npm start

← → browse · C compress
D decompress · O open folder