Settings

Theme

Tensors vs. Tables: Why tabular tools trip over gridded data

earthmover.io

3 points by tomnicholas1 9 months ago · 1 comment

Reader

tomnicholas1OP 9 months ago

The scientific community works primarily with array (or "tensor") data, using tools like numpy, xarray, and zarr. People familiar with modern relational database tools such as DuckDB and Parquet often ask why can't we just use those? This article explains why: it's massively inefficient to use tabular tools on array data, and demonstrates with a benchmark showing a 10x difference in query speed.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection