parquet-sampler
sample downsamples parquet files.
join reduces a dimension table using a downsampled fact table.
Build
git clone git@github.com:andreaskipf/parquet-sampler.git
cd parquet-sampler/
cargo build --release
cd target/release/
Sample fact tables individually
./sample attrib.parquet attrib_0.001.parquet 0.001
./sample fact.parquet fact_0.001.parquet 0.001
Semi-join reduce dimension tables
./join dim1.parquet d1_id fact_0.001.parquet f_d1_id dim1_sj_reduced.parquet
./join dim2.parquet d2_id attrib_0.001.parquet attrib_d2_id dim2_sj_reduced.parquet
./join dim3.parquet d3_id fact_0.001.parquet f_d3_id dim3_sj_reduced.parquet