4 days ago
DuckDB Power 💪 like no other!
We benchmarked Duckle against 4 open source ingestion tools.
Then we published the harness so you can prove us wrong.
96 million rows of TPC-H lineitem, 14 GB in Postgres, out to Parquet, one laptop.
Press enter or click to view image in full size
The harness ships in the repo. “./bench.sh” all brings up Postgres, generates the data at any scale factor, and times every tool you have installed. No run gets a time until its output has been reopened and checked for the right row count and the right sum(l_orderkey), so a tool that writes a fast wrong file gets a failure instead of a number.
Run it. If you get different numbers, we want to see them.
👉 Repo -https://github.com/slothflowlabs/duckle
👉 Benchmark -https://github.com/slothflowlabs/duckle/tree/main/benchmarks