Show HN: Sqlite3-dump - a fast SQLite to CSV and parquet
github.coma fast zero-copy sqlite implementation only for dumping the contents of a sqlite3 file Thanks for sharing! If converting SQLite to CSV/parquet is a one-off task for you, you can also import SQLite files (among the other 50+ formats for tabular data) and export to multiple supported formats (CSV and parquet included) using Datagrok (https://datagrok.ai/). Just drag-and-drop the sqlite file, then use the "export" button on top. Everything happens inside the browser (so there is a limit on the max size of the dataset, probably around 1GB). Disclaimer - I'm one of the developers of Datagrok :) $ duckdb f.db -c 'COPY table1 TO table1.csv;COPY table1 TO table1.parquet;' on my machine that i did the basic run, the one in the link is way more faster. ```
$ time ./duckdb_cli-linux-amd64 ./basic_batched.db -c "COPY user TO 'user.csv'"
100% (00:00:20.55 elapsed) real 0m24.162s
user 0m22.505s
sys 0m1.988s
``` ```
$ time ./duckdb_cli-linux-amd64 ./basic_batched.db -c "COPY user TO 'user.parquet'"
100% (00:00:17.11 elapsed) real 0m20.970s
user 0m19.347s
sys 0m1.841s
``` ```
$ time cargo run --bin parquet --release -- basic_batched.db user -o out.parquet
Finished `release` profile [optimized] target(s) in 0.11s
Running `target/release/parquet basic_batched.db user -o out.parquet`
Database opened in 14.828µs SQLite to Parquet Exporter
==========================
Database: basic_batched.db
Page size: 4096 bytes
Text encoding: Utf8
Output: out.parquet
Batch size: 10000 Exporting table: user
Output file: out.parquet real 0m6.052s
user 0m10.455s
sys 0m0.537s
``` ```
$ time cargo run --bin csv --release -- basic_batched.db -t user -o out.csv
Finished `release` profile [optimized] target(s) in 0.03s
Running `target/release/csv basic_batched.db -t user -o out.csv` real 0m6.453s
user 0m5.252s
sys 0m1.196s
``` Nice! Thank you
Export completed successfully!
==========================
Table: user
Rows exported: 100000000
Time taken: 5.85s
Output file: out.parquet
Throughput: 17095564 rows/sec
File size: 310.01 MB user: 100000000 rows (310.01 MB) - 5.85s (17095636 rows/sec)