CQ - run q/sql on parquet/iceberg in the cloud

2 min read Original article ↗

CQ - a new q/qsql runtime for your cloud data lake

Freedom to keep using q/qsql without having to ETL/ELT data into a proprietary database. No more gateways for handling concurrent queries — point CQ at an S3 bucket or an Apache Iceberg metadata catalog and get to work!

Features:

  • Read/Write Apache Iceberg tables with snapshots and time-travel support
  • Read/Write Parquet and gzipped CSVs on S3
  • Load multiple databases in one instance, each in its own namespace
  • Hive-style multi-column partitioning and predicate pushdown filtering
  • Compatible with the KDB IPC wire protocol
  • SIMD (AVX2) + multi-threaded support. Performance equal to or better than KDB
  • Compression support: gzip, lz4, zstd, and snappy
  • Thousands of grammar and feature tests
  • And much, much more...

Download

Free non-commercial trial version, exits after 4 hours. By downloading or using this software you agree to the License Agreement.

Latest version: 0.25.7

Install from the command line

curl -L https://cqdb.io/dist/cq-linux-x86_64.tar.gz | tar xz
./cq/cq
curl -L https://cqdb.io/dist/cq-macos-$(uname -m).tar.gz | tar xz
./cq/cq

Parameters

$ cq -h
cq — a terse array & query language

usage:
  cq                start the interactive REPL
  cq [file]         load a script (.q or .cq), then enter the REPL
  cq [file] -p N    listen on TCP port N for IPC clients
  cq [file] -epoch 0|1  temporal epoch: 0 = 2000.01.01, 1 = Unix (default)
  cq [file] -w N    cap workspace memory at N MB ('wsfull when exceeded)
  cq [file] -s N    use N secondary threads for peach
  cq [file] ARGS    extra args are exposed to the script as .z.x