Docs · Report Bug · Roadmap · Get Help · Watch Demo · Free Swag
Ingest your data into ClickHouse from day one
Backfill historical data, keep CDC in sync, handle schema changes, normalize messy data, and keep ClickHouse queries correct.
GlassFlow is an open-source stream processing engine designed for data ingestion and transformation from multiple sources into ClickHouse. GlassFlow comes with the following core functionalities:
- Stateless transformations: Powered by the expr expression engine, enabling flexible data transformations using helper functions and standard operators (e.g., removing null values or replacing missing timestamps).
- Stateful transformations: A built-in state store allows deduplication logic and temporal joins over configurable time windows.
- Filtering: Drop events you don’t want to ingest into ClickHouse before they reach your tables.
- Ingest only: Direct data transfer from many sources to ClickHouse without transformations.
- Metrics & OTEL: Built-in pipeline metrics with OpenTelemetry support.
- Dead-Letter-Queue: Keep pipelines running when faulty events occur. Inspect failed events and reprocess them later.
⚡️ Quick Start
To get started with GlassFlow, you can:
- Install on Kubernetes: Follow our Kubernetes Installation Guide for any production deployment
- Learn More: Explore our Usage Guide to start creating data pipelines
🧭 Installation Options
GlassFlow is open source and can be self-hosted on Kubernetes. GlassFlow works with any managed Kubernetes service like AWS EKS, GKE, AKS, and more.
| Method | Use Case | Docs Link |
|---|---|---|
| ☸️ Kubernetes with Helm | Production and development deployment | Kubernetes Helm Guide |
🎥 Demo
Demo Video
📚 Documentation
For detailed documentation, visit docs.glassflow.dev. The documentation includes:
🆘 Support
⚖️ License
This project is licensed under the Apache License 2.0.