Docs · Report Bug · Roadmap · Get Help · Watch Demo · Free Swag
Run any data transformation at TB scale for ClickHouse
GlassFlow is an open-source stream processing engine designed for high-volume data ingestion and transformation from multiple sources into ClickHouse. GlassFlow comes with the following core functionalities:
- Stateless transformations: Powered by the expr expression engine, enabling flexible data transformations using helper functions and standard operators (e.g., removing null values or replacing missing timestamps).
- Stateful transformations: A built-in state store allows deduplication logic and temporal joins over configurable time windows.
- Filtering: Drop events you don’t want to ingest into ClickHouse before they reach your tables.
- Ingest only: Direct data transfer from many sources to ClickHouse without transformations.
- Metrics & OTEL: Built-in pipeline metrics with OpenTelemetry support.
- Dead-Letter-Queue: Keep pipelines running when faulty events occur. Inspect failed events and reprocess them later.
⚡️ Quick Start
To get started with GlassFlow, you can:
- Try the Live Demo: Experience GlassFlow running on a live cluster at demo.glassflow.dev
- Install on Kubernetes: Follow our Kubernetes Installation Guide for any production deployment
- Learn More: Explore our Usage Guide to start creating data pipelines
🧭 Installation Options
GlassFlow is open source and can be self-hosted on Kubernetes. GlassFlow works with any managed Kubernetes service like AWS EKS, GKE, AKS, and more.
| Method | Use Case | Docs Link |
|---|---|---|
| ☸️ Kubernetes with Helm | Production and development deployment | Kubernetes Helm Guide |
🎥 Demo
Live Preview
Log in and see a working demo of GlassFlow running on a GPC cluster at demo.glassflow.dev. You will see a Grafana dashboard and the setup that we used.
Demo Video
📚 Documentation
For detailed documentation, visit docs.glassflow.dev. The documentation includes:
🆘 Support
⚖️ License
This project is licensed under the Apache License 2.0.