InfluxDB 3 Open Source Now in Public Alpha Under MIT/Apache 2 License
influxdata.comI want to be excited, because this is such a smart combination of excellent utterly modern zero-copy data technologies. I love that Influx is using such a strong base of tech to build this.
> These products are built on more than four years of development and powered by the FDAP stack—Apache Flight, DataFusion, Arrow, and Parquet—and delivered on our rebuilt time series database architecture.
So awesome.
Its a bit harder to be quite as merry when the list of things that are enterprise only is so long. I know companies need to pay the bills, & don't begrudge them their decisions, but this still hurts;
> InfluxDB 3 Enterprise adds historical query capability, read replicas, high availability, scalability, and fine-grained security.
Having 72 hour max limit, single instance only, without high availability, and reduced security each seem like really big feature gates to deny the open source edition. Any even mildly serious adopter is probably going to need at least one of these.
I want to stay positive & hopeful & still am, but this feels like one of those really narrow open core strategies that makes me nervous.
We think that Core will fill some of the use cases of previous OSS versions of InfluxDB, but not all. But we also expect that Core will be useful in many more places that previous OSS versions of InfluxDB were not.
So Core isn't intended to be a full historical TSDB. It's more like a data collector, processing engine, data shipper and recent data buffer/DB.
For a full historical TSDB, that's the product we sell. Keeping the two separate gives us the ability to have real open source vs. combining them and requiring a different license that lets us do freemium.
We'll likely have a freemium tier for the commercial product (Enterprise), but that's separate from the open source project.
> InfluxDB 3.0 open source will be called InfluxDB 3 Core, a recent-data engine persisting Parquet files and enabling queries against the last 72 hours of data. Development of Core will carry on under the permissive MIT or Apache 2 license
> InfluxDB 3 Core gives developers a new tool for time series data management—a high-performance recent-data engine optimized for querying the last 72 hours of data. This focused approach enables Core to deliver exceptional performance for real-time monitoring, data collection, and streaming analytics use cases. By optimizing specifically for this pattern, we’ve achieved query response times under 10ms for last-value queries and under 50ms for hour-long ranges.
The limitation of 72 hours of data is a bit disappointing. There are use cases that don't require 50ms query response times, but instead need longer on-prem storge. InfluxDB 1.X and 2.X supported these well. Would it be possible to reduce query performance and extend data retention?
Is there any guidance on a planned EOL for InfluxDB 1.X and 2.X once Core reaches GR?
The data is persisted as Parquet files on object storage (or locally attached disk) and is queryable from any tool that can read Parquet. It isn't evicted by the DB, the 72 hour limit is just what is visible by the running database process.
It's a constraint that we could relax over time, but for now we wanted to limit the scope so we can focus on the recent data. We're also considering a free tier of Enterprise for at-home use cases (i.e. non-commercial hobbyist).
As for EOL on previous versions, we don't have anything planned at the moment. We're partnered with AWS on their hosted versions of InfluxDB 2.x OSS so I expect that to continue for quite some time.
Adding Merge Tree compaction, a la ClickHouse, should solve this, right?
That's right, compaction is the way to solve for performance over longer time ranges. This is what we have in our commercial Enterprise product.
It's a shame the v3 OSS version is simply not a functional alternative to InfluxDB v1 and v2. I understand money needs to be made, but not even being able to query further than 72 hours in the past... But seeing the changes from v1 to v2 to the development around v3, I am not surprised. InfluxData has been moving more and more into the direction of enterprise, honestly more surprised there is an OSS offering at all
I talk a little bit more about this comment on a different submission of this post: https://news.ycombinator.com/item?id=42704526
Can you say more about your use case?
Creator of InfluxDB and cofounder and CTO here. Happy to answer any questions. We're very excited to finally have this release in public. It's been years in the making and we hope to iterate quickly on any feedback.