An Update on PromQL Compatibility Across Vendors
promlabs.comOut of all the projects/products on this list, we've found Victoria Metrics to be by _far_ the best.
This summary makes it seem like Victoria Metrics is barely compatible with Prometheus, but that can't be further from the truth in practice.
This is what is the problem with such compatibility tests... they tend to test full power of the language while you really may use quite small subset in your application, as such even solution which is "20% compatible" may well meet all your application needs.
I remember in its early days MySQL had pretty poor SQL support (if you think about full standard) which did not prevent it from having huge success.
Or more recent example ClickHouse which I think similar to VictoriaMetrics as it does not fully implement SQL, but also adds many convenient extensions which are not part of the standard.
Chances are if you chose VictoriaMetrics you will find a lot more utility in advanced features of MetricsQL than you loose from exact compatibility with PromQL
Did you start with Victoria directly? Or move to it from Prometheus? Keen to hear your experiences.
We're currently running Prometheus + Thanos, and high cardinality timeseries are a real issue, which Victoria claims to be good at.
We went from Prometheus (bursting at the seams and begging to be scaled) -> a failed Cortex implementation -> a rock solid Victoria Metrics implementation.
We found Cortex to be extremely over-engineered, extremely hard to tune (because of multiple configuration/argument refactors with incomplete doc cleanups) and found that it would just fall over under load. (Though to be fair, there wasn't much in the way of first-party deployment tooling at the time, so it was a hand-rolled Helm chart and at least 50% of the issues were likely my config).
In comparison, the first-party Victoria Metrics Helm chart worked straight out of the box, the maintainers have fixed multiple small issues within hours of me reporting them and we've thrown an extremely large amount of metrics at it with zero problems.
(Promscale Team Lead Here) Promscale, which connects Prometheus to TimescaleDB also handles high-cardinality well and allows you to use both PromQL and SQL for data analysis. I'd humbly suggest taking a look. Plus our PromQL implementation is 100% standards compliant.
It would be great conducting a benchmark between Promscale and VictoriaMetrics similar to this one - https://valyala.medium.com/prometheus-vs-victoriametrics-ben... . It differs from TSBS in the following aspects:
* It is based on real-world data instead of synthetic data.
* It measures resources usage (RAM, CPU, disk IO, disk space) during production-like load instead of measuring peak performance under the maximum synthetic load.
I would love to have some kind of integration between Victoria Metrics and Promscale, using VM for ingest from things like InfluxDB and Promscale for SQL queries on stable data.
This should be already possible - just collect Influx line protocol data with vmagent [1] and write it to Promscale.
Prometheus can't even do bulk import.
How do you include that in a CI/CD environment with regression testing?
This is great -- thanks for putting it together! Will these be kept up-to-date? If so, what will the cadence be?
Hi! On the road right now, but back later :) There's no established cadence yet, but twice a year or so sounds like a reasonable update period to me.
VictoriaMetrics CTO here.
These tests are great, because they spotted a few minor bugs in our PromQL implementation and these bugs were fixed quickly. See https://victoriametrics.github.io/CHANGELOG.html .
The majority of failed tests for VictoriaMetrics cannot be "fixed" due to deliberate choice made when designing MetricsQL: to rethink and to fix the most annoying and confusing parts of PromQL, while providing drop-in PromQL replacement for the majority of practical cases. See more details at https://victoriametrics.github.io/MetricsQL.html .