So, you think you've got dbt test bloat?
medium.comI did this write up of a recent dbt meetup talk by an analytics engineer from Delivery Hero. They had dbt tests getting out of control triggering hundreds of alerts. Their solution is a mix of categorizing critical models, weighting alerts, and formalizing the response.
The video is here if you want to watch that directly, instead: https://www.youtube.com/watch?v=Nk_K8mW-N9A
The whole subject of dbt tests, bloat or rot, is an interesting one. Here's a few stats from a couple of public facing dbt project:
- Mattermost: 194 models / 318 tests - Cal-ITP (California Integrated Travel Project): 361 models / 941 tests
What's the model/test ratio in your dbt project? How do you handle alerts?