Gantry: The tool to iterate on machine learning-powered products
gantry.iohow to filter failed cases automatically? (let's say, an object detection model)
manually inspecting all logs is not practical.
Josh here -- cofounder of Gantry.
Depends on the use case. If you're able to gather feedback from your users, then that's a great starting point.
If not, then it can be helpful to use a proxy to detect data points that are more likely than average to be failures. For example, a simple proxy is your model's confidence, but this is a deep topic. You can look at the literature on active learning if you want to learn more. This blog post is a great starting point: https://lilianweng.github.io/posts/2022-02-20-active-learnin...