Ask HN: How are engineers evaluating non-deterministic ML/LLM based deployments?
So, we process data as well as documents from various sources, then,
- convert all of its text (using different OCRs)
- pass it to LLM models - depending on the customer, it can be a cheaper model, and we do have model fallbacks
1. New models & new libraries are coming all the time
2. Even a third-party's deployment model will change over time and might improve/regress our systems
No comments yet.