Ask HN: Why use Jev/decision models at all?
Honest question. Jev seems to cost 10x as much as Deepseek and I can easily just put the same payload to Deepseek and add "follow this strict format ...". What am I missing?
I found some success in using Jev/decision models by replacing some of my existing workflows which were using LLMs for classification tasks.
Replacing vector store: https://subworkflow.ai/blog/ragextract/rag-vs-jag Starter examples: https://community.n8n.io/t/n8n-jev-4-awesome-examples-to-get...
For me, it's less about cost and more about accuracy, scale and speed.
In real-world production scenarios, generation tasks are not that common; decision-making remains critical. We have recently been building an intelligent ticket-booking agent and have been using Qwen2-7B. Its inference speed is slow, and even after our optimizations, performance is still unsatisfactory. Under concurrency, it becomes practically unworkable. In fact, Qwen2-7B is mainly used here to parse user intent, which JEV can handle perfectly well. Based on our current empirical results, we plan to replace all open-ended branching steps within our complex agent with JEV.
If DeepSeek works better, use it. The benefit is faster, cheaper, and more accurate.
I'm using it in a RPG where it decides what dice to roll and what difficulty is needed to pass. Proper gen AI can handle the outcome of that roll.
It only makes sense if traditional LLMs are either too expensive or too slow.
That’s it, that’s the only use case.
However there are use cases that meet that requirement, for example, near instant labeling of dynamically generated content on the screen.
Check Deterministic State Machine idea - basically, on prod regulated / high stakes env. you don't want LLM to run almost anything directly.
You rather need exact reproducebale, tracable path of decisions. Either a hard-coded graph where LLM is a point of contact for the customer/user and parsing, function call etc. is all done deterministic way.
OR you use smth like JEV to decide about the next path in the chain (under the condition that JEV is trained for that path only and its only job is to do the best routing possible with close to 0 costs).
I find this answer a bit confusing, can you expand?
E.g. why is reproducible and traceable needed for LLMs and not for Jev.
posttrained models for a specific purpose usually outperforms model that are not posttrained for that purpose
i’m using it as a router.
i have side effects (schedule alert, assign to, set priority, etc)
it’s cheaper and faster to decide next secondary action (s) in addition to the primary action.
it keeps primary action focused and easier for evals.