Jev, Prolog, Pi, and the dream of probabilistic logic programming

tldr; Jev is causing quite a stir and the latest versions of DeepClause and its extension for Pi now also support Jev. Jev is a natural match for DeepClause and its core concepts map nicely onto logical predicates in DML/Prolog. This gives us more speed and determinism for those use cases where a pure LLM/Agent approach would be either too expensive or too inderministic and it should greatly help for SOP2AGENT-style applications. Will it revive the older notion of probabilistic logic programming?

In case your maximum attention span is already below a 15s threshold, here is the quick summary:

1. Get a Typesafe API key

2. Install the Pi extension and let it use Jev to build things for you

If everything works out, this gives you a DML code like this:

This code can be run like this:

Or automatically by Pi depending on context if “/dc-tool enable” is set.

This will give something like

What’s happening here?

First, there is a “judge” predicate, which will call Jev (or an LLM as fallback) with the following:

1. A state consisting of a message and a policy (resp. instrructions)

2. Several questions for Jev. These can be

   1. “choose”: multiple choice type (“choice” in Jev)

   2. “rate”: multiple choice type “on a scale of 1-to…” (“score” in Jev)

   3. “verify”: is yes/no questions (“noul” in Jev, but directly checks if probability is > 0.5)

   4. “probability”: calibrated probability that something is true (“noul” in Jev)

Wait a second, I am really old and remember this has been done before!

Well…yes and no. For instance, there is a a paper called “Meta-Interpreters for Rule-Based Inference under Uncertainty” from 1990 which combines Prolog with predicates that have an assigned probability of being true. A meta-interpreter then performs a kind of probabilistic inference on queries and returns a likelihood that a given query is true or false.

Over the years, this idea has come and gone in many different forms. A more recent instance is "DeepProbLog”:

DeepProbLog is an extension to ProbLog, another probabilistic logic programming language and it extends ProbKog by adding “neural predicates”, allowing to do gradient descent through the entire program down to the models behind the neural predicates.

Unfortunately, neither of the above or similar approaches really took off, as they likely never really left academia and most examples stayed on the level of toy problems. A big problem here is of course: “Where do the probabilities comes from?” - and the old school answer, which always used to be “from Domain Experts!”, obviously doesn’t scale well.

Now, Jev is a model that’s likely been trained on huge amounts of data (bitter lesson, ey?) and can give reasonable probabilities (almost) for free and at scale. So finally, will the dream of probabilistic logic programming be realized?