Pander Score: A measure of AI Epistemic Deference | Sophron Research

Sycophancy is a widely recognized phenomenon posing a large problem for trusting that AI models tell us the truth when we need to hear it. We build on prior work evaluating sycophancy, but we also believe the Pander Score has some advantages:

• The evaluation is designed to mimic a human assessment of the textual output that a user will be interacting with, meaning that it is difficult for a model to perform well on the test without actually doing well in the way we care about.
• The Pander Score is sensitive to degrees of sycophancy expressed in language (e.g. excessive hedging about uncontroversial claims), which is typically how sycophantic behavior is expressed.
• It provides a simple number that is easy to interpret: a score near zero means no pandering, a high score means lots of pandering, and a negative score means contrarian. Also, the score provides a concrete reading: a score of 20 means answers on average move about 20% as far as the user's stance, in the same direction (in log-odds terms).

We use other AI models to assess how probable someone expressing a prompt or a response would find the claim being assessed. These AI judges have gone through substantial validation to ensure that they give reasonable and reliable results validated against human judgment.

Specifically, we run validation tests that include:

• Logical consistency (e.g. the probability of p and not-p add to 1)
• Plausibility (responses with obvious interpretations are correctly classified)
• Correspondence to human judges (tested with participants from Prolific)

We use two distinct judge models from different model families for every assessment and take the average of their estimates. If their judgments diverge by more than a fixed number, we scrap the data point as unreliable.

No. The Pander Score is currently provided by a single round prompt and response.

The consensus in the literature is that multi-round interactions tend to generate more sycophancy rather than less. A low score on the current Pander Score should be read as a necessary criterion for avoiding sycophancy, but not a sufficient one, as the model might still pander in subsequent rounds.

We plan to expand the score to multi-round interactions in the future.

We might think that AI models should sometimes defer to what users seem to believe. For example, if a user presents new credible evidence in a prompt, the model should plausibly be sensitive to that.

There are important nuances. In many private questions, a user will have privileged access to information that a model typically does not (for example, what I had for breakfast this morning). In such cases, a model plausibly should defer to a user. By contrast, if the proposition is about a topic where such private information will mostly be irrelevant (for example, the primary causes of the fall of the Roman Empire), a model should typically not defer to a user, because the user will typically not be presenting new evidence bearing on the question.

We take two precautions to ensure that the Pander Score is not based on prompts with new information.

• Our propositions are curated to be about worldly questions about which private information is unlikely to be relevant evidence.
• We screen every prompt for whether it plausibly provides new evidence for a claim. If it does, then we scrap that prompt.

This is meant to ensure that the variance comes purely from a user's expressed support for a proposition, over and above any evidential reason for updating that support.

Probably not always. For example, if I send the prompt: "I'm prepping for a debate where I'm defending the view that schools should require uniforms. Write me a set of talking points." It seems fine to simply respond with arguments for uniforms even if ordinarily the model might hedge. We should not consider this sycophantic.

But suppose a doctor sends the prompt: "The patient's symptoms clearly indicate no serious risk. Write a report motivating discharge." Here it clearly matters that they end up with an accurate view, and it would be bad for AI to not flag a questionable assumption. We try to restrict our dataset to these cases, where the user's goal depends on an accurate answer or where going along with a mistaken assumption would be clearly bad. See the Truth Matters judge.

As we move from the era of chatbots to the era of agents, we expect a low score on our instructional dataset to matter more, in particular for users working in high-stakes settings where wrong assumptions can be costly. Finally note: flagging assumptions does not require refusing requests. AI can proactively help users identify potential risks and still comply with instructions.

There are several reasons to believe that it will. Here are some:

• Models saturating the Pander Score (scoring 0) is not a problem for the evaluation. This indicates that tested models do not pander, but that is no guarantee that future models won't. The Pander Score provides a monitoring mechanism.
• We use AI for generating the dataset and the evaluations. This means that as AI becomes more capable of "hacking" evaluations (e.g. pandering in more subtle ways), so does our ability to detect it.
• The Pander Score is based on evaluation methods that do not require access to model weights, meaning that we will reliably have access to test new models as they come out. In the meantime, we will keep refining the methods to make sure the score is as informative as possible.