Which AI has the best sense of humor?

September 4: GPT-6 improves on GPT-5.6 Sol as a judge

GPT-6 Astra scored 69% across the four human-voted tests, compared with 62% for GPT-5.6 Sol. Each test contributes equally to that average.

The gain is 7.0 percentage points. In a comparison scored as a correct pick, an incorrect pick, or half credit when the two presentation orders disagree, that is about seven more points per hundred than GPT-5.6 Sol on this mix of tests.

The improvement is clearest on clean and dark jokes. Paired 95% bootstrap intervals exclude zero on the mixed, clean, and dark tests. On frontier-written jokes the gain is 1.5 points, with an interval from −4.2 to +6.9, so that test does not establish an improvement. These are individual intervals without adjustment for multiple comparisons.

Our judge still has the highest overall point estimate at 74%, 5.1 points above Astra. On clean jokes Astra scores 74% against our judge’s 73%, with a paired interval that includes zero. The remaining advantage is concentrated in other tests, especially mixed register. Several tests come from the same apps and reader population used to train our judge, which limits how broadly we can apply its lead.

Astra completed 1,833 pairs in both orders across six tests, with no missing verdicts. The four headline tests contain 1,339 pairs with a directional human majority. The curator test and the separate 100-pair clean test are excluded from the headline mean. An overlap check found no exact pairs or normalized jokes shared between these six tests and our judge’s training set.

The generation result uses our fine-tuned judge to score Astra’s writing. This test instead measures how Astra’s own choices agree with recorded human votes.

Which model agrees with people most?

For every pair of jokes, did the judge pick the same one as the majority of human raters?

Among the frontier models, GPT-6 Astra agrees with people the most

Overall averages four human-voted tests; the editor’s curated picks are shown separately. Shading marks each column’s best score. Hover for uncertainty ranges.

How wide are the gaps between the best and worst judges?

We took our main test and worked out the uncertainty range for each judge.

95% interval, marker at the point estimate

Does taste change on dark jokes?

We compared how well each frontier model matches human taste on clean jokes and on dark jokes.

The clean-to-dark change varies by model

Change is dark minus clean, in percentage points, largest gain to largest loss.

Can a judge rank comedians?

Picking the winner of one pair and ranking whole systems are different jobs. Judges shown in this chart scored 5,028 pairs from the SemEval-2026 MWAHAHA competition, where about a hundred annotators ranked 31 joke-writing systems.

Does a smarter model judge humor better?

We compared the models with an available LMArena text score against their agreement with people here. GPT-6 Astra is absent from this earlier comparison.

What do refusals and ties tell us?

Whether a judge answers at all, and whether it changes its mind when the jokes swap sides.

Claude Fable 5 refuses 5% of pairs. Nobody else refuses, and its successor Fable 5.1 answers everything

Refused is the share of judgments declined and excluded from scoring. Undecided is the share of pairs where swapping joke order changed the pick.

• More thinking does not help. In separate runs we tried the higher reasoning settings on Gemini 3.7 Flash, Gemini 3.1 Pro, GPT-5.5 and both Kimi models. None improved and several got worse, so the frontier runs used the low-reasoning requests and provider fallbacks listed in the methodology.
• Writing and judging give different rankings. Fable 5 leads the Humor Arena, while GPT-6 Astra has the highest agreement among the general judges. Astra places third as a writer by point estimate.
• Intelligence helps, moderately. Across the 15 frontier models, the LMArena text score and agreement with people correlate at +0.42 on a scale from -1 to +1 (rank correlation +0.30): a real but loose link, and with 15 models the uncertainty is wide. The top LMArena model, Claude Fable 5, sits mid-table here, so a high general score is no guarantee.
• Undecided pairs are common. Most judges change their pick on 15 to 30% of pairs when the jokes swap sides. Mistral Medium 3.5 flips on more than 40%.

The judge panel

The frontier models tested so far, earlier releases for comparison, and our own judge. The most recent addition is GPT-6 Astra on September 4, 2026.

• What it is. A Kimi K2.6 model fine-tuned to pick the funniest jokes (adapter kimi-k2p6-judge-haprolific), trained on 1,822 head-to-head votes from readers of our apps. It answers without explanation or reasoning.
• What it was checked against. A panel of 51 raters who never trained it: on the pairs where it commits to an answer, it agrees with that panel more often than an extra human rater does (72% against 52%).
• Where it stops. It declines to pick on about a quarter of pairs, and it is poor at ranking whole systems, so we route that job to a frontier model.
• None of these pairs were in its training. Every test on this page was checked for overlap against everything it was trained on.

• LLM and human judges never saw the author, model name, or vote count.
• Every pair was shown twice with the jokes swapped, so a judge that follows screen position counts as undecided.
• A judge scores 1 for picking the human side, 0 for the other side, 0.5 when its two answers disagree.
• The frontier models used one shared prompt and the low-reasoning requests and provider fallbacks listed below; our fine-tuned judge used its terse A/B protocol. The initial grid ran on August 25; later entrants used the same frozen tests. GPT-6 Astra ran through the direct OpenAI API on September 4.

Models, settings and judging materials

These are the requested settings recovered from the runner. Where the API rejected a setting, the runner tried the listed alternatives; it did not save which fallback succeeded. We cannot recover those accepted settings from the saved verdicts.

Caveats

We built the judge that wins most of these tables, so here is where the deck is stacked and where it is not.

• Home advantage. The mixed, clean and dark tests come from our own apps and largely the same readers whose votes trained our judge, and our judge answers in the format it was trained on while the frontier models got one untuned prompt. The frontier-jokes test is the fair one: 280 pairs of jokes written by the Humor Arena models, voted on by 51 raters who never trained our judge. Our judge scores 67.7% there and leads the best frontier models by 3 to 6 points, with overlapping intervals. Treat the frontier numbers as a floor.
• Ties and refusals. An undecided pair counts as half right for every judge, and a refused pair is dropped rather than scored wrong. On the pairs our judge commits to, its agreement is higher still, so the tie rule does not manufacture its lead.
• Humans disagree too. On the closest calls two people agree with each other only about 54% of the time, and two halves of the same panel agree on about 80% of pairs overall. This measures disagreement in this sample. It does not impose a fixed ceiling on every judge or test; use the reported intervals when comparing models.
• Not about joke writing. This page says nothing about which model writes the funniest jokes. That is the Humor Arena.