I’ve previously written an introduction to multimodal LLM-as-a-judge [LINK], and a deep dive into long-form video understanding - the algorithms [LINK] and the benchmarks [LINK]. This post zooms out to the big picture of GenAI evaluation:
Why it’s the foundation of product and technical strategy, not a report card (Section 1).
How the building blocks compose into a single evaluation system (Section 2):
A rubric and a guideline define what you measure and how to apply it consistently. (A checklist is a special case of a rubric - see below.)
An LLM/MLLM judge and a human annotator are the two kinds of rater who apply them (Here the annotator is critiquing or scoring model output - distinct from the annotator who produces ground-truth labels.).
A benchmark works by freezing the whole package: the data, criteria, and scoring mechanism - whether an auto-rater, stored annotator labels (industry's "golden set"), or both - are locked together so results can be compared over time. That reusability is useful for short-term iterations, but it also limits the value of benchmarks over a longer product lifecycle, where everything else evolves.
These five building blocks are all offline - they score quality on an eval set. The online half is the A/B test on real users and creators, where product impact is estimated with causal inference (Section 3).
Which block you reach for is two questions at once (Section 4):
A measurement question - does this instrument measure the right thing (validity), and measure it consistently (reliability)?
An economics question - what does each evaluation cost in money, turnaround time, and updating effort?
What this looks like in practice, using image generation as the worked example throughout - chosen to offer a different angle (generation) from long-form video understanding.
One scope note before we go further: this post is about quality evaluation - is the output good, and does “good” actually move the product metric we care about? I’m deliberately not covering safety or policy evaluation here. Not because safety is secondary - it isn’t - but because it is a different problem with a different risk, impact, and cost profile. It deserves its own treatment, and folding it into this post would make the argument longer and less clear.
In generative AI products, evaluation is not the scoreboard after the game. It is the mechanism that decides what game the team is actually playing.
That statement is not an exaggeration - whatever your evaluation rewards becomes the product spec in practice. For example, if your image-generation eval scores prompt adherence but never aesthetic coherence, you will ship a model that follows instructions and looks wrong. The evaluation system operationalizes the project objectives - design it carelessly and you optimize for the wrong thing without knowing it.
GenAI evaluation is much more than measuring models - it needs to drive (at least) four different decisions:
Capability measurement and offline model selection → technical feasibility and design: is this even buildable, and which approach do we bet on?
Launch-readiness evaluation → the ship/no-ship decision: have we closed enough of the gap to launch?
Online behavior and feedback → which product gaps to close next: which user and creator needs are we still not meeting today?
Failure-mode diagnosis and root-cause analysis → R&D prioritization: what do we fix or build next?
These four are not a coincidence - they are the product lifecycle reflected in the evaluation system. And they all draw on two different regimes of measurement.
Offline: eval set, rubric, guideline, judge, annotator, benchmark. Everything offline rests on the eval set: its coverage, difficulty, and match to production traffic decide whether the whole stack measures anything real.
Online, you measure impact on real users and creators. Online has two modes: the A/B test asks is the new system better? (a controlled, causal comparison), and continuous monitoring asks is the live system still good? (drift, regressions, failures that appear only in production). This article focuses on the A/B test; continuous monitoring is its own topic and I won't go into it here. The offline metrics are proxies for these online outcomes - and the online metrics are themselves proxies for long-term user and creator value. Engagement can rise while value falls (e.g., novelty effect).
These two regimes hand off across the lifecycle rather than running in parallel. Feasibility and launch-readiness are settled almost entirely offline. The ship/no-ship call spans both: offline readiness has to clear the gate before you'd risk an experiment, and the A/B test makes the final call on live traffic. After launch the loop reverses - A/B tests and monitoring surface where the product still falls short, and offline evaluation diagnoses those gaps and ranks what to build next.
The rest of this article builds out the offline building blocks first, then the online A/B test and the causal inference that makes it trustworthy - with image generation as the running example.
Every evaluation building block is a measurement instrument, not ground truth - and an instrument has two ways to fail:
Validity (bias): are we measuring the right thing? A rubric whose only dimension is image saturation, or a judge that rewards one aesthetic style over others, is biased by construction - it does not measure the quality you actually care about.
Reliability (variance): are we measuring it consistently, with low enough noise to trust the result? A guideline that leaves room for subjective interpretation drives inter-annotator agreement down; an eval set that is representative but too small carries so much sampling noise that a real difference between two designs goes undetected.
Most evaluation decisions and debates are really one of these two in disguise. A rubric is mostly a validity instrument - it needs to capture all the quality dimensions that matter. A guideline is mostly a reliability instrument - its goal is to cut down the room for subjective interpretation.
The eval set is the anchor for every other offline building block: a perfect rubric scored by a perfect judge on the wrong examples will still mislead your product and technical strategy. Three properties decide whether an eval set is worth anything:
Coverage and representativeness. For image generation this can be framed as: does the distribution of prompts match what users and creators actually ask for? Popular academic and industry benchmarks are useful for a ballpark read on different models’ capabilities, but they are not your product’s traffic. If your users mostly want stylized portraits, a high compositional-reasoning score is measuring the wrong thing - a validity failure.
Slicing. A single aggregate score can't support principled product-gap analysis or R&D prioritization, because it hides heterogeneity - aggregate quality can rise while your highest-value slice falls. So you slice (by prompt type, content category, difficulty, user cohort) and read each slice on its own. Slicing buys validity - the average is not a valid read on the slice you actually care about - but it costs reliability: slice too finely and each slice has too few examples to trust. The discipline is finding the grain where slices are both meaningful and powered - validity and reliability pulling against each other (the classic bias-variance tradeoff).
Staleness and contamination. Any eval set you reuse across releases is effectively frozen, and freezing brings two slow failures: the set can leak into training data (contamination), and the production distribution can drift away from it (staleness). Both quietly destroy validity while the number keeps looking fine.
A rubric defines two things:
What you measure: the quality dimensions.
How you score: there are two modes:
Pointwise gives each dimension a scale (aesthetic coherence: 1-5), with every level a defined standard; a checklist is the binary (0/1) special case.
Its weakness is reliability: the more levels you add, the harder raters agree on fine distinctions.
Pairwise asks which of two outputs is better.
It is more reliable for subjective quality - relative judgments are easier and more consistent than absolute ones.
However, it only gives you a ranking, not a bar, and holistic preference can drift from the specific dimension you care about - a model can rank well on overall appeal while being weaker on prompt fidelity.
Pairwise generalizes to listwise (rank N outputs at once), which is more efficient - one pass instead of many comparisons - but ranking is a harder, noisier judgment than a binary choice, so you trade reliability back for that throughput.
Pointwise is reliability-fragile, pairwise/holistic is validity-fragile. Most image-gen leaderboards are pairwise for the reliability win; launch bars need at least some pointwise scoring for the threshold. In image generation these modes range from deterministic compositional checks to MLLM judges to crowd arenas - collected at the end of this section, where each system is read across the full stack.
A guideline is the annotation protocol that tells a rater - LLM judge or human annotator - how to apply the rubric: disambiguation rules, worked examples, tie-breaking. It is mostly a reliability instrument: it makes two annotators (or one judge across two runs) more likely to agree.
A rater applies the rubric and guideline. There are two kinds - the human annotator and the LLM/MLLM judge - with drastically different cost and trust profiles.
The human annotator is the closest thing to ground truth you have - and still not ground truth. Human annotation tends to be costly, slow, and noisy: inter-annotator agreement on subjective dimensions is often low, and for aesthetics worse, because taste is personal and culturally shaped. Its validity is determined by the rubric’s construct; its reliability hinges on the guideline. Measure inter-annotator agreement continuously - don’t assume it.
The LLM/MLLM judge is what makes evaluation scale - cheap, fast, runnable on every change. Its failures are mostly validity - systematic biases, not random error:
Length bias (longer, more detailed outputs score higher), position bias (in pairwise mode, presentation order sways the winner), and self-preference (the judge rewards outputs that resemble its own style).
Silent drift: the judge can be sensitive to the prompt, rubric wording, and examples - change any of them and a “fixed” judge isn’t fixed.
Reward hacking: once a judge becomes an optimization target, it gets gamed. The clearest evidence comes from judges distilled from human preference data - optimizing against them pushes generators toward surface aesthetics over prompt fidelity: vivid, over-saturated, high-appeal images that no longer match the prompt. (This issue was mostly discussed in the context of RL post-training, not as an offline eval issue, but the validity flaw is the same one you inherit if you use such a model to evaluate.)
A benchmark is the previous blocks frozen together: a fixed eval set, a fixed criteria, and a fixed scoring mechanism (an auto-rater, stored annotator labels - a golden set - or both). Freezing is what enables model comparison and regression detection. By construction it inherits every weakness of the eval set it froze - and it goes stale the moment your rubric itself evolves.
A golden set is a benchmark whose scoring is stored labels - and for image generation you usually can’t build one. There is no single correct image for “a cat wearing a hat,” so there is no gold label to store. That absence of a single ground truth is the norm, not the exception, in GenAI products. Rubrics, judges, and comparison arenas are what you reach for precisely because reference-free is the default.
Saturation and contamination both shrink a benchmark’s useful life, by related but distinct mechanisms. Contamination is Goodhart’s law in the open - the test leaks into training, so the score measures memorization, not generalization. Saturation is quieter - as models improve, a once-discriminating set stops separating them, so the number stops being informative even without any gaming. Both destroy validity while the number keeps looking fine.
Validity/reliability read: a benchmark’s headline number is only as valid as the eval set and rubric it froze, and only as reliable as the guideline and scoring mechanism behind it.
Every example below is a bundle - a particular set of choices across eval set, rubric, guideline, rater, and benchmark. In rough chronological order:
FID (Heusel et al., 2017): Distribution-level realism. Compares Inception feature statistics of generated vs reference images - an auto-rater with no rubric and no prompt, so it measures “do these look real in aggregate,” nothing about prompt match. Sensitive to sample size and backbone.
CLIPScore (Hessel et al., 2021): The first cheap reference-free alignment metric. Image-text cosine similarity in CLIP space - one continuous pointwise score, embedding model as rater. Broad but coarse: blind to composition, counting, and spatial relations.
GenEval (Ghosh et al., 2023): A compositional checklist. Decomposes prompts into verifiable items (object, count, color, position) scored by detectors and classifiers - a deterministic auto-rater. High reliability, validity bounded to enumerable composition; silent on aesthetics.
TIFA (Hu et al., 2023) / Davidsonian Scene Graph (Cho et al., 2023): Question-based checklists. An LLM decomposes the prompt into yes/no QA pairs answered by a VQA model, score = fraction correct; DSG adds dependency structure so a question is skipped when its premise fails. Same checklist philosophy as GenEval, via QA instead of detection.
VIEScore (Ku et al., 2023): MLLM-as-judge with a graded rubric. Prompts a general MLLM to score semantic consistency and perceptual quality 0-10 with a rationale - the closest thing to a classic rubric applied by a judge. Richer (covers quality + explanation), inherits judge bias and drift.
PickScore (Kirstain et al., 2023) / ImageReward (Xu et al., 2023) / HPS (Wu et al., 2023): Learned reward models. Trained on large human pairwise-preference datasets, then used as a frozen pointwise scorer - implicit rubric (alignment and aesthetics blended), distilled-model rater. Cheap and human-correlated, but this is exactly where the judge-section validity failure lives: they favor surface aesthetics over fidelity.
VQAScore + GenAI-Bench (Lin et al., 2024): Single-question continuous alignment. Rather than decompose, it asks one question and takes the VQA model’s probability of “yes” - a continuous pointwise score the authors report beats both prompt-decomposition and LLM-graded scoring on alignment. Not a checklist; alignment-only.
T2I-CompBench (Huang et al., 2023; ++ version - 2025) / UniGenBench++ (Wang et al., 2025) / WISE (Niu et al., 2025): Large structured benchmarks - frozen eval sets with multi-dimensional rubrics, scored by a mix of detectors and MLLM judges. T2I-CompBench++ targets compositionality; UniGenBench++ broadens coverage (multilingual, fine-grained); WISE probes world knowledge and commonsense. The current “eval set + rubric + auto-rater, frozen together.”
GenAI Arena (Jiang et al., 2024) / LMArena / Artificial Analysis / GenArena (Li et al., 2026): Live pairwise preference. Crowd votes on blind side-by-side outputs aggregate into a dynamic Elo leaderboard - live-human rater, implicit holistic rubric, a living rather than frozen benchmark. Most human-aligned, hardest to use as a launch bar. GenArena (2026) automates the same pairwise protocol with off-the-shelf VLMs, which beat specialized pointwise reward models without fine-tuning.
Takeaways:
The pattern is pretty clear: evaluation has been moving away from reference-based realism metrics like FID and toward richer raters - decomposed checklists, VQA models, MLLM judges, preference models, and live arenas. Reference-free evaluation is becoming the default for generative tasks.
The load-bearing decision is the rater: a detector, an embedding model, a VQA model, an MLLM, a reward model, or a live crowd. Once that is chosen, the rest of the benchmark mostly organizes itself around it.
PickScore/HPS and the arenas make the frozen-versus-live trade-off especially clean. Both start from human preference data. PickScore/HPS freeze that preference signal into a model: cheap, scalable, but static and gameable. Arenas keep the signal live: more current and more aligned with real users, but also slower and noisier.
No method spans the whole space. Deterministic checks are reliable but narrow. Arenas are human-aligned but too coarse to tell you exactly what changed. Real evaluation systems usually have to compose several methods, because no single rater gives you everything.
The offline stack scores quality on selected examples; the online stack - the A/B test and causal inference - tells you whether that quality changes what real users and creators actually do. A/B testing for GenAI is not simple web experimentation, for several reasons.
Heterogeneity in both treatment and effect. On the surface the treatment itself is well-defined (model A vs. model B), but what each user receives is hard to describe as one experience - it depends heavily on the user’s own prompts and workflows, so within-arm variance can be extremely high. And the effect is hardly ever uniform: model B is better on photoreal and worse on anime, better in English and worse in Japanese.
In the presence of high heterogeneity, a single ATE (Average Treatment Effect) routinely hides slices moving in opposite directions - even a clean overall win can mask a loss on your highest-value segment. This is the online twin of the offline slicing problem: we need HTE (Heterogeneous Treatment Effect), not just ATE. And the same bias-variance tension applies (slice finer, lose power) - so pre-register the slices that matter.
The outcome signals are not well-defined either.
First, user behaviors are ambiguous. A click is unambiguously good - generation signals mostly are not. Regeneration can mean dissatisfaction or engaged exploration; download can mean success or an escape hatch to go fix it elsewhere; long dwell can mean delight or confusion; a prompt edit can mean refinement or that the model misunderstood.
Second, the outcome leaks off-platform. The real success - the image gets posted, dropped into a deck, monetized, pulled into Photoshop - happens where you cannot see it, so observable metrics are a censored, biased proxy.
Third, novelty inflates the early read: a new model gets a short-term bump from sheer novelty that decays toward zero, and generative tools are especially prone to this.
Editing is a different measurement regime from generation.
Editing starts with the baseline image, so unlike generation it admits fidelity and preservation metrics: did it change only what was asked and leave the rest intact?
Editing tends to be a trajectory, not a single shot, so the unit of analysis shifts to the session: did the user converge on a result they kept, and in how many steps? Per-edit metrics can mislead - a step that looks worse in isolation may be productive exploration toward a better final image.
The A/B result is itself a proxy. Short-term behavioral metrics stand in for long-term user and creator value, and the gap is real - engagement can rise while value falls. The key question for any proxy is directional accuracy, not correlation magnitude: a proxy that flips sign on your highest-value segment is worse than a noisier one that never does.
The proxy ladder is a universal pattern - offline stands in for online, online for true long-term value - with decades of research behind it (e.g., surrogate index). For example, recommender systems have invested heavily in offline proxies that forecast the online A/B and validating them against historical experiments (e.g., Gilotte et al., 2018). But they can do this because logged feedback lets them estimate a new policy counterfactually (off-policy estimation is its own hard problem!); generation typically has no logged counterfactual to replay, so it leans harder on the offline stack instead. So GenAI evaluation is solving the same problem - with a far larger treatment and effect space, and one hand tied behind its back.
Cheap auto-metrics on every commit (think unit and integration tests) - near-free, high reliability, low validity. Fixed proxies, not rubric-appliers: good for catching serious regressions, nowhere near enough for a training or launch decision.
LLM/MLLM judge for per-iteration and routine sweeps - scalable, but biased (length, position, self-preference) and prone to silent drift. Judge cross-family to blunt self-preference; re-anchor against humans periodically.
Human annotation for bootstrapping (you need it to build the rubric and the calibration set in the first place), periodic calibration, and the critical ship/no-ship gate - trusted, but it doesn’t scale.
Online A/B sits on a different axis - it’s the only thing that measures impact instead of quality, and it’s what the entire offline ladder is trying to predict. Expensive in time and risk, so you earn the right to run it with the offline stack.
The same cost-versus-trust logic settles the other forks: pointwise scoring when you need an absolute bar (launch gates), pairwise when you need a reliable ranking; a frozen benchmark to track regressions cheaply over time, a fresh eval set when the decision must reflect current production; offline to iterate fast on a proxy, online to make the call on the truth.
Evaluation isn't the scoreboard - it decides what game you're playing, which is why it can't be the step you scramble to add after the model. And there's no ground truth in it: the offline metric proxies the online result, the online result proxies real value, each rung lying in its own way. The job was never to produce a number. It's to produce one you know how much to trust.