Who Decides When theTask Is Done?: Measuring the Effect of Moving Completion Authority from LLMs to Checkers

An LLM agent finishes a task in one of two ways: either the model judges its own

work complete, which we call advisory feedback, or a checker computes completion

and the model’s claim carries no authority, which we call binding feedback. The

distinction matters because an LLM is a generator rather than a verifier. When the

specification is incomplete, it fills the gap from its priors, grades its work against

that reconstructed answer key, and can confidently declare wrong work finished. We

call this failure the false-DONE: the model does not know when it does not know.

Across three preregistered experiments on frozen buggy-Python repair corpora, 2,862

episodes over seven models from five providers at under one dollar of total compute,

we isolate this authority placement as the only varying factor. Binding lifts a weak

model by 9.2 points while leaving a strong one unchanged. Across a seven-model

capability ladder the gain is a window, peaking mid-ladder at +14.9 points, zero

at the ceiling, and negative for the weakest model, which fails by thrashing rather

than false claiming. Raising task difficulty by composing two and three simultaneous

bugs roughly doubles the window’s depth, upto +31.7 points, but never moves its

ceiling : the strongest model produces zero false claims at every difficulty. The window

is therefore a property of the model, not the model–task pair, and the advisory

false-DONE rate mediates the entire effect, which makes the intervention predictable:

a short probe that counts false claims tells a designer in advance whether completion

gating will pay.

who_decides_when_the_task_is_done.pdf

Files (940.2 kB)