1040Bench — Verona

core57-task-v2.md

You are preparing the 57 whole-dollar monetary entry cells on a
taxpayer's 2025 U.S. Individual Income Tax Return.

Your evidence environment is two read-only virtual file systems (VFS): a VFS
here is a paged document store you traverse with tools, not a mounted disk.
Each VFS holds complete documents split into ordered, immutable pages; every
page has machine-OCR text and an authoritative JPEG page image you can view.

The Binder VFS holds the client package - exactly two documents:
- PYR-S001, the taxpayer's complete prior-year (TY2024) Form 1040 return;
  its page IDs are PYR-S001-P0001, PYR-S001-P0002, and so on; and
- CYS-S001, the complete current-year (TY2025) supporting-document packet;
  its page IDs are CYS-S001-P0001, CYS-S001-P0002, and so on.
Page labels are neutral position markers ("Current-year supporting documents
page 12"), not content descriptions. Page text is machine OCR and can misread
digits, signs, and checkboxes; view the page image whenever visual detail
could change a conclusion.

The Law Pack VFS holds the pinned official IRS authorities as two sealed
annual releases, TY2025 and TY2024, selected by tax_year. Law Pack page IDs
have the exact form TY2025-S###-P#### or TY2024-S###-P#### (for example
TY2025-S003-P0041).

The client package is complete and self-contained: it consists exactly of
every CYS-S001 page and every PYR-S001 page. Do not posit an unprovided
document, transaction, account, payment, conversation, or exception merely to
manufacture uncertainty. Never use a prepared current-year return, filed
value, prior adjudication, benchmark answer, or anything outside these two
VFS namespaces.

Binder pages establish client facts. Law Pack pages establish legal context;
they never establish a client fact. Keep their references separate.

Model-visible tools:
- binder_list_documents, binder_document_overview,
  binder_search_document_text, binder_read_document_text, and
  binder_view_document_pages traverse only PYR-S001 and CYS-S001.
- law_pack_list_sources, law_pack_source_overview, law_pack_search_text,
  law_pack_read_source_text, and law_pack_view_source_pages traverse only the
  pinned TY2025/TY2024 Law Pack pair.
- execute_code runs Python in a persistent, network-disabled, secret-free
  workspace. Neither VFS is mounted; pass only facts already retrieved.
- update_notepad maintains durable run-local working notes. It supports
  replace_all, append, and apply_edits (ordered exact replacements), each
  fenced by the notepad's current revision, so update incrementally rather
  than rewriting. The complete notepad is injected on every turn as untrusted
  working memory and is capped at 60,000 characters.
- submit_core57 is the terminal tool and must be called alone.

Record evidence as you work. submit_core57 requires, for every one of the 57
fields, the exact Binder page IDs supporting the decision where available,
plus any Law Pack page IDs actually relied on. Maintain the notepad's
"57-Field Decision Ledger" section as you resolve each field: record the
resolution and value together with those exact page IDs, copied from tool
results. Older tool results may later be compacted out of your context; the
notepad is re-injected every turn and is your durable record, so keep the
ledger current enough to submit from directly.

The package is the entire universe of client documents. Apply package closure
only after inventorying both documents and running targeted searches:
- Absence: when nothing anywhere in the package - no document, no
  transaction, no organizer entry, no profile fact, no cross-source trigger -
  indicates an ordinary document-driven category, that category is
  inapplicable and its optional Form 1040 cell is a computed blank. A blank
  organizer cell alone is neutral; it is not a "no" answer.
- Presence: once any visible transaction triggers a category, closure no
  longer fills gaps. Do not infer or invent a missing basis, ownership,
  qualified-use fact, eligibility fact, payment actuality, election, or other
  outcome-changing treatment fact; if such a fact is genuinely missing and
  different lawful values result, the field is insufficient_information.

From PYR-S001, carry forward only:
- stable profile facts that professional software ordinarily continues when
  current-year evidence shows no change; and
- deterministic statutory carryforwards the prior return itself creates (for
  example, a capital-loss carryover computed by the standard worksheet).
Never carry forward annual amounts, expenses, payments, qualified use,
transaction treatment, discretionary elections, or annual eligibility
conditions. Current-year evidence always controls.

For every field, submit exactly one decision:
- computed with a JavaScript-safe whole-dollar integer when the complete
  package and applicable law determine a presented amount;
- computed with JSON null only when evidence and the presentation rule
  establish a computed blank; or
- insufficient_information when a genuine missing client fact or unresolved
  client-controlled treatment permits different lawful values for that field.

Integer zero, computed blank, and insufficient_information are distinct. Do
not use a tolerance. Different acceptable rounding methods may produce
different whole-dollar results, but they must use one identical fact-and-tax-
treatment ledger. Compute one conventionally valid result; never use
insufficient_information merely because rounding could change the answer.

Each field record must include Binder evidence page IDs where available, any
Law Pack page IDs actually relied on, and a concise reason only for
insufficient_information. Evidence references must resolve in the Binder VFS;
Law references must resolve in the Law Pack VFS.
List each page at most once per field.

The exact ordered field roster is:
line_1a, line_1b, line_1c, line_1d, line_1e, line_1f, line_1g, line_1h, line_1i, line_1z, line_2a, line_2b, line_3a, line_3b, line_4a, line_4b, line_5a, line_5b, line_6a, line_6b, line_7a, line_8, line_9, line_10, line_11a, line_11b, line_12e, line_13a, line_13b, line_14, line_15, line_16, line_17, line_18, line_19, line_20, line_21, line_22, line_23, line_24, line_25a, line_25b, line_25c, line_25d, line_26, line_27a, line_28, line_29, line_30, line_31, line_32, line_33, line_34, line_35a, line_36, line_37, line_38

Perform source review, legal retrieval, calculations, reconciliation, and
worksheet work with the tools. When every field is resolved, call
submit_core57 by itself with the complete ordered vector. If submit_core57 rejects
the submission, correct every returned validation diagnostic and call
submit_core57 again by itself. An accepted submission ends the task.

submit_core57 accepted one object containing one decision for each of the 57 core fields of Form 1040. An accepted call ended the task.

{
  "fields": [
    {
      "field_id": "line_1a",
      "resolution": "computed",
      "value": 1234,
      "evidence": [{ "page_id": "CYS-S001-P0001" }],
      "law_refs": [{ "page_id": "TY2025-S001-P0001" }]
    },
    {
      "field_id": "line_1b",
      "resolution": "insufficient_information",
      "reason": "The client record does not establish the required fact.",
      "evidence": [],
      "law_refs": []
    }
  ]
}

The benchmark’s highest score came from a methodical run: repeated document and Law Pack reads, three Python calls, two notepad updates, and one valid submission.

Wall time

71m 48s

Tool calls

113

Input tokens

27.4M

Output tokens

392K

Est. list price

$15.9

Seven models got none of the task’s 26 insufficient-information decisions right. GPT got 19 and won at 39/57, even though six peers computed more supported fields correctly.

Wall time

20m 34s

Tool calls

114

Input tokens

11.7M

Output tokens

76K

Est. list price

$23.8

Opus reached 46/57 in just 31 tool calls, tying Gemini and Grok for second on the task, one field behind Muse and at least 19 fields ahead of every other model. It used 35% fewer calls than Gemini, 44% fewer than Grok, and 66% fewer than Muse.

Wall time

14m 4s

Tool calls

31

Input tokens

2.6M

Output tokens

65K

Est. list price

$14.9

Gemini tied Opus and Grok for second on the task at 46/57, one field behind Muse and at least 19 fields ahead of every other model. It finished in 3 minutes 31 seconds; across the full benchmark, it averaged 3.6× faster than Opus and 8.9× faster than Grok.

Wall time

3m 31s

Tool calls

48

Input tokens

8.2M

Output tokens

40K

Est. list price

$1.32

Grok made 386 successful tool calls, more than four times its next-longest trace. It spent 25 turns in a 51-call page inspection loop, yet scored 24/57 and placed sixth on the task.

Wall time

173m 19s

Tool calls

386

Input tokens

16.1M

Output tokens

483K

Est. list price

$25.5