Score
Harvey LAB-AA: Criterion Pass Rate
Share of atomic pass/fail rubric criteria the deliverables satisfy, graded by a single LLM judge · Independently benchmarked by Artificial Analysis
Cost
Harvey LAB-AA: Cost per Task
Average cost per task (USD), broken down by input, cache hit, cache write, reasoning, and answer tokens
Average cost per task in the evaluation. Costs are split by input, cache hit, cache write, reasoning, and answer token pricing where canonical token counts are available.
Token Usage
Harvey LAB-AA: Output Tokens per Task
Output tokens used to run one task, broken down by reasoning and answer tokens
The average number of answer and reasoning tokens produced per benchmark task in this evaluation.
Speed
Harvey LAB-AA: Time per Task
Weighted average decode time (minutes) per task; excludes TTFT and overhead time · Lower is better
The weighted average time (seconds) per evaluation task. This is calculated by dividing output tokens per task by output speed, weighted by the relative weights of each benchmark in the evaluation.
Turns
Harvey LAB-AA Benchmark Leaderboard: Average Turns per Task
Average number of model turns per Harvey LAB-AA task · Lower is better
This chart shows the average number of turns the agent takes per task. It is a rough proxy for how many actions, tool calls, and iteration cycles an agent is using to complete benchmark tasks.
Score vs. Release Date
Harvey LAB-AA: Criterion Pass Rate vs. Release Date
Browse representative Harvey LAB tasks from the public task set, the reference files each model was given, and the deliverables it produced.
Mergers & Acquisitions
Instructions
Review the attached acquisition data room contracts and internal memo for change of control and assignment provisions, and prepare a comprehensive deal team report.
Output: coc-analysis-report.docx
Deliverables
Expected outputs the model must produce
coc-analysis-report.docxA comprehensive deal team report analyzing change of control and assignment provisions across the target’s material contracts.
Reference files
Provided to the model
Model submissions
Deliverables produced by each model
Claude Fable 5 (with fallback) - coc-analysis-report.docx