MacBook SVG Benchmark: 12 AI Models | Playcode Blog

NEW2026-07-24: Added Claude Opus 5 (Anthropic) at high/xhigh/max, both tasks - the most detailed Anthropic laptop here, but the geometry is broken at every level (skewed at high, broken isometry at xhigh, no laptop at all at max), and thinking-on-by-default makes it 10-25x pricier per drawing than Opus 4.8.

For two years the best quick test of a new model was Simon Willison's pelican: ask for a pelican riding a bicycle as SVG and look at the result. It worked because it could not be faked. It no longer separates anyone - so we built the test we actually wanted: draw a MacBook Pro 16 in SVG, in a dark colour, one shot. Everyone reading this knows exactly what a MacBook looks like, so everyone reading this can grade it. Below is every run we have: 12 models, both tasks, the full effort ladder, 64 runs, unedited.

Why the pelican had to go #

A benchmark is useful exactly as long as it separates models. We ran the pelican as a control through the identical harness - same models, same effort ladder, same one-shot rule - and every single model produced a pelican, on a bicycle, first try. When every student gets an A, the exam has stopped grading. Two problems got it there:

• Everyone has trained on it. It is the most famous drawing prompt in AI. Three years of pelicans and public commentary about them sit in every training corpus. A good pelican no longer tells you whether the model understands geometry or has simply seen ten thousand graded pelicans.
• It never had a gradation. Nobody knows how long a pelican's beak should be relative to its wingspan, so grading tops out at "recognisable". There is no scale on which one passing pelican beats another.

A MacBook Pro fixes both. Hundreds of millions of people have stared at one for thousands of hours, so the lid, the keyboard grid, the notch, the hinge and the trackpad have a precise shared ground truth - your eye flags a small error instantly. And it has a real difficulty ladder: dozens of parts plus a 3D perspective view. Models can now fail at many different heights instead of clustering at "recognisable". The numbers say the same thing: the models spend roughly 4x fewer output tokens on a pelican than on the MacBook (Opus 4.6: 2,091 vs 8,156), and cranking effort barely moves the pelican at all.

How we ran it #

Create an SVG 3D model of a MacBook Pro 16 in a dark color. Respond with ONLY the SVG markup, no explanation.

One API call per cell. No retries, no system prompt, no examples, no cherry-picking - the first answer is the answer. Each model runs the effort ladder its provider supports (high, xhigh, max); Grok 4.5 tops out at high, Gemini 3.6 Flash exposes low/medium instead of xhigh, and Kimi K3 runs low/high/max with no xhigh. Costs are the providers' own usage-reported billing. Runs marked * were requested as xhigh and clamped to high by the model's generation.

On the images. Each drawing is a browser render of the model's unedited SVG - nothing touched up, and the raw SVGs stay in the repo. We rasterise deliberately: the models all reach for the same generic ids (#screenGrad, #screenClip, #shadow), so inlining fifty of them into one page makes the browser resolve one model's clip-path into another model's drawing. Rendering each SVG in its own document is the only way to show you what the model actually drew.

On reproducibility. We re-ran the pelican ladder nine days later on a different output budget; token counts landed within about 10% of the originals (Fable at high: 2,264 then 2,002). Each run below records the sweep it came from, so mixed dates stay visible rather than quietly blended.

Every run #

The whole matrix, both tasks. Filter to one task or vendor, sort by cost to line them up cheapest-to-priciest, or show only the newest model.

TaskVendorSortNew models only64 of 64 runs

Model by model #

Each model with both of its drawings side by side - the pelican it aces and the MacBook that actually grades it. Newest first.

What the effort dial buys #

The effort ladder was supposed to be the boring part. It produced the sharpest result of the benchmark, and it took two sweeps to state it correctly.

Our first sweep capped output at 16,000 tokens, and at max four models never finished. That looked like models falling off a cliff. So we re-ran the whole max column with a generous 64K budget - streaming, which Anthropic's SDK requires once a request could run past ten minutes. The honest picture is more interesting than the first one:

• Up to xhigh, effort buys real quality on the strongest models. Fable's and Sol's best drawings both happen at xhigh, and the improvement over high is visible detail. On mid models it buys nothing: Opus 4.8's three drawings cost $0.05, $0.10 and $0.18 and look identical.
• Most of the truncations were our cap, not the model. Given 64K, Fable 5 finishes - and draws the best MacBook in the benchmark. Terra finishes too. Gemini 3.6 Flash finishes at max for $0.16.
• But finishing is not getting it right. Terra completes a full drawing at max and the form is still wrong - a wide, thin generic ultrabook. All the budget in the world does not buy an understanding of the object.
• Once in a while, max genuinely pays off. Kimi K3 is the clean case: its 30,579-token, 17.5-minute max MacBook is materially better than its own high one - correct 16-inch proportions, speaker grilles, no visible errors - and lands in Fable's quality tier for a fifth of the price. It is the exception that shows how rarely the top setting is worth its bill.
• And some runs genuinely never end. Sonnet 5 - and now Opus 5 - burned an entire 64K budget without closing the SVG. Sol timed out after 36 minutes with nothing to show, twice. That is not a budget you can raise your way out of.

So max is mostly a trap. It reliably buys you a bigger bill and a longer wait; it occasionally buys a better drawing, and it sometimes buys nothing at all. If you run an effort dial in production, measure it on your own workload before you let users pay for the top setting.

What one drawing costs #

Because every cell is a real API call, we can price the same task across the market. This is every MacBook run, cheapest first - including the ones that billed and handed back no finished drawing.

The best drawing costs about 495x the cheapest passable one, and the cheapest entries are not embarrassing. Note the greyed rows: you can pay full price and receive nothing. They billed $2.26 for files that were never closed - and Sol's max is not even counted there, because it never returned to bill. Whether Fable's detail is worth 13x Gemini 3.6's coherence depends entirely on what you are building - but you cannot even ask that question from a $/Mtok pricing page. (Why the sticker price misleads across vendors is its own story: The Same TypeScript Costs 73% More on Claude Than on GPT.)

The bug this found #

The first full run crashed two cells with an API error we had never seen in production: This model does not support effort level 'xhigh'. Anthropic's xhigh exists only on the newest generation (Sonnet 5, Opus 4.8, Fable 5); Opus 4.6 and Sonnet 4.6 reject it, and their supported list is not even a superset-consistent scale across generations. Our product maps a user-facing quality dial onto these levels, which means one combination of settings produced a hard error on every message. The benchmark surfaced it in ninety seconds and we shipped the fix the same night. A static leaderboard would never have caught it.

The second sweep found a second one, in our own harness: Gemini's effort dial had been silently doing nothing. Gemini exposes thinking_level rather than a reasoning-effort parameter, so every Gemini run in the launch sweep was quietly at the default. Wiring it up is why Gemini 3 Flash finally has an effort ladder in the matrix above - and why Gemini 3.6 Flash could be measured properly at all.

Try it yourself #

The method is deliberately reproducible. The prompt, ready to paste into any model:

Create an SVG 3D model of a MacBook Pro 16 in a dark color. Respond with ONLY the SVG markup, no explanation.

Or swap in any object your audience knows intimately - a Coke can, a Vespa, a Stratocaster. The only requirements are that the ground truth lives in your reader's head and that style cannot hide broken geometry. We re-run this gallery when new frontier models ship; the pelican had a good run, and the MacBook has years of discrimination left in it.

Method notes: runs on 2026-07-13/14, 2026-07-21/22 and 2026-07-24 via each provider's public API, sampling left at provider defaults, reasoning effort set through each provider's native parameter (Anthropic output_config.effort, OpenAI reasoning.effort, xAI reasoning_effort, Google thinking_level, Moonshot reasoning_effort). Output capped at 16,000 tokens in the first sweep and 64K thereafter; the cap is stated wherever it changed a result. Exact model ids: claude-fable-5, claude-opus-5, claude-opus-4-8, claude-opus-4-6, claude-sonnet-5, claude-sonnet-4-6, gpt-5.6-sol, gpt-5.6-terra, grok-4.5, gemini-3-flash-preview, gemini-3.6-flash and kimi-k3. Costs come from the providers' own usage-reported token counts at list prices; total measured spend across every run shown here is $13.80. Drawings are unedited; failures are reported as failures rather than re-rolled. The measurements are ours; the prose was drafted with AI assistance and edited by a human.

Playcode keeps all of these models one click apart, with the same effort dial we benchmarked here - so you can run your own MacBook test on a real project instead of taking our word for it. Try it at playcode.io.