Releases · youssofal/MTPLX

MTPLX 2.9.2

MTPLX stops rewriting agent transcripts, greedy decoding gets faster below 12k context, and the model forge gets a correctness fix that rescues packs whose draft acceptance had collapsed.

Your transcript is yours (#282)

• The serving endpoints are passthrough by default. MTPLX no longer compacts tool results, trims file reads, or injects steering text into agent transcripts unless you explicitly turn a rewrite feature on. MTPLX_AGENT_REWRITES is the master switch, and each individual feature only arms when you set its own environment variable.
• The macOS app stopped exporting the legacy compaction settings when it launches coding agents, so app-launched Pi and OpenCode sessions get the same clean passthrough as the CLI.
• Managed client configs respect your edits. mtplx start and the app only update files they wrote themselves, and never overwrite a config you have customized.
• The request log records exactly what was and was not rewritten on every request, so you can verify the passthrough yourself.

Faster greedy decode below 12k context

Chained greedy drafting is now on by default for temperature 0 requests with prompts under 12,288 tokens (#313, #315, #318). Gated A/B runs on an M5 Max measured +2.5 to +9.8 percent decode speed across 0.5k to 8k prompts. The same stack measured -2.9 and -2.7 percent at 16k and 32k, so a context fence keeps it off there. Sampled requests are untouched. Move the fence with MTPLX_GREEDY_TRIO_MAX_CONTEXT or disable with MTPLX_GREEDY_DRAFT_CHAIN=off.

Model forge correctness

• The forge decides the MTP norm convention once per tensor set instead of blind-shifting three norm tensors by +1.0 (#301). Packs extracted from absolute-encoded sources no longer ship with draft acceptance collapsed to 0 to 2 percent.
• The runtime refuses to load a double-shifted trunk with a clear error instead of serving a broken drafter (#306).
• quantize: false module overrides are honored during quantization (PR #281 by @shiftedx).

Fixed

• Images survive user-message canonicalization on consecutive or retried turns (#327), and vision rows survive near-prefix cache restores (#296). Both were silent vision-drop bugs.
• The literal-repetition stop covers width 2+ batched MTP cohorts (#311), closing the 8,192-token "!" loop seen on raw CSS.
• The default request log is content-free, as its documentation always claimed (#326).
• The installer and the app write the PATH line through a symlinked ~/.zshrc instead of replacing the symlink with a plain file (#292).
• The dashboard Hardware card reports the real chip. A Mac Studio M2 no longer shows up as an M3 (#329).
• bench --harness depth-sweep honors --depths, --seed, and --generation-mode, and refuses --stock-ar loudly instead of silently ignoring all four (#285).
• Exactness: the fp16 fused add+rmsnorm kernel uses the exact 1024-lane dispatch (#319), and the packed-concats exactness gate tests its element-identity claim honestly (#320).
• The NAX turbo verify path no longer uses padded M=5 lanes that measured slower than stock.
• The flight recorder samples non-streaming requests too.
• The app renders <br> variants inside markdown table cells (PR #273 by @El-Patronum).
• Capture tooling persists exact completion token ids on all three lanes (PR #330 by @CharliePetch).

Experimental, off by default

• MTPLX_FUSE_PROJ: load-time projection fusion for GDN, attention, and MLP (port of PR #316 by @grzracz).
• MTPLX_VK_CROSSROW: crossrow wide-verify kernel.
• Draft-confidence tracing and confidence-gated draft width, groundwork for head calibration.
• Marathon postcommit protection for very long sessions.

Docs

• New: HISTORY.md, the dated record of putting native MTP on Apple Silicon.

Updating

• App: Sparkle offers 2.9.2 automatically, or download the DMG at mtplx.com.
• CLI: pip install -U mtplx or brew upgrade mtplx.

MTPLX 2.9.1

Agent coding sessions run to completion: long-context crash fixes, no hidden output caps, reasoning preserved across turns, and a built-in flight recorder for diagnosing any session.

Engine

• Fixed: agent sessions could truncate and crash near 19,000 tokens (#310). The paged KV cache derived its capacity from a stompable claim instead of the pages it had actually allocated. Long coding sessions now run to the model's full advertised context.
• Fixed: shutdown segfault (#303). The daemon parks its model-owner thread and clears MLX streams at exit, so quit and restart are clean.
• Turbo profile truth. 2.9.0 shipped one turbo fast-path flag that was runtime-dead, so turbo did not apply its full intended configuration. The fast-path environment is now a single shared block, /health reports exactly what the profile set, and a per-lane kernel selfcheck runs at startup. If you benchmarked turbo on 2.9.0, re-run it.
• Multi-turn cache reuse holds at scale. All encode paths now share one tokenization policy, so warm agent turns no longer hit cache walls at assistant reasoning boundaries; tool-call turns bank their just-generated output directly from live KV with no GPU recompute (follow-up turns restore the full prior context at exact length); interrupted background commits retry instead of silently giving up.
• The model no longer re-derives its own reasoning on long sessions. When a client echoes prior reasoning back, MTPLX renders it for turns its committed cache has not yet covered instead of an empty scaffold. In live sessions this ended a failure mode where one marathon turn re-thought a 57,000-token derivation from scratch.
• The model pack owns draft sampling. Stamped draft-sampler settings win over stale client-side pins, so speculative decoding runs the configuration each pack was tuned with.

Agent clients

The app and mtplx start now write identical client configurations for every supported coding agent, and mtplx doctor reports the truth about each lane.

• OpenCode: uncapped generation by default — the managed plugin strips exactly OpenCode's injected 32,000 output ceiling while explicit caps pass through untouched. Reasoning and reasoning-effort selection are honored end to end, prior reasoning round-trips across turns, and each session carries a stable cache identity.
• Pi: the reasoning-effort picker works and maps to the loaded model family's levels. The real output ceiling is advertised (Pi silently applies 16,384 when a model's metadata omits it), and a managed extension strips Pi's generated default cap and adds per-session cache identity.
• Hermes: requests now carry client identity and the configured reasoning effort (agent.reasoning_effort), and the server strips Hermes's injected 65,536 default cap — Hermes cannot express "no cap" on its own. Explicit user caps are honored on every lane.

Flight recorder and mtplx trace

Every request now records a per-second flight log: tokens/sec, context growth, speculative acceptance by depth, verify/draft time split, prefill, and the final outcome — including cancelled and disconnected requests, which previously left no trace.

• GET /v1/mtplx/flight answers "is it hung or thinking?" while a request runs: phase, live tok/s, acceptance, stall age, and the tail of the text being generated.
• mtplx trace turns any coding session into a diagnosis: per-turn timelines joined to your OpenCode history, cache-reuse analysis, automatic pathology flags, repetition autopsies, and a self-contained HTML report per session.
• Recording is local-only JSONL under ~/.mtplx/metrics — a few MB per day of heavy use, capped at 256 MB by rotation. Set MTPLX_FLIGHT_RECORDER=off to disable it.

Updating

• App: Sparkle offers 2.9.1 automatically, or download the DMG at mtplx.com.
• CLI: pip install -U mtplx or brew upgrade mtplx.

MTPLX 2.9.0

Faster decode, smooth streaming at any context length, smaller model packs, and built-in model updates.

Engine

• Decode is 15 to 20% faster than 2.8.3 on typical workloads, combining the decoder work below with the re-quantized draft heads. Code-heavy generations measured up to 60% faster in paired same-machine runs.
• Streamed output is released on token boundaries and grouped into fixed-cadence rounds. Long streams no longer freeze and then dump text: 102 visible freezes per 8k-context session is now 5, and the worst stall dropped from 725 ms to 109 ms.
• Engine CPU while streaming dropped from 26 to 28% down to 18 to 23%.
• The daemon stops rebuilding its full dashboard payload ten times a second when nothing is running. Idle means idle now, and live metrics are unchanged while a request is active.
• The depth tuner warms every candidate before its timed rows. It used to pay model load and kernel compile inside the first measurement, which penalized deeper depths and made it save shallow configs that lost to a static depth 3 (#271).
• If you consume raw SSE and want every token the instant it decodes, set MTPLX_STREAM_COALESCE=0.

Models

• Every mtplx pull now records which revision it downloaded and pins the download to that exact commit. A pack on disk can no longer silently differ from what is published.
• New: mtplx models --check shows the update state of every cached pack. mtplx models --update <repo> syncs one, downloading only the changed files: typically 240 to 450 MB instead of a 15 to 21 GB re-pull.
• Works for packs downloaded by older MTPLX versions too. When the check cannot prove your copy is current, it says unknown instead of guessing.
• All six Qwen 3.8 packs re-shipped with quantized speculative-decoding draft heads: 4-bit heads on 4-bit trunks, 8-bit on the 8-bit trunk, FP16 variants for M1/M2 contain no BF16 anywhere. Trunk weights are unchanged, so upgrading rides the delta updater.
• Same answers, verified: every pack ran a paired multi-seed acceptance battery against its previous head and shipped only flat-or-better at every speculation depth. The Quality FP16 head reproduces its source head token for token on every gated run.

App

• The model picker shows when a cached pack has an update, with its download size. One click updates it. If the model you are running was updated, you get a restart prompt.
• Rendering a live stream now costs the same whether the message is 10 lines or 10,000. CPU with Settings open during a stream dropped from 82 to 97% down to 30 to 37%.
• Long code streams flow inside the message with no nested scrollbars and no sideways travel. Code wraps, grows live with its content, and hands off cleanly to the final syntax-highlighted view. Verified on a 42,000-token generation.
• Streamed Markdown stays visually stable. Prose no longer reflows or flashes as lines arrive, and code keeps its colors while it grows.
• Reasoning streams as append-only text in the order the model produced it. No more briefly duplicated or reshuffled thoughts.
• Forged models are first-class: your own Qwen 3.8 builds report the right capabilities (vision included), Use Now puts them straight into the model picker, Forge profile metadata is honored, and verification is tied to the exact artifact you built.

Fixes

• Switching models while the daemon was degraded could hang forever. It restarts cleanly now.
• Fans no longer stay pinned at max after a failed startup.
• Pressing Return to send no longer drops the first characters of fast typing. Shift+Return inserts a newline.
• The status dot now says why the daemon is degraded instead of only changing color.
• Catalog download sizes were re-audited against the live repos and corrected.

Updating

• App: Sparkle offers 2.9.0 automatically, or download the DMG at mtplx.com.
• CLI: pip install -U mtplx or brew upgrade mtplx.
• After updating, run mtplx models --check to pick up the smaller packs.

MTPLX 2.8.3

Released 2026-08-18. Follows 2.8.2.

Why this patch

2.8.2 fixed the start wizard's re-tune loop, but field reports of slow,
stuttering chats kept coming — and they were right. Two more regressions
shipped in 2.8.0 and survived until now because both live on a code path
release QA never exercised: the uncapped request, which is what every
desktop, web, and agent chat actually sends. Benchmarks and gates always
cap max_tokens, and capped requests take a different path.

The streaming freeze

2.8.0 added a wire safeguard for the runaway-loop trimmer: while an
uncapped request is armed, the stream held a fixed ~448-token tail so a
trim could never chase bytes already delivered. At chat speeds that
silenced the wire from roughly token 320 to token 768 on every
uncapped response: reasoning froze for 6–11 seconds while the speed
readout collapsed, then the backlog arrived in one burst — and the last
~448 tokens of every response arrived the same way. The engine was
decoding normally the whole time, which is why every server-side number
looked healthy.

The holdback is now engaged only while the output actually shows a
forming loop. Healthy responses stream live, byte for byte, exactly like
2.7.1. A genuine runaway loop still gets trimmed with at most a short
prefix of the repeated run ever reaching the wire — still strictly
better than 2.7.1, which streamed all of it.
MTPLX_REPETITION_STREAM_HOLDBACK=candidate|strict|off selects the new
default, the 2.8.0–2.8.2 behavior, or the pre-2.8 wire.

The boot burn

2.8.0's turbo profile walked a background warm ladder up to 32,768-token
prefills after every boot so deep-context benchmark rows would start
warm. On real machines that meant 30–60+ seconds of full-throttle GPU
after every launch (2.7.1: about 4.5 s), warm rungs re-firing between
chat turns, and a chat sent mid-rung waiting several seconds for its
first token.

The product ladder is back to the two rungs interactive chat actually
reaches early. Benchmark harnesses that want deeper buckets pre-warmed
opt in with MTPLX_WARMUP_LADDER. Background warm steps now also wait
for 90 seconds of request quiet (MTPLX_WARMUP_IDLE_GRACE_S) before
touching the model, so warming never competes with a conversation.

The app was the other half

Founder testing at temperature 1.0 on a long, multi-turn conversation
caught what a fresh chat never showed: the desktop app itself rendered
in O(entire transcript) per frame. The window's SwiftUI hosting view
re-measured every realized message on every constraint invalidation
(a third of the main thread at idle, up to 62×/s while streaming — the
guard written to prevent exactly this had silently never applied), the
markdown pipeline re-counted every character of every block per frame,
and per-delta paths copied the whole answer to ask if it was empty.
When those walks stalled a frame, the typewriter's catch-up path pasted
the whole backlog at once — the literal freeze-then-vomit. The engine
was streaming cleanly the entire time; replaying the wire gate over the
founder's exact 37k tokens showed zero holdback engagements.

2.8.3's app build fixes all of it: the transcript min-size walk is
dead, fence classification and syntax-lex state are computed once per
block instead of once per frame, scroll pacing no longer invalidates
the view tree, catch-up is rate-limited to read as fast typing (never a
paste), the 10 Hz metrics chip no longer re-evaluates every bubble or
burns a core parsing its stream byte-by-byte, and markdown tables no
longer draw rows on top of each other (cells measured single-line but
drew wrapped).

QA changes

The release pillar gate now includes an uncapped streamed chat measured
for delivered cadence — any gap over 2 seconds fails the release. The
capped-only blind spot that let both regressions through is closed.

App-side, the bar moved too: streaming QA runs on a heavy multi-turn
conversation (fresh chats hide every O(transcript) render term), and
"smooth" is judged at three layers — engine wire, app ingest telemetry,
and the actual on-screen text — because the first two were green while
the third was frozen.

Measured on an M5 Max, 27B Bare-Speed, uncapped chat

Round two: what the founder's retest caught

The first 2.8.3 candidate fixed the measured pathologies and then
failed a human in three ways the instruments weren't pointed at. This
round is those three, fixed and re-measured:

• Blank transcript mid-generation — the candidate's lazy transcript
  culled every visible row under the app's own scroll driver. Eager
  again; streamed tables (wrapping cells included) verified in pixels
  while arriving.
• Thinking well rewriting itself — reasoning now renders as plain
  text, and the live ticker's window anchors at line breaks so a
  rendered thought line never changes after you've read it.
• Freeze-then-burst cadence — server-side: whitespace-free runs
  flush incrementally, token emission precedes cache housekeeping
  barriers, and the auth middleware no longer buffers stream frames.
  Same-prompt A/B: ~30 sub-second delivery silences per answer → single
  digits; generator gaps over 200 ms → zero; every request now records
  a producer gap census so this regression class can never ship silent
  again.

Round three: the stutter that only humans could see

The founder retested with his own hands and the stutter was still
there — while every hands-off verification pass stayed clean. Both
observations were correct, and the difference between them was the
bug:

• Interaction starved the UI's layout guard. The guard that stops
  the window from re-measuring the entire conversation ran in a
  run-loop phase macOS skips while input events keep arriving. Touch
  the mouse and the guard stopped running; every layout pass walked
  the whole transcript and screen updates coalesced into visible
  freezes. Phase-aligned A/B with synthesized human input on the same
  build and prompt: hands-off was clean on both binaries; 40 s of
  wheel-scrolling went from 70 stalls / 18.7 s frozen / worst 1.31 s
  to one 197 ms stall, and 30 s of cursor movement went from
  91 stalls / 26.7 s frozen to zero. The guard now runs every
  run-loop turn, input storms included.
• Auto-follow fought the user. Scrolling up mid-generation raced a
  synchronous bottom-pin whose user-scroll signal was set
  asynchronously; momentum and classic wheel mice weren't covered at
  all. The pin now yields to any user scroll instantly and re-engages
  only at the bottom.
• Cancelled runs were unmeasured. The founder's stutter report came
  from a run he cancelled — and cancelled requests logged no stream
  census at all. They now log the full producer gap census and sliding
  windows for the streamed portion.

About multi-turn TPS

Follow-up turns in a conversation decode slower than the first — the
founder measured 43 → 40 → 33 tok/s across three turns at under 10k
context — and this release deliberately does not paper over it. Two
real mechanisms, both now precisely attributed in every request
record: each verify cycle costs more as context grows (~60 ms/cycle at
1k context → ~77 ms at 8k, honest attention physics plus a
context-linear draft cost), and speculative acceptance tracks content
entropy — at temperature 1.0 it collapses in free-form prose (down to
~0.4 by depth 1 in wrap-up passages) so each cycle commits fewer
tokens. Neither is a regression: the same curves are measurable in
every 2.8.x build. The per-cycle cost work is the standing decay
track, continuing in 2.8.4; acceptance-vs-entropy is the 3.8 MTP head
calibration campaign. What 2.8.3 ships is the instrumentation that
makes both visible per-request instead of anecdotal.

Why a second patch today

Within hours of 2.8.1, two independent reports showed the same failure: load a model, and with no requests at all the machine heats up while API requests never arrive (#280, and a report on X). If that happened to you, 2.8.1's start wizard was silently re-running the multi-minute tuning benchmark on every launch — with the API port still closed while it ran. 2.8.2 fixes that loop and everything found underneath it. If you updated to 2.8.0/2.8.1 today: update again, it takes one minute, and your saved tune results now stick.

Fixed

• mtplx start no longer re-offers tuning on every launch. The tune record's lookup key and its save key had drifted apart in 2.8.0 (the per-model profile work updated one side only), so the wizard could never find the record it had just saved. Accepting the re-offer meant minutes of maxed GPU with the server not yet listening — the reported "heats up + API unreachable" pair. One shared key constructor now, with a regression test that fails if the two sides ever drift again.
• The wizard tunes the model you actually picked. Picking a non-default model tuned the machine's default instead (picking the 4B tuned the 27B) and saved a record the launched model could never use.
• A picked local model is never silently swapped for the default. LM Studio-style folders whose name matches the verified default were re-routed to the Hugging Face repo id, producing "Model is missing. Download?" for a model already on disk (#279).
• Idle daemons no longer hammer session-bank/manifest.sqlite. Every health/dashboard poll opened the SSD-cache manifest and ran a full-table aggregate (~8 sqlite opens per poll, continuously visible in Activity Monitor, #280). Steady-state polling now costs at most one manifest read per 5 seconds.
• mtplx --version reports the right version again. The published 2.8.1 wheel identified itself as 2.8.0 (2.8.1); 2.8.2 prints 2.8.2 (2.8.2) — worth checking after you update.

Nothing on the generation path changed: decode, prefill, sampling, tool calling, and the session cache behave exactly as in 2.8.1 (verified flat-or-better in interleaved A/B before ship).

MTPLX 2.8.1

Released 2026-08-17. Hotfix on top of 2.8.0, published the
same morning.

Why a patch an hour after 2.8.0

2.8.0's headline work made long agent sessions commit their cache
frontier on every turn, including turns that carry images. Our release
pipeline runs a correctness sentinel for exactly that surface: send a
transcript with one image, then repeat the identical transcript with a
different image, and require that the second request never reads cached
state from past the image position. That sentinel failed on the desktop
build gate.

The cause: the session frontier is keyed by token ids, and every image
placeholder shares a single id, so two different images look identical
to it. Once vision histories started committing, a repeated transcript
with different pixels could restore the previous image's KV wholesale
and answer about the wrong image.

The published PyPI wheel for 2.8.0 carried this defect for about an
hour. The desktop DMG never shipped with it; the gate blocked it.

What changed

Vision conversations keep their full prompt-cache behavior through the
content-keyed store introduced in 2.8.0: it derives cache keys from the
actual image bytes, so identical pixels restore the whole prefix and
different pixels can never match past the first image token. What
changed is the raw-id session frontier: image-bearing histories no
longer advance it, on any of its four write paths. Same speed for the
honest case, hard stop for the aliasing one.

Five new regression tests pin the behavior, and the release gate that
caught it now runs against a hermetic cache directory so a previous
run's state can never mask or fake a result.

Upgrading

• PyPI: pip install -U mtplx (2.8.1)
• Homebrew: brew upgrade mtplx
• Desktop app: 2.8.1 (build 2008001) via Sparkle or the website DMG

Nothing else changed from 2.8.0; its release notes remain
the reference for what is new.

Known cosmetic quirk in this wheel

The mtplx --version banner in the 2.8.1 wheel reads mtplx 2.8.0 (2.8.1): the display constant missed the hotfix bump. The package
version, pip show, and all behavior are correct, the desktop build
displays 2.8.1, and the banner is fixed on main. Not worth a third
release in one morning.

Read more

MTPLX 2.7.2

An emergency fix for mtplx pull. On 2.7.1 and older, re-pulling a model
that changed upstream — such as the Qwen 3.8 repos, re-published on
2026-08-15 with their vision towers restored — can corrupt your local copy.
Upgrade before you pull.

Fixes

• mtplx pull no longer corrupts files that changed upstream (#258,
  #234). The downloader treated a complete local file whose size no
  longer matched the server as an interrupted download, and byte-range
  appended the remote tail onto the old content. Updating a repo in place
  corrupted config.json and model.safetensors.index.json and left the
  model unloadable. Stale files are now discarded and re-fetched whole;
  genuinely interrupted *.incomplete downloads still resume. (Only pulls
  with progress reporting — the interactive CLI, --progress-json, and the
  app — had the bug; mtplx pull --json routes through hf_hub's etag
  downloader and was never affected.)

  Already hit by this? A file the old downloader corrupted ends up at
  exactly the size the server reports, so mtplx pull on 2.7.2 still sees
  it as complete and cannot repair it on its own. Delete the affected
  model's config.json and model.safetensors.index.json from its folder
  under ~/.mtplx/models/, then run mtplx pull again on 2.7.2.

• The Qwen 3.8 models can see again (#263). All six published 3.8 repos
  (Bare Speed, Optimized Speed, Optimized Quality, and their FP16 siblings)
  shipped without their vision towers: the forge convert lane kept only the
  text model. The repos were re-published on 2026-08-15 with the official
  bf16 tower grafted back in as an index-registered
  model-vision.safetensors; language and MTP tensors are untouched, so an
  existing install picks the repair up as a ~0.9 GB delta. mtplx forge build now grafts the vision tower, vision_config, and preprocessor
  sidecars on every lane and fails closed rather than producing a blind
  artifact (a repair script for already-forged artifacts ships as
  scripts/graft_vision_tower.py). Catalog and app download sizes now
  include the tower.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on. (Unchanged
  from 2.7.1.)

Upgrading

• Upgrade before pulling the repaired Qwen 3.8 repos.
• CLI: pip install -U mtplx or brew upgrade mtplx.
• App: Sparkle will offer 2.7.2 (build 2007002), or grab the DMG.
• If a pull on 2.7.1 already corrupted a model, see the first fix above for
  the two files to delete before pulling again.

MTPLX 2.7.1

A bug-fix release. It clears the known-issues list 2.7.0 shipped with.

Fixes

• xhigh stays selected. Picking it in Inference settings while the model
  was running snapped straight back to medium, and
  mtplx config set reasoning_effort xhigh was refused outright. Those two
  places each carried their own copy of the effort list and neither had been
  told about xhigh, even though the engine had understood it since 2.7.0.
  The live-settings save is all-or-nothing, so one unrecognized level threw
  away the entire save and the picker reverted. Every place that accepts an
  effort level now reads the same list, and which levels a given model offers
  is still decided by that model alone.
• KV cache quantization reaches Qwen 3.8. The toggle displayed q8 but
  the launch path only recognized Qwen 3.5 and 3.6, so a 3.8 run quietly got
  nothing.
• mtplx doctor names the model it actually checked rather than the old
  default, and turbo's profile note reports the real compiled-verify fence of
  32,768 instead of the pre-2.7.0 number.
• A new build can't offer you an older one. Updates are ordered by build
  number, and the one derived for 2.7.1 came out below the 2.7.0 already in
  the wild — so a fresh 2.7.1 proposed 2.7.0 to itself. Fixed at the
  derivation, and the update feed now reads its number off the built app.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on.

Upgrading

• App: Sparkle will offer 2.7.1 (build 2007001), or grab the DMG.
• CLI: pip install -U mtplx or brew upgrade mtplx.

Read more