Port Bun.stringWidth to C++ with explicit SIMD by robobun · Pull Request #31351 · oven-sh/bun

16 min read Original article ↗
Adds support for compiling and linking Bun for macOS (arm64 **and** x64)
on a Linux host, and switches CI over to it: the darwin build lanes now
run on the Linux fleet and **the macOS binaries that ship from the
release step are cross-compiled** — the mac fleet only runs tests.

Requested in Slack: get cross compilation for macOS working locally
first, then in CI.

> [!NOTE]
> `WEBKIT_VERSION` now points at a regular `autobuild-<sha>` release
from oven-sh/WebKit main (the macOS-cross + Windows-cross WebKit work
has merged there). The macOS `-lto` prebuilts are ThinLTO bitcode; the
Linux `-lto` prebuilts are full-LTO bitcode — matching how each
platform's final link runs (see the LTO section below).

### How it works

Same model as the existing Android/FreeBSD cross targets — host clang +
`--target`/sysroot — extended to Mach-O:

- **SDK**: `resolveMacosSdkPath()` (new `scripts/build/macos-sdk.ts`)
finds a macOS SDK via `--macos-sdk` / `$MACOS_SDK_PATH`, a well-known
install (`/opt/MacOSX*.sdk`, osxcross tree), or fetches the pinned
`MacOSX26.5.sdk` into the build cache at configure time via `xmac`
(vendored `scripts/build/xmac.mjs`), which downloads the Command Line
Tools package straight from Apple's public software-update CDN and
extracts the SDK (~59 MB, same model as xwin uses for the Windows SDK —
nothing is redistributed, Apple's license notice is printed before
`--accept-license`). Deployment target defaults to the existing 13.0
floor; the SDK version only affects `.tbd` stub coverage.
- **Compile**: `--target=arm64|x86_64-apple-macosx -isysroot <SDK>
-mmacosx-version-min=13.0`, and C++ uses the SDK's libc++ headers
(`-nostdinc++ -isystem <SDK>/usr/include/c++/v1`) so out-of-line libc++
symbols match the `libc++.tbd` actually linked (host libc++ 21 headers
reference `std::__hash_memory`, which Apple's libc++ doesn't export).
- **Link**: lld's Mach-O port (`ld64.lld`, ships with the LLVM 21
toolchain we already require) driven by clang with
`-mlinker-version=705` so the driver emits modern `-platform_version`
args. `-Wl,-ld_new` is native-only (ld64.lld doesn't know it).
- **Post-link**: `llvm-strip` + LLVM `dsymutil` (both handle Mach-O on
Linux) instead of GNU strip / native-only dsymutil.
- **Rust**: `aarch64/x86_64-apple-darwin` are Tier 2 targets — the
staticlib cross-compiles on Linux with prebuilt std; `SDKROOT` /
`CFLAGS_<triple>` / `MACOSX_DEPLOYMENT_TARGET` are forwarded for any
`cc`-crate build script.
- **Gaps filled**:
- x64 only: vendored compiler-rt `cpu_model/x86.c` (Apache-2.0 WITH
LLVM-exception) provides `___cpu_model` / `___cpu_indicator_init` for
`__builtin_cpu_supports` — on native builds these come from Apple's
static `libclang_rt.osx.a`, which the Linux toolchain doesn't ship.
Registered in `workarounds.ts` so it self-obsoletes if the SDK ever
exports them.
- x64 only: `-Wl,-sectalign,__BUN,__bun,0x1000` — ld64.lld otherwise
16K-aligns the placeholder section's file offset inside a 4K-aligned
segment and produces `filesize > vmsize` (invalid Mach-O that
llvm-strip/dsymutil reject). The placeholder only needs 8-byte alignment
and `bun build --compile` rewrites the segment at 16K anyway.
- `workaround-missing-symbols.cpp`: `std::__libcpp_verbose_abort` now
matches the header's exception spec for libc++ ≤ 19 (Apple SDK headers
declare it without `noexcept`).
- ASAN/UBSan are forced off for darwin cross builds (no darwin sanitizer
runtimes in the Linux toolchain), mirroring the Android/FreeBSD
behavior. Release CI cross builds get **ThinLTO with cross-language
importing**: C++/JSC link against the ThinLTO macOS `-lto` WebKit
prebuilts, Rust participates via `-Clinker-plugin-lto`, the Mach-O link
runs through rustc's `ld64.lld` (rustc's LLVM is newer than clang's, and
bitcode is only forward-compatible), and `-Wl,-object_path_lto` keeps
the dSYM usable under LTO. See the LTO section below for why darwin is
thin and linux stays on full LTO.

### Usage (local)

```sh
bun scripts/build.ts --profile=release --os=darwin --arch=aarch64 --build-dir=build/darwin-arm64
bun scripts/build.ts --profile=release --os=darwin --arch=x64     --build-dir=build/darwin-x64
# optional: --macos-sdk=/path/to/MacOSX26.5.sdk  or  MACOS_SDK_PATH=...
```

Debug profiles work too (`--profile=debug --os=darwin ...`); ASAN is
automatically disabled for the cross target. Native Linux/macOS builds
are unaffected — the native Linux debug graph reconfigures byte-for-byte
identical with this change.

### CI

The darwin build lanes now run on the Linux fleet and produce the
artifacts that ship — the native mac build lanes are deleted and the mac
fleet only runs tests:

- `darwin-{aarch64,x64}` `build-{cpp,rust,bun}` run on Linux agents
(cpp/link on the amazonlinux Docker fleet, rust on the shared Linux rust
box), driven by the `crossCompile` flag + explicit `--os`/`--arch` build
args.
- Step keys, labels, and artifact names are the canonical ones
(`bun-darwin-{aarch64,x64}[-profile].zip`) — no `-cross` suffix — so the
release step, the darwin test steps, and `upload-release.sh` consume
them with no further changes. The release ships the Linux-built darwin
artifacts and binary-size tracking tracks them.
- The darwin test steps download these artifacts and run the full macOS
test suite on real macOS agents (macOS 14 + 26, Intel + Apple Silicon).
- No image changes required: the SDK is fetched per build from Apple's
CDN (~59 MB, cached like the WebKit prebuilt). `bootstrap.sh` now
preinstalls the apple-darwin rust targets on Linux so the rust step
stops needing `rustup target add` at build time once images are next
refreshed.

### Verification

Both targets were built end-to-end on a Linux x64 host (clang 21.1.8 +
ld64.lld + the MacOSX15.5 SDK those early builds used; numbers below are
from the pre-LTO local builds — the CI darwin lanes now build with full
+ cross-language LTO against the `-lto` WebKit prebuilts, and the SDK
pin has since moved to 26.5 fetched from Apple's CDN):

| | darwin-arm64 | darwin-x64 |
|---|---|---|
| stripped `bun` | 57.5 MB Mach-O arm64 | 61.8 MB Mach-O x86_64 |
| `bun-profile` + dSYM + linker map | ✅ | ✅ |
| `LC_BUILD_VERSION` | minos 13.0, sdk 15.5 | minos 13.0, sdk 15.5 |
| ad-hoc code signature + entitlements.plist (XML + DER blobs) | ✅ | ✅
(the shipping config has since dropped the x64 signature — see below) |
| `LC_MAIN.stacksize` | 0x1200000 (18 MB) | 0x1200000 (18 MB) |
| linked dylibs | libicucore, libresolv, libc++, libSystem | same |
| dyld export trie | all 554 symbols from `src/symbols.txt` incl. 145
`napi_*` | same |
| segment layout | valid | valid (after `-sectalign`) |

`ninja check` (smoke test) is skipped for cross targets, same as
Android/FreeBSD — the cross-built binaries now run the **full test suite
on real macOS hardware** (Intel + Apple Silicon test lanes) and have
been benchmarked on an M-series MacBook — see the Benchmarks section
below.

### Benchmarks (M-series MacBook, arm64)

Two questions, two comparisons. **(1) Is the artifact this PR ships
faster than the last released Zig build?** — `bun-darwin-aarch64` from
this branch (ThinLTO + cross-language importing + `--lto-O3` +
`--icf=safe` + the ThinLTO WebKit prebuilt) vs bun v1.3.14. **(2) What
does LTO itself buy?** — the same artifact vs a native non-LTO release
build at a nearby commit. Microbenchmarks via mitata / oha (3
interleaved rounds for servers); real-app suite via
`oven-sh/bun-perf-tester` (hyperfine + bombardier + GNU time).

#### vs bun v1.3.14 (the binary this replaces): wins 24, ties 7, loses 5

| | LTO cross build | v1.3.14 (zig) | Δ |
|---|---|---|---|
| HTTP plaintext, oha -c64 (req/s) | **150,757** | 149,358 | +0.9%,
ahead in 3/3 interleaved rounds |
| HTTP React SSR (req/s) | **96,780** | 94,854 | +2.0% |
| `next start` (req/s / max RSS) | **13,594 / 391 MB** | 12,887 / 433 MB
| +5.5% / −10% |
| `next build` | **4.99 s** | 5.39 s | +7.5% |
| vite build / `tsc -b` / eslint | **794 / 434 / 326 ms** | 818 / 448 /
371 ms | +3.0% / +3.1% / +12% |
| elysia build / test suite | **1.50 s / 1.61 s** | 1.57 s / 1.66 s |
+4.2% / +3.0% |
| `bun build` three.js ×10 --minify --sourcemap | **77.5 ms** | 83.1 ms
| +6.7% |
| JS↔native binding micros — `Headers.set`, `Response.json`,
`structuredClone`, `URLSearchParams`, `FormData.append`, `Bun.semver`,
`createHash`, promise resolution, small base64 | **wins all** | | +6% to
+37% |
| stripped binary size | **56.0 MB** | 60.2 MB | −7.0% |
| WebSocket pub/sub, startup, `bun install`, `Bun.serve` req/s under
bombardier, gzip, sqlite | | | ties |
| `TextEncoder.encode` ASCII / `escapeHTML` no-escape / `stringWidth`
ASCII / `formData()` round-trip | | **2× / 7.7× / 4.4× / 1.5× faster** |
pre-existing Zig→Rust port gaps — see notes |

#### vs the native non-LTO release build (what LTO itself buys)

| workload | LTO vs native non-LTO | notes |
|---|---|---|
| JS ↔ native binding micros | **+5–17%** (`Headers.set` +21%) | exactly
where cross-language inlining should help |
| `next build` / vite / tsc / eslint / prettier / elysia tests |
**+1–4.5%** | sweeps every "run a real JS app/tool" workload |
| `Bun.serve` max RSS | **lower** | single sample; directionally
consistent |
| stripped binary size | **−2.2 MB** | on top of the −2.8 MB from ICF |
| HTTP plaintext throughput | −6% on the first LTO build → recovered |
see the HTTP note below |
| `bun install` (real registry) | −6% | |
| `Bun.stringWidth` ASCII SIMD path | −2× | SLP vectorizer — see below;
superseded by #31351 |

**The `stringWidth` regression is root-caused** (by replaying the exact
bitcode through the exact linker — the repro is bit-identical to the
shipped binary): `ld64.lld`'s LTO pipeline never runs the **SLP
vectorizer** at any `--lto-O` level. `lld/ELF/LTO.cpp` sets
`PTO.SLPVectorization = OptLevel > 1`; `lld/MachO/LTO.cpp` is missing
that line, so it inherits the default of *off* (#31343 tracks
upstreaming the one-liner). `printable_mask_latin1` (the `stringWidth`
ASCII kernel) is a Zig port whose `u8x16` "vectors" are scalar `for i in
0..16` stand-ins that depend 100% on SLP autovectorization — under the
Mach-O LTO it comes out fully scalar (131 instructions / 16 bytes, 0
NEON ops) instead of partially vectorized (75 instructions, 28 NEON
ops). Loop-vectorized and explicit-intrinsic code (simdutf, highway,
zlib, WebKit) is unaffected, which is why everything else got faster.
**#31351 supersedes this**: it rewrites `stringWidth` in C++ with
explicit highway SIMD, which doesn't depend on any autovectorizer — that
build measures 1.5–4.7× *faster* than the v1.3.14 Zig build on the same
inputs.

**The HTTP plaintext deficit is recovered.** The first LTO build trailed
both the native build (−6% under oha) and v1.3.14 on plaintext
throughput; `--lto-O3` alone did not move it (±0.1%). Marking the
`uws_*` C-ABI wrappers and the 1-line usockets accessors `always_inline`
did: the current build is ahead of v1.3.14 in all 3 interleaved rounds
(it previously trailed in 3/3) and ahead on React SSR, `next start`, and
the bundler.

The remaining v1.3.14 wins (`escapeHTML` no-escape 7.7×, `TextEncoder`
ASCII 1.4–2×, `formData()` round-trip 1.5×) predate this PR, are
unaffected by it, and share the same root cause as the `stringWidth`
row: the original `@Vector` SIMD code was ported as scalar stand-ins
behind `ENABLE_SIMD = false` / `ScalarVec` in
`bun_core/string/immutable.rs`. They reproduce identically on the native
non-LTO build.

### LTO: ThinLTO on darwin, full LTO on linux

The darwin release links use **ThinLTO** (`-flto=thin
-fno-split-lto-unit` + `-mllvm -whole-program-visibility` +
`--lto-O3/--lto-CGO3` on arm64): every module carries a summary, the LTO
backends run in parallel, and the link drops from **860 s to 81 s** on a
64-core box. ThinLTO also makes **Rust↔C++ cross-language inlining real
for the first time** — under full LTO the C++ lands in one regular-LTO
partition and the Rust CGUs in ThinLTO partitions, and the two never
exchange function bodies, so every one of the ~15,700 calls across the
`extern "C"` boundary (e.g. 940 calls to `JSC__JSGlobalObject__vm`,
which is a single load) stayed an out-of-line call. With ThinLTO +
`__attribute__((always_inline))` on the hottest tiny boundary functions
(the JSC binding getters, the `uws_*` C-ABI wrappers, the 1-line
usockets accessors), **4,663 cross-language calls are eliminated** for
+131 KB.

**Linux stays on full LTO.** The LLVM 22 ThinLTO backend pipeline
(rust-lld) miscompiles JavaScriptCore on linux — a JIT-tier correctness
bug (`require("axobject-query")` fails once the surrounding functions
tier up to the DFG) plus several bundler-test hangs, on x64 and aarch64,
glibc and musl, absent on darwin. Bisected with a relink-only harness:
fixed at `--lto-O0`, broken at O1/O2/O3, broken with cross-module
importing disabled, with WPD disabled, with split LTO units disabled,
and with TBAA disabled; ICF was separately ruled out. The miscompiling
pass is in the core O1 simplification pipeline; isolating it needs a
per-module bisection and is a follow-up. Linux therefore keeps the exact
full-LTO configuration every previous green build used (the `-flto=full`
entry in `flags.ts` documents the repro).

### Differences vs the previously mac-built binaries (these now ship —
validated by the full test suite on real macOS hardware and the
benchmarks above)

- `-Wl,-no_compact_unwind` is accepted but not implemented by ld64.lld
21 → the shipped binary now carries a `__unwind_info` section the
mac-built one didn't. (`-Wl,-stack_size` is also unimplemented, but the
post-link fixup below patches `LC_MAIN.stacksize` to the same 18 MB the
native link got.)
- The darwin link now enables `-faddrsig` + `-Wl,--icf=safe`
(identical-code folding, same as the Linux release link) and, for
release CI builds, full + cross-language LTO. Apple's ld has no ICF and
the previous native release builds didn't do LTO, so the shipped binary
comes out smaller than before (~5% from ICF alone in the pre-LTO local
builds, another −2.2 MB from LTO — see Benchmarks).
- arm64: the ad-hoc signature is generated by `macho-postlink` rather
than Apple's `codesign`: same CodeDirectory layout, entitlements, and
DER encoding (byte-identical to the previously shipped binary's), but no
Developer ID / hardened runtime — those need Apple tooling and are
applied by the release pipeline, not the build. x64 ships unsigned,
exactly like the previously mac-built x64 binary (Apple's toolchain only
auto-signs arm64).
- The x64 `__BUN` placeholder section is 4 KB-aligned instead of 16 KB
(see above; rewritten on `bun build --compile` anyway).
- Binaries are linked against SDK 26.5 stubs fetched from Apple's CDN
(previously whatever Xcode shipped on the mac builders); deployment
target is unchanged at 13.0.


### Post-link fixups (`scripts/build/shims/macho-postlink.c`)

A standalone host tool (single C11 file, libc only — `cc -O2
macho-postlink.c` works on any POSIX host including a Mac) appended to
the link **and** strip rule commands for darwin cross targets (`… -o
$out && macho-postlink $out --stack-size=0x1200000
--entitlements=entitlements.plist`, the `--entitlements` part on arm64
only):

- **Stack size** — ld64.lld parses `-stack_size` but doesn't implement
it, so the tool sets `LC_MAIN.stacksize` in place. Registered in
`workarounds.ts` with a probe that fails configure once ld64.lld starts
honoring the flag we already pass.
- **Code signature + entitlements (arm64 only)** — the header edit
invalidates the linker's ad-hoc signature (page 0 is hashed into the
CodeDirectory), so the tool regenerates it, embedding
`entitlements.plist` as both the XML blob and the DER blob (AMFI
requires the DER form for minos ≥ 12). `-Wl,-adhoc_codesign` is forced
on the arm64 link so `LC_CODE_SIGNATURE` always exists for the tool to
replace, and the tool errors if asked to embed entitlements into an
unsigned binary — arm64 can't silently ship unsigned. x64 ships unsigned
like the native x64 build always did (x64 macOS runs unsigned binaries
fine; the CodeDirectory would cost ~0.8% of the binary) and only gets
the stack-size patch. Debug builds sign with `entitlements.debug.plist`
(adds `get-task-allow`/`cs.debugger`), matching `scripts/trace.sh`.
llvm-strip regenerates a bare signature on its output, so the stripped
arm64 `bun` gets its own fixup pass.
- Verified against the official `bun-v1.3.14` darwin release binaries
(arm64 + x64) and the actual cross-built artifacts with an independent
parser that recomputes every code-page hash and special-slot hash; the
DER entitlements blob comes out byte-for-byte identical to the one
Apple's `codesign` embedded in the shipped binary for the same plist.
The cross-built binaries pass the full test suite on real macOS hardware
(Intel + Apple Silicon test lanes).

### features.json

The cross lanes previously uploaded a `{ crossTarget }` stub because the
binary can't run on the build host. Every field is a build-time
constant, so it's now generated host-side: the feature list is parsed
out of the `define_features!` invocation in `src/analytics/lib.rs`
(`scripts/build/features-json.ts`), and version/is_canary/revision come
from the build config. A test runs the debug build's
`crash_handler.getFeatureData()` and requires the parsed list to match
exactly, so the parser can't silently drift from the binary (the list
maps crash-report bit indices to names — order matters).

### Tests

`test/internal/macos-cross-config.test.ts` pins the cross-compile
configure logic: `--os=darwin` on a non-darwin host resolves to
`arm64|x86_64-apple-macosx` with ld64.lld/llvm-strip, sanitizers forced
off (LTO stays on for release CI cross builds), SDK path resolution
(`--macos-sdk` / `MACOS_SDK_PATH` / cache fallback, and the cache never
serving a stale SDK version), the cross compile/link flag set
(`-isysroot`, SDK libc++, no `-Wl,-ld_new`, x64 `-sectalign`),
apple-darwin rust triples, macOS WebKit prebuilt naming, and that native
Linux configs resolve exactly as before. Also: the post-link command on
cross links only (stack size on both arches, per-profile entitlements
plist on arm64), `-adhoc_codesign` on arm64 but not x64, and the
host-side features.json parser pinned against the real binary's
`getFeatureData()`.

### Follow-ups (tracked, none blocking this PR)

- [x] Repoint `WEBKIT_VERSION` at a regular `autobuild-<sha>` tag —
done; oven-sh/WebKit#238 merged and the pin now tracks main.
- [ ] Migrate the Linux release links to ThinLTO once the LLVM 22
ThinLTO-backend miscompile of JSC is isolated (repro + bisection harness
documented in the `-flto=full` comment in `flags.ts`); that also unlocks
Rust↔C++ cross-language inlining on Linux and drops the 14-minute serial
link.
- [x] #31342 — the darwin arm64 LTO link now passes
`--lto-O3`/`--lto-CGO3` (#31348 applied onto this branch; the bare `-O2`
it replaces was silently dropped by the driver). x64 keeps lld's default
`--lto-O2/CGO2` — O3 codegen costs +3.1 MB there vs +0.3 MB on arm64.
The HTTP plaintext A/B has since been re-run: `--lto-O3` alone did not
move it; `always_inline` on the uws_* boundary wrappers recovered it
(see Benchmarks).
- [ ] #31343 — `ld64.lld`'s LTO pipeline never runs the SLP vectorizer
(`lld/MachO/LTO.cpp` is missing the `PTO.SLPVectorization` line the ELF
port has); upstream the one-liner. Root cause of the `Bun.stringWidth`
−2×.
- [ ] #31351 — `Bun.stringWidth` → C++ with explicit highway SIMD,
deleting the Rust scalar-stand-in implementation. Measures 1.5–4.7×
faster than the 1.3.14 Zig build (and 1.4–3.8× faster than main).
- [ ] #31344 — Windows ASAN bun-webkit prebuilt (the cross-compile
plumbing is fully wired; needs a matrix entry and a consumer).
- [ ] #31345 — re-add `bun-webkit-windows-arm64-lto` once LLVM's
CodeView emitter maps ARM64 NEON register tuples.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Jarred Sumner <jarred@jarredsumner.com>