Screen Capture: Recordings Your AI Can Actually Read

4 min read Original article ↗

Your AI agent has never actually seen your website move. It reads DOM snapshots, it takes screenshots, and between those stills, everything interesting — the modal that stutters, the skeleton that flashes, the button that changes state a beat too late — happens invisibly.

CBrowser 18.72 ships Screen Capture: record any page to GIF, WebP, or WebM, and get back something no video file can give you — a frame manifest your AI can actually read.

One capture, two artifacts

cbrowser capture start "https://your-site.com" --duration 5s --format gif

That produces a human artifact — the GIF you drop into a bug report — and manifest.json, the machine artifact: every frame with its wall-clock timestamp, the crop rect it came from, a change score against the previous frame, and the console messages and network requests that landed in that frame's window.

The premise is blunt: a language model cannot watch a video. Hand your agent an .mp4 and you've handed it nothing. Hand it the manifest and "what happened when the cart badge updated" becomes a lookup — find the change point, read the frame's network entries, open exactly that JPEG.

Change points, and the frames that matter

Every frame carries a change score, and the manifest reports two tiers on top of them:

  • change_points — every frame where anything changed

  • key_frames — the frames where something notable happened

On a page with a progress bar, nearly every frame is technically a change — so change_points saturates, says so honestly via change_points_saturated: true, and key_frames still hands you the six frames worth looking at.

The change detector is worth a word: it takes the minimum SSIM over a grid of windows rather than a whole-frame average, because a whole-frame average is blind to small changes — a ticking counter on a large page scores as "nothing happened" at any threshold. Measuring the change where it happens means a status badge flipping in a corner registers the same whether your viewport is 640px or 4K. The manifest records the detector and thresholds it was built with, so the numbers are never ambiguous.

Aim it at exactly what you care about

cbrowser capture start "$URL" --duration 5s --device iphone-15

# a fixed region
cbrowser capture start "$URL" --duration 5s --region 100,200,640,480

# an element — the crop FOLLOWS it as it moves
cbrowser capture start "$URL" --duration 5s --element "#cart-badge" --element-padding 10

Element capture re-resolves the bounding box every frame, so the crop tracks the element through animations and layout shifts while output dimensions stay locked for clean encoding.

Declarative start and stop

Stop babysitting recordings. Compose triggers instead:

cbrowser capture start "$URL" \
  --after networkidle \
  --after-element ".modal" \
  --until-idle 1500 \
  --timeout 30s

Start when the page settles and the modal appears; stop when the pixels stop moving; never hang past the timeout. A trigger that never fires exits non-zero — and still writes the manifest with everything it captured.

Record real flows, not just URLs

# record a natural-language test run
cbrowser test-suite checkout.txt --capture

# record a cognitive journey
cbrowser cognitive-journey --persona first-timer --start "$URL" --goal "sign up" --capture

And with the daemon, a capture survives across commands — start recording, then drive the same page from separate invocations while it rolls:

cbrowser daemon start
cbrowser capture start "$URL"
cbrowser fill "#email" "[email protected]"
cbrowser click "Sign up"
cbrowser capture stop

The details that bit us so they won't bite you

Capture is event-driven — Chromium emits frames when the page repaints, so a static page produces a handful of frames and an honest frame_gaps entry rather than padded duplicates, and playback still runs at true speed. Network entries are redacted to origin + path, because query strings carry tokens and a manifest is exactly the kind of file that gets shared. And capture is a different command from record — the older record captures your actions for test generation; capture records the screen.

Try it

npm i -g cbrowser
cbrowser capture start "https://your-site.com" --duration 5s --format gif

Full reference — targets, triggers, the manifest schema, MCP tools, live daemon capture — in the Screen Capture docs.