TUI automation for AI agents. A daemon + CLI that lets AI agents programmatically drive terminal applications via a gRPC API over Unix domain sockets.
"It's like Playwright, for TUI apps"
Use Cases
- Proof of work — Have agents showcase the feature they built or bug they fixed, and submit recordings alongside their PRs for easy reviews.
- Tight feedback loops — Tell agents to USE the TUI to make sure the UI works properly.
Install Skill
Add the virtui skill to your AI agent:
npx tessl i honeybadge/virtui
npx skills add honeybadge-labs/virtui
Then prompt your agent:
Use /virtui to create a recording of you opening vim, saving a file named hello.txt and exiting.
Install
Homebrew (macOS & Linux):
brew install honeybadge-labs/tap/virtui
Go:
go install github.com/honeybadge-labs/virtui/cmd/virtui@latest
Binary releases: download from GitHub Releases.
Quick Start
# 1. Start the daemon virtui daemon start # 2. Launch a terminal session virtui run bash # Output: session_id: a1b2c3d4 # 3. Run a command (type + Enter + wait for output) virtui exec a1b2c3d4 "echo hello" --wait "hello" # 4. Take a screenshot virtui screenshot a1b2c3d4 # 5. Clean up virtui kill a1b2c3d4 virtui daemon stop
JSON Mode
All commands support --json (-j) for machine-readable output:
virtui --json run bash # {"session_id":"a1b2c3d4","pid":1234,"recording_path":""} virtui --json screenshot a1b2c3d4 # {"screen_text":"...","screen_hash":"5da7...","cursor_row":3,"cursor_col":10,"cols":80,"rows":24}
Note: Fields backed by proto3
int64(elapsed_ms,created_at) are serialized as JSON strings per the proto3 JSON mapping.
Architecture
AI Agent / LLM
|
CLI (virtui) or Go SDK (import "github.com/honeybadge-labs/virtui")
|
gRPC over Unix domain socket (~/.virtui/daemon.sock)
|
Daemon (session manager + terminal emulator per session)
|
PTY (creack/pty) + VT100 emulation (vt10x)
The daemon manages multiple terminal sessions. Each session owns a pseudo-terminal and a VT100 emulator. Every response includes a SHA-256 screen hash for cheap change detection without transferring screen contents.
CLI Reference
Global Flags
| Flag | Short | Env | Default | Description |
|---|---|---|---|---|
--json |
-j |
false |
Output in JSON format | |
--socket |
VIRTUI_SOCKET |
~/.virtui/daemon.sock |
Daemon socket path |
virtui daemon start
Start the daemon process.
virtui daemon start # background (detached) virtui daemon start --foreground # foreground (blocks) virtui --json daemon start # {"pid":1234,"socket":"/Users/you/.virtui/daemon.sock"} virtui --json daemon start --foreground # {"socket":"/Users/you/.virtui/daemon.sock"}
| Flag | Default | Description |
|---|---|---|
--foreground |
false |
Run in the foreground instead of detaching |
virtui daemon stop
Stop the daemon gracefully. Sends a shutdown request and waits for the daemon to exit before returning.
virtui daemon stop
virtui --json daemon stop # {"ok":true}virtui daemon status
Check if the daemon is running.
virtui daemon status # daemon: running (socket: /Users/you/.virtui/daemon.sock) virtui --json daemon status # {"running":true,"socket":"/Users/you/.virtui/daemon.sock"}
virtui run <command...>
Spawn a new terminal session running the given command.
virtui run bash virtui run --cols 120 --rows 40 vim file.txt virtui run --record bash virtui run --record --record-path ./demo.cast bash virtui run -e TERM=dumb -e FOO=bar bash
| Flag | Default | Description |
|---|---|---|
--cols |
80 |
Terminal columns |
--rows |
24 |
Terminal rows |
-e, --env |
Environment variables (KEY=VALUE), repeatable |
|
--dir |
Working directory for the child process | |
--record |
false |
Record session in asciicast v2 format |
--record-path |
auto | Custom recording path (default: ~/.virtui/recordings/<id>.cast) |
Output (JSON):
{
"session_id": "a1b2c3d4",
"pid": 1234,
"recording_path": "/Users/you/.virtui/recordings/a1b2c3d4.cast"
}virtui exec <session> <input>
The primary command for AI interaction. Types the input, presses Enter, and optionally waits for a screen condition before returning.
Caveat: Wait conditions check the screen immediately after input is sent. If the target text already appears (e.g., inside the typed command itself), the wait can resolve in 0 ms — before the command's actual output appears. For reliable results use a pipeline with separate
type→press Enter→waitsteps, or followexecwith a standalonewaitcommand.
# Type + Enter (fire and forget) virtui exec a1b2c3d4 "ls -la" # Type + Enter + wait for text to appear virtui exec a1b2c3d4 "npm install" --wait "added" # Type + Enter + wait for screen to settle (500ms of no changes — does NOT guarantee the process finished) virtui exec a1b2c3d4 "make build" --wait-stable # Type + Enter + wait for text to disappear virtui exec a1b2c3d4 "make" --wait-gone "compiling..." # Type + Enter + wait for regex match virtui exec a1b2c3d4 "node --version" --wait-regex "v\d+\.\d+" # With custom timeout virtui exec a1b2c3d4 "npm install" --wait "added" --timeout 60000
| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
input |
yes | Text to type (Enter is appended automatically) |
| Flag | Default | Description |
|---|---|---|
--wait |
Wait for this text to appear on screen | |
--wait-stable |
false |
Wait for 500 ms of no screen changes (does not guarantee process finished) |
--wait-gone |
Wait for this text to disappear from screen | |
--wait-regex |
Wait for a regex pattern to match on screen | |
--timeout |
30000 |
Timeout in milliseconds |
Output (JSON):
{
"screen_text": "$ ls -la\ntotal 42\n...",
"screen_hash": "5da7a532...",
"cursor_row": 10,
"cursor_col": 2,
"elapsed_ms": "150"
}virtui screenshot <session>
Capture the current terminal screen contents.
virtui screenshot a1b2c3d4 # plain text to stdout virtui --json screenshot a1b2c3d4 # structured JSON with hash
| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
Output (JSON):
{
"screen_text": "$ echo hello\nhello\n$",
"screen_hash": "a3f2...",
"cursor_row": 2,
"cursor_col": 2,
"cols": 80,
"rows": 24
}virtui press <session> <keys...>
Send one or more key presses to the terminal.
virtui press a1b2c3d4 Enter
virtui press a1b2c3d4 ArrowDown --repeat 5
virtui press a1b2c3d4 Ctrl+C
virtui press a1b2c3d4 Escape q # press Escape then q| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
keys |
yes | One or more key names (see Key Names) |
| Flag | Default | Description |
|---|---|---|
--repeat |
1 |
Number of times to repeat the key sequence |
virtui type <session> <text>
Type text into the terminal without pressing Enter. Use this for partial input, search fields, or when you need to type without submitting.
virtui type a1b2c3d4 "hello world"
| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
text |
yes | Text to type (Enter is NOT appended) |
virtui wait <session>
Block until a screen condition is met. Returns the screen state when the condition is satisfied.
virtui wait a1b2c3d4 --text "Ready" virtui wait a1b2c3d4 --stable virtui wait a1b2c3d4 --gone "Loading..." virtui wait a1b2c3d4 --regex "v\d+\.\d+" virtui wait a1b2c3d4 --text "Done" --timeout 60000
| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
| Flag | Default | Description |
|---|---|---|
--text |
Wait for this text to appear | |
--stable |
false |
Wait for 500 ms of no screen changes (not process completion) |
--gone |
Wait for this text to disappear | |
--regex |
Wait for a regex pattern to match | |
--timeout |
30000 |
Timeout in milliseconds |
Output (JSON):
{
"screen_text": "...",
"screen_hash": "...",
"elapsed_ms": "2340"
}virtui kill <session>
Terminate a session and its child process.
virtui resize <session>
Resize the terminal dimensions of a running session.
virtui resize a1b2c3d4 --cols 120 --rows 40
| Flag | Required | Description |
|---|---|---|
--cols |
yes | New column count |
--rows |
yes | New row count |
virtui sessions
List all active sessions.
virtui sessions # ID PID COMMAND SIZE STATUS # a1b2c3d4 1234 bash 80x24 running # e5f6a7b8 5678 vim 120x40 running
virtui sessions show <session>
Show details for a specific session.
virtui sessions show a1b2c3d4 virtui --json sessions show a1b2c3d4
Output (JSON):
{
"sessions": [
{
"session_id": "a1b2c3d4",
"pid": 1234,
"command": ["bash"],
"cols": 80,
"rows": 24,
"running": true,
"exit_code": -1,
"created_at": "1711900000",
"recording_path": ""
}
]
}virtui pipeline <session>
Execute a batch of operations in a single call. Steps run sequentially. Reads
step definitions from a file (--file) or stdin.
# From file virtui pipeline a1b2c3d4 --file steps.json # From stdin echo '{"steps":[...]}' | virtui pipeline a1b2c3d4
| Argument | Required | Description |
|---|---|---|
session |
yes | Session ID |
| Flag | Default | Description |
|---|---|---|
--file |
Path to JSON file with steps (reads stdin if omitted) |
Pipeline JSON format:
{
"steps": [
{
"exec": {
"input": "echo hello",
"wait": { "text": "hello" },
"timeout_ms": 5000
}
},
{
"sleep": { "duration_ms": 500 }
},
{
"screenshot": {}
},
{
"press": {
"keys": ["ArrowDown"],
"repeat": 3
}
},
{
"type": { "text": "search query" }
},
{
"wait": {
"condition": { "stable": true },
"timeout_ms": 10000
}
}
],
"stop_on_error": true
}Available step types: exec, press, type, wait, screenshot, sleep
Key Names
The press command accepts the following key names:
| Category | Keys |
|---|---|
| Standard | Enter, Tab, Backspace, Escape, Space, Delete |
| Arrows | ArrowUp / Up, ArrowDown / Down, ArrowRight / Right, ArrowLeft / Left |
| Navigation | Home, End, PageUp, PageDown, Insert |
| Function | F1 through F12 |
| Ctrl | Ctrl+A through Ctrl+Z, Ctrl+[, Ctrl+], Ctrl+\ |
Single characters (e.g., a, 1, /) are also accepted and sent as-is.
Asciicast Recording
Sessions can be recorded in asciicast v2 format,
compatible with asciinema play:
# Record with auto-generated path virtui run --record bash # recording: /Users/you/.virtui/recordings/a1b2c3d4.cast # Record with custom path virtui run --record --record-path ./demo.cast bash # ... use the session normally ... virtui kill a1b2c3d4 # Replay asciinema play ~/.virtui/recordings/a1b2c3d4.cast
Recording captures both input (agent keystrokes) and output (terminal responses) with timestamps. Recording stops automatically when the session is killed or the process exits.
Go SDK
For embedding virtui in Go programs:
package main import ( "context" "fmt" "log" "github.com/honeybadge-labs/virtui" ) func main() { ctx := context.Background() // Connect to the daemon c, err := virtui.Connect("~/.virtui/daemon.sock") if err != nil { log.Fatal(err) } defer c.Close() // Start a session sess, err := c.Run(ctx, []string{"bash"}) if err != nil { log.Fatal(err) } defer c.Kill(ctx, sess.SessionID) // Execute a command and wait for output screen, err := c.Exec(ctx, sess.SessionID, "echo hello", virtui.WaitText("hello"), virtui.WithTimeout(5000), ) if err != nil { log.Fatal(err) } fmt.Println(screen.Text) // Take a screenshot ss, err := c.Screenshot(ctx, sess.SessionID) if err != nil { log.Fatal(err) } fmt.Printf("Screen hash: %s\n", ss.Hash) // Send key presses c.Press(ctx, sess.SessionID, "Ctrl+C") // Type without Enter c.Type(ctx, sess.SessionID, "exit") c.Press(ctx, sess.SessionID, "Enter") }
SDK API
| Method | Description |
|---|---|
Connect(socketPath) |
Connect to the daemon |
Run(ctx, command, ...RunOpts) |
Start a session |
Exec(ctx, sessionID, input, ...WaitOption) |
Type + Enter + optional wait |
Screenshot(ctx, sessionID) |
Capture screen |
Press(ctx, sessionID, keys...) |
Send key presses |
Type(ctx, sessionID, text) |
Type text (no Enter) |
Kill(ctx, sessionID) |
Terminate session |
Resize(ctx, sessionID, cols, rows) |
Resize terminal |
Wait Options
| Function | Description |
|---|---|
WaitText(text) |
Wait for text to appear |
WaitStable() |
Wait for screen to stabilize |
WaitGone(text) |
Wait for text to disappear |
WaitRegex(pattern) |
Wait for regex match |
WithTimeout(ms) |
Set wait timeout in ms |
Structured Errors
All errors returned by the daemon include structured information. When --json is set,
errors are output as JSON to stdout:
{
"code": "SESSION_NOT_FOUND",
"category": "ERROR_CATEGORY_SESSION",
"message": "session \"abc\" not found",
"retryable": false,
"suggestion": "Check the session ID with 'virtui sessions'.",
"context": {"session_id": "abc"}
}| Field | Description |
|---|---|
code |
Machine-readable error code (e.g., SESSION_NOT_FOUND, TIMEOUT) |
category |
Error category (SESSION, TERMINAL, TIMEOUT, VALIDATION, DAEMON) |
message |
Human-readable description |
retryable |
Whether the operation can be retried |
suggestion |
Suggested action to resolve the error |
context |
Additional key-value context (e.g., session_id) |
Screen Hashes
Every response that includes screen content also returns a screen_hash (SHA-256
of the screen text). Use this for cheap change detection:
HASH=$(virtui --json screenshot $SID | jq -r .screen_hash) # ... later ... NEW_HASH=$(virtui --json screenshot $SID | jq -r .screen_hash) if [ "$HASH" != "$NEW_HASH" ]; then echo "Screen changed!" fi
