GitHub - vishal-dehurdle/state-harness: Runtime safety net for LLM agents. Detects token spirals, kills doomed tasks early, tells you exactly why. Rust core, Python SDK. pip install state-harness

Runtime safety net for LLM agents. Does nothing when things work. Saves your budget and tells you exactly why when they don't.

from state_harness import GrowthRatioGuard, FailureReport

guard = GrowthRatioGuard(token_budget=50_000)

with guard:
    for turn in agent_loop:
        result = llm.invoke(turn.prompt)
        guard.record_step(tokens_used=result.usage.total_tokens)

# What went wrong? (zero-cost, no LLM calls)
report = FailureReport.from_guard(guard)
print(report)

⚠️  STABILITY TRIPPED at turn 12

Pattern: Context Accumulation Spiral (confidence: 92%)
  • Last 5 turns all exceeded 1.5× baseline (4/4 were accelerating).
  • Peak growth ratio: 5.2× baseline.
  • Without intervention, projected cost was $0.0396 (actual: $0.0039).

Energy: ▁▁▁▁▁▂▂▃▄▆█
  Baseline: 1050 tokens/turn
  Peak ratio: 5.2× baseline

Cost: $0.0039 (saved ~$0.0357 by tripping early)

Suggested actions:
  🔴 1. Enable RG history compression in your agent loop.
     → Compressing older messages reduces prompt tokens by 40-60%.
  🟡 2. Lower the growth ratio threshold to 1.8×.
     → A lower threshold would have caught it earlier.
  🟢 3. Add a sliding-window context strategy.
     → Send only the last N messages plus a summary of earlier ones.

Why this exists

Every team running LLM agents in production has experienced this: an agent gets stuck in a loop, token usage spirals, and you find a $15 charge for a single failed request the next morning. You kill the process — but you have no idea why it happened or how to prevent it next time.

A hard budget cap solves the cost problem — but tells you nothing. You know the task was killed. You don't know if it was a context accumulation spiral, a retry storm, or policy drift. You can't fix what you can't diagnose.

State-harness is a library, not a platform. pip install and go. It uses Lyapunov stability theory to detect runaway behavior before it becomes expensive — and when it trips, it classifies the failure pattern and tells you exactly what went wrong, how to fix it, and how much you saved. All at zero cost — no extra LLM calls, no external APIs.

What it catches

What you get — and what you don't

The value is diagnostics. A budget cap tells you "task killed." State-harness tells you "task killed because of a context accumulation spiral — enable history compression to fix it." That difference is why this exists.

Who should use this

• Teams running search-tree agents (MCTS, beam search) — the architecture behind SWE-bench solvers and tools like Devin. Branches, not loops, drive cost. A per-branch iteration cap looks fine in isolation; the tree-level cost explosion happens silently.
• Platform teams running 1,000+ agent tasks/day — manual trace inspection doesn't scale. State-harness classifies failure patterns at the edge (zero cost, no LLM calls) and exports them as OpenTelemetry attributes for aggregate analysis.
• Researchers benchmarking agents — the nondeterminism floor (~4–5% stdev on Gemini 2.5 Flash) means single-run comparisons with <8% delta are noise. State-harness quantifies this.

Who should NOT use this

• Chatbots, RAG pipelines, or single-turn apps — these don't spiral. You don't need monitoring.
• Simple ReAct loops with <10 turns — max_iterations=10 and a budget cap are sufficient. Every modern framework (LangGraph, CrewAI) supports this natively.

Installation

pip install state-harness

Requires Python ≥ 3.10. Pre-built wheels are available for Linux, macOS, and Windows (x86_64 and ARM64). No Rust toolchain needed.

From source (for development)

git clone https://github.com/vishal-dehurdle/state-harness.git
cd state-harness

python -m venv .venv && source .venv/bin/activate

# Install Rust (if not already installed)
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

pip install maturin
maturin develop --release

# Run tests
pip install pytest
pytest tests/

Quickstart

Basic: GrowthRatioGuard (recommended)

The GrowthRatioGuard normalizes token usage against a baseline, so it only trips on disproportionate growth — not the natural growth of multi-turn context windows.

from state_harness import GrowthRatioGuard, StabilityViolation

guard = GrowthRatioGuard(
    token_budget=100_000,     # hard ceiling
    ratio_threshold=2.0,      # trip when turn is 2× the baseline
    window=3,                 # 3 consecutive escalating turns to trip
    budget_gate=8_000,        # don't trip until 8K tokens spent
)

with guard:
    for turn in agent_loop:
        try:
            result = llm.invoke(turn.prompt)
            guard.record_step(
                tokens_used=result.usage.total_tokens,
                errors=0,
            )
        except StabilityViolation as e:
            print(f"Agent killed: {e}")
            break

print(f"Total cost: {guard.total_tokens} tokens")
print(f"Baseline: {guard.baseline} tokens/turn")
print(f"Peak ratio: {guard.current_ratio}×")

Failure Diagnostics

After any execution (tripped or not), get a structured failure report:

from state_harness import FailureReport

report = FailureReport.from_guard(guard, model="gemini-2.5-flash")

# Human-readable terminal output
print(report)

# Structured dict for logging / dashboards
import json
print(json.dumps(report.to_dict(), indent=2))

The report classifies the failure pattern, provides evidence, estimates cost impact, and suggests specific fixes — all without any LLM calls.

Classic: BoundaryGuard

For lower-level control using raw token counts (no normalization):

from state_harness import BoundaryGuard

with BoundaryGuard(token_budget=100_000, lambda_=1.0, window=5) as guard:
    for turn in agent_loop:
        result = llm.invoke(turn.prompt)
        guard.record_step(
            tokens_used=result.usage.total_tokens,
            errors=0,
            tool_name="search",
        )

Decorator: @boundary_guard

from state_harness import boundary_guard

@boundary_guard(
    token_budget=50_000,
    token_counter=lambda r: r.usage.total_tokens,
)
def agent_step(prompt: str):
    return llm.invoke(prompt)

Framework Integration

LangGraph (recommended)

from langgraph.prebuilt import create_react_agent
from state_harness.adapters import monitor_graph

agent = create_react_agent(model, tools=[search, calculate])
safe = monitor_graph(agent, token_budget=100_000)

result = safe.invoke({"messages": [("user", "Fix the login bug")]})

# After execution — always available:
print(safe.total_tokens)  # cumulative usage
print(safe.tripped)       # did stability trip?
print(safe.report)        # full FailureReport with pattern + suggestions

For streaming:

for chunk in safe.stream({"messages": [("user", "Refactor this module")]}):
    print(chunk)

With a trip callback (e.g., for Slack alerts):

safe = monitor_graph(
    agent,
    token_budget=100_000,
    on_trip=lambda report: slack.send(f"Agent tripped: {report.pattern}"),
)

from state_harness import BoundaryGuard
from state_harness.adapters import LangGraphMiddleware

guard = BoundaryGuard(token_budget=150_000)
middleware = LangGraphMiddleware(guard)

@middleware.wrap_tool
def search_database(query: str):
    return db.search(query)

with guard:
    result = agent.invoke({"messages": [...]})

CrewAI

from crewai import Agent, Task, Crew
from state_harness.adapters import CrewAICallback

callback = CrewAICallback(token_budget=200_000)

crew = Crew(
    agents=[researcher, writer],
    tasks=[research_task, write_task],
    step_callback=callback.step_callback,
    task_callback=callback.task_callback,
)

result = crew.kickoff()
print(callback.report)  # FailureReport
callback.close()

Vanilla Python Hooks

from state_harness import BoundaryGuard
from state_harness.adapters import VanillaHook

guard = BoundaryGuard(token_budget=50_000)
hook = VanillaHook(guard)

with guard:
    for step in agent_loop:
        hook.before_call(tool_name="search")
        result = execute_tool(step)
        hook.after_call(tokens_used=result.tokens)

CLI

# Simulate a token trajectory — see what the guard would do
state-harness simulate 1000 1200 1500 2000 3000 5000 8000 --budget 50000

# Analyze a saved report
state-harness analyze report.json
state-harness analyze report.json --json    # JSON output
state-harness analyze report.json --otel    # OpenTelemetry attributes

# Batch analyze all reports in a directory
state-harness batch --dir ./reports/ --output results.csv

Structured Output

Every FailureReport supports multiple output formats:

report = FailureReport.from_guard(guard)

# JSON (for logging, APIs, storage)
report.to_json()            # pretty-printed
report.to_json(indent=None) # compact, single line

# CSV (for batch analysis of 1000s of runs)
with open("results.csv", "w") as f:
    f.write(FailureReport.csv_header() + "\n")
    for r in reports:
        f.write(r.to_csv_row() + "\n")

# OpenTelemetry (for Datadog, Grafana, Honeycomb)
from opentelemetry import trace
span = trace.get_current_span()
span.set_attributes(report.to_otel_attributes())
# Adds: state_harness.pattern, state_harness.confidence, etc.

Architecture

State-harness combines three physics-inspired mechanisms, implemented in Rust for microsecond-speed enforcement:

graph TD
    A["Agent Loop"] --> B["GrowthRatioGuard\n(Python SDK)"]
    B --> |"Normalizes tokens → growth ratio\nWarmup baseline · Budget gate"| C{" "}
    C --> D["Lyapunov Monitor\nV(k) = S + λθ\nΔV ≥ 0?"]
    C --> E["RG Decimator\nTF-IDF\nCompression"]
    C --> F["Holographic Engine\n(VSA)\nDrift Detection"]
    
    style D fill:#1a1a1a,stroke:#555,color:#e8e8e8
    style E fill:#1a1a1a,stroke:#555,color:#e8e8e8
    style F fill:#1a1a1a,stroke:#555,color:#e8e8e8
    style B fill:#0d1117,stroke:#30363d,color:#e6edf3

All three mechanisms are implemented in Rust (via PyO3) for microsecond-speed enforcement.

Benchmarks

Evaluated across four complementary benchmarks with a 5-condition ablation study (3,175 total runs) isolating each mechanism's contribution. Full methodology and data in the research paper.

Ablation Conditions

Summary: Non-invasive monitoring with zero-cost diagnostics

What the harness does — and doesn't do:

• ✅ Never interferes with healthy agents — zero stability trips across 1,886 short/medium-loop runs (MINT + τ³)
• ✅ Saves compute on spiraling tasks — 38.6% fewer search nodes, 30% faster wall time on SWE-bench
• ✅ Tells you why tasks failed — zero-cost failure diagnostics (context spiral, retry storm, policy drift) with actionable fixes
• ⚠️ Does not improve resolve rate — multi-trial SWE-bench (333 runs) confirms: harness 40.5% ± 2.7% vs naive cap 45.9% ± 5.4% vs baseline 44.1% ± 4.1% — all within noise

A naive budget cap achieves comparable task success rates. The harness's unique value is diagnostics (understanding why failures happen) and compute efficiency (33% fewer nodes than naive cap).

SWE-bench Verified (central result)

37 Django instances from SWE-bench Verified. Agent: moatless-tools SearchTree with 50-node budget. Model: Gemini 2.5 Flash.

Single-trial ablation (148 runs)

Note: Single-trial resolve rates have ~±8pp standard error. E's apparent 56.8% is not statistically significant vs A's 40.5%. Multi-trial results below confirm this.

What the harness provides:

• Compute-efficient: 38.6% fewer search tree nodes than baseline, 33% fewer than naive cap
• Faster: 30% wall-time reduction (80 → 56 min)
• Eliminates burnout: Baseline had 7 tasks burning the full 50-node budget (all failed). With monitoring: zero
• Diagnostics: Every tripped task gets a classified failure pattern with actionable fix suggestions — at zero LLM cost
• Simple integration: Lyapunov monitoring alone (Condition B) delivers ~90% of total benefit — 5 lines of code

Ablation — each mechanism contributes independently:

Lyapunov monitoring alone delivers ~90% of the total benefit. RG decimation and VSA add incremental value.

Multi-trial validation (333 runs)

To quantify nondeterminism and validate the single-trial findings, we ran 3 independent trials per condition (A, D, E) across all 37 instances — 333 total runs (12 runs resulted in stuck Docker containers killed after 28+ min; counted as failures):

Key finding: Cross-condition variance (2.9%) ≤ within-condition nondeterminism (4.1%). The differences between conditions are entirely within the noise band of LLM nondeterminism — confirming non-invasiveness with statistical rigor.

Note on nondeterminism: The ~4% within-condition stdev converges with τ³-bench findings (±4.6%), establishing a ~4–5% nondeterminism floor as a fundamental property of Gemini 2.5 Flash on code tasks. Any single-run benchmark comparison is unreliable for deltas < 8%.

Statistical validation: Bootstrap confidence intervals (10,000 resamples) and Welch's t-tests confirm no significant pairwise differences: A−D = +3.6pp [−0.9, +8.1], p ≈ 0.17; A−E = −1.8pp [−8.1, +4.5], p ≈ 0.68; D−E = −5.4pp [−10.8, 0.0], p ≈ 0.09. Full analysis in the research paper §7.3.1.

τ³-bench Airline (non-invasiveness confirmation)

50 tasks × 3 trials × 5 conditions = 750 total runs. Agent handles airline reservations via tool calls. Model: Gemini 2.5 Flash. Concurrency=1.

Key findings:

• Zero stability trips across all 750 runs. The monitor correctly identifies all airline tasks as stable and never intervenes — confirming non-invasiveness on medium-loop customer-service agents.
• Pass-rate variance is LLM nondeterminism, not harness impact. The naive cap (E) — which has zero monitoring — shows a −16pp drop from baseline, worse than full-stack monitoring (D, −10pp). This confirms the ~10–16pp spread is intrinsic benchmark variance, not monitoring-caused regression.
• 25% of tasks flip pass/fail within the same condition across 3 trials — the airline domain's intrinsic nondeterminism floor (~±12pp).
• 8.1% cost savings from full-stack monitoring, with the harness observing passively (zero interventions).

MINT (non-invasiveness validation)

284 tasks × 4 conditions = 1,136 total runs across GSM8K (48), MATH (100), HumanEval (45), MBPP (91). Agent uses up to 5 turns per task.

Zero stability violations across all 1,136 runs. The monitor correctly identifies short-loop tasks as stable and never intervenes. Token usage is invariant (<2% overhead).

Failed tasks cost disproportionately more — validating the economic thesis:

Note: HumanEval and MBPP show 0% across all conditions due to a MINT framework limitation in code execution evaluation — consistent across all conditions, confirming the harness does not introduce new failure modes.

Local Model Validation (edge deployment)

20 custom tasks (5 easy, 10 medium, 5 hard) × 4 models × 3 conditions = 240 runs. Hardware: Apple M4 MacBook Pro, 16 GB RAM, Ollama local inference.

Key findings:

• Zero false positives across all 80 harness runs — 4 model families, 3 difficulty tiers. The growth-ratio metric generalizes across architectures without threshold retuning.
• Small-model self-sabotage: Naive cap outperforms baseline by +17.5pp on average (median +12.5pp). Small models solve early turns correctly but destroy solutions in later turns. Strongest on Gemma4:E4B (+35pp).
• Model-family behavioral signatures:
  • Llama 3.2:3B: Classic spirals (ratios: 2.3×, 5.9×, 7.6×) — 3 true-positive trips
  • Phi-4-Mini: Spike-and-recover — 20.7% passive savings
  • Qwen3:4B: 255K tokens but flat ratios (≤1.06×) — correctly classified as stable despite 3× volume
  • Gemma4:E4B: Decreasing ratios — 37.9% passive savings, zero trips

Practical takeaway: If you’re deploying ≤4B models via Ollama, state-harness works out of the box with zero false positives. The self-sabotage finding suggests you should also add a turn limit (2–3 turns) for open-ended code generation tasks.

MINT on Qwen3:4B (568 runs)

Zero harness interventions across all 284 tasks. On MINT’s short-loop tasks (max 5 turns), the harness monitoring window (W=3) cannot trigger within the available post-warmup turns — a structural guarantee, not a probabilistic observation.

Reproducing the benchmarks

# 1. Clone repos
git clone https://github.com/vishal-dehurdle/state-harness.git
git clone https://github.com/sierra-research/tau-bench.git tau3-bench

# 2. Install state-harness
cd state-harness
python -m venv .venv && source .venv/bin/activate
pip install maturin && maturin develop --release

# 3. Install τ³-bench (with state-harness agent)
cd ../tau3-bench
uv sync
cp ../state-harness/tau3_integration/harness_agent.py src/tau2/agent/
cp ../state-harness/tau3_integration/naive_cap_agent.py src/tau2/agent/

# 4. Configure Vertex AI
export GOOGLE_CLOUD_PROJECT=your-project-id
export VERTEXAI_LOCATION=asia-south1

# 5. Run τ³ 5-phase benchmark
bash benchmarks/tau3/run_5phase_airline.sh

# 6. Run SWE-bench (requires Docker images)
bash benchmarks/swe_bench/run_benchmark.sh
bash benchmarks/swe_bench/run_benchmark_dbe.sh

# 7. Run MINT
bash benchmarks/mint/run_mint_fullstack.sh

See benchmarks/ for full setup, configs, and reproduction instructions for all three benchmarks.

Future evaluations

• Multi-trial SWE-bench — 333 runs (3 trials × 3 conditions × 37 instances) confirming non-invasiveness within ±4% noise band
• Local model validation — 240 runs across 4 open-weight models (Llama, Phi, Qwen, Gemma) + 568 MINT runs on Qwen3:4B
• Terminal-Bench — Terminal-based agent tasks; command-line tool loops where spirals manifest as repeated failed commands
• SWE-bench Pro — Harder, contamination-resistant variant of SWE-bench
• Cross-model validation — 7 models total: GPT-4o-mini, Claude Haiku 4.5, Gemini 2.5 Flash + Llama 3.2:3B, Phi-4-Mini, Qwen3:4B, Gemma4:E4B

Known limitations

1. 37 SWE-bench instances — A larger sample would improve statistical power (n=3 trials gives limited degrees of freedom for t-tests).
2. No causal intervention — The harness currently kills spiraling tasks. Redirect/repair is on the roadmap.
3. Physics-inspired, not physics-equivalent — Terms like "Renormalization Group" and "Lyapunov stability" are used as structural inspirations. The mathematical mapping is analogical, not isomorphic.
4. Custom benchmark scale — The 20-task local battery is smaller than standard benchmarks. The self-sabotage finding (mean +17.5pp, median +12.5pp) is consistent across 4 models but requires larger-scale replication.

Configuration Guide

Environment variable overrides (highest precedence, for threshold sweeps):

Tuning tips:

• More aggressive (catch spirals earlier): ratio_threshold=1.8, window=2
• More conservative (fewer false positives): ratio_threshold=2.5, window=3
• High-value tasks: Increase budget_gate to 20K+ to let expensive tasks run longer
• Complex domains (retail, multi-tool): Start with ratio_threshold=2.5

Theoretical Foundations

State-harness applies control theory to LLM agent execution:

• Lyapunov stability: The energy function V(k) = S(k) + λθ(k) models token consumption as a dynamical system. When ΔV ≥ 0 for W consecutive steps, the system is provably unstable.
• Renormalization Group (RG) theory: Message compression is modeled as coarse-graining — eliminating high-frequency noise while preserving scale-invariant task objectives.
• Vector Symbolic Architecture (VSA): Domain policies are bound to high-dimensional bipolar vectors (10,000-d, i8 space), enabling constant-time semantic drift detection outside the LLM context window.

Research

This library implements the framework described in:

Empirical Lyapunov Stability: Growth-Ratio Energy Functions as Leading Indicators of Agent Task Failure Vishal Verma, 2026 Read the full paper →

Key findings from the paper (updated with multi-trial and local model validation):

• Non-invasiveness confirmed across 333 SWE-bench runs — resolve rate delta (−3.6pp) falls within the ±4.1% nondeterminism band
• Zero stability violations across 1,886 short/medium-loop cloud runs (MINT + τ³) — the monitor never interferes with healthy agents
• Zero false positives across 80 local-model harness runs — spanning Llama 3.2:3B, Phi-4-Mini, Qwen3:4B, Gemma4:E4B on Apple M4 via Ollama
• Zero-cost failure diagnostics — every tripped task is classified (context spiral, retry storm, policy drift) with actionable fix suggestions, requiring no additional LLM calls
• Lyapunov monitoring alone delivers ~90% of the total benefit — the simplest integration (5 lines of GrowthRatioGuard code) captures the majority of the value
• On long-loop agents (SWE-bench), full-stack monitoring reduces compute by 38.6% and wall time by 30%
• Failed tasks cost 1.6–3.4× more than successful ones — economic justification for early termination
• Eliminates all max-budget burnout events (7 → 0 tasks hitting the 50-node ceiling on SWE-bench)
• ~4–5% nondeterminism floor established across both τ³-bench and SWE-bench — any single-run comparison is unreliable for deltas < 8%
• Small-model self-sabotage: Naive turn-limiting outperforms unconstrained baselines by +17.5pp on average on ≤4B models — runtime governance is capability-preserving, not just cost-saving

Based on the theoretical framework from:

The Fluid Dynamics of Multi-Agent AI: Resolving d'Alembert's Paradox of Generative Workflows Vishal Verma, 2026 Read →

Contributing

Contributions are welcome. See CONTRIBUTING.md for dev environment setup, code style, and PR guidelines.

Roadmap

• Adaptive threshold — Auto-tune τ based on task complexity signal from early turns
• Causal intervention — Instead of killing spiraling tasks, redirect them (prompt injection, tool restriction)
• Streaming support — Token-level monitoring for streaming LLM responses
• Multi-model validation — 7 models validated: GPT-4o-mini, Claude Haiku 4.5, Gemini 2.5 Flash + 4 local models via Ollama
• Dashboard / observability — Optional lightweight UI for monitoring energy trajectories in real-time

Security

For security vulnerabilities, see SECURITY.md. Please do not open public issues for security reports.

License

Split-core licensing:

See LICENSE.md for full details.