GitHub - sebringj/autonomo: Tired of 'it works' lies? Autonomo MCP makes your AI prove it—on real hardware, right in your editor.

Give your AI eyes and hands.

Tired of "it works" lies? Autonomo MCP makes your AI prove it—on real hardware, right in your favorite editor.

Your AI can finally see and drive your running apps — all of them, at once. It reads structured JSON state, interacts with UI elements, and knows instantly whether something worked or broke.

No screenshots. No hallucinations. No "trust me bro."

Just real, live, visual pair programming.

Works with: GitHub Copilot • Claude Code • Cursor • Windsurf • Any MCP-compatible AI

🌐 https://sebringj.github.io/autonomo/

What is Autonomo MCP?

Your AI coding assistant is brilliant. It can write code, refactor systems, and reason about architecture. But it's been doing all of that blind — unable to see what it built or drive the interfaces it created.

It writes a component, tells you "that should work," and moves on. You compile. You run. You check the screen. You report back: "Nope, still broken."

Autonomo fixes this.

It gives your AI live access to your app's structured JSON state — screens, elements, errors, user info — across web, iOS, Android, and desktop. Your AI can read that state, interact with UI elements, and know instantly whether something worked or broke. Not by looking at pixels — by reading real data. While you're still developing. Not after.

The old loop:

Write code → Compile → Run → Squint at screen → Describe what you see → Hope the AI understands

The Autonomo loop:

Tell the AI what you want → Watch it build and interact live → Correct in real time

How It Works

After every action, your AI gets a unified snapshot of everything that happened:

AI sends command: {"action": "press", "target": "Submit"}
                              ↓
            ═══ UNIFIED SNAPSHOT RETURNED ═══

UI State:
  screen: "confirmation"
  elements: ["Order.Number", "Order.Details", "Home.Button"]

App State:
  orderId: "ORD-12345"
  cartCleared: true

Network:
  POST /api/orders → 201 (245ms)
  response: { id: "ORD-12345", status: "confirmed" }

Errors:
  [] (none)

Console:
  ["Order created successfully"]

Your AI doesn't guess what happened. It sees everything. And if something broke, it fixes it immediately.

Why Not Screenshots?

Installation

JavaScript/TypeScript (npm)

# React
npm install autonomo-react

# Core (for vanilla JS/TS)
npm install autonomo-core

MCP Server (Global)

See QUICKSTART.md for complete setup instructions.

⚡ Get Started in 30 Seconds

Just tell your AI assistant:

Install Autonomo in my project. Read https://github.com/sebringj/autonomo/blob/main/QUICKSTART.md

Your AI handles the rest — installing packages, configuring MCP, and adding the bridge to your app.

Platform-Specific Prompts

Copy-paste the prompt for your platform:

React / Next.js / Remix:

Install Autonomo for my React app. Read https://www.npmjs.com/package/autonomo-react

React Native / Expo:

Install Autonomo for my React Native app. Read https://www.npmjs.com/package/autonomo-react-native

Swift / iOS:

Install Autonomo for my Swift iOS app. Read https://github.com/sebringj/autonomo/blob/main/packages/autonomo-swift/README.md

Flutter:

Install Autonomo for my Flutter app. Read https://github.com/sebringj/autonomo/blob/main/packages/autonomo_flutter/README.md

Python:

Install Autonomo for my Python app. Read https://github.com/sebringj/autonomo/blob/main/packages/autonomo-python/README.md

Ruby:

Install Autonomo for my Ruby app. Read https://github.com/sebringj/autonomo/blob/main/packages/autonomo-ruby/README.md

Kotlin / Android:

Install Autonomo for my Kotlin app. Read https://github.com/sebringj/autonomo/blob/main/packages/autonomo-kotlin/README.md

C# / .NET:

Install Autonomo for my C# app. Read https://github.com/sebringj/autonomo/blob/main/packages/Autonomo.CSharp/README.md

Angular:

Install Autonomo for my Angular app. Read https://www.npmjs.com/package/autonomo-angular

Deno Fresh:

Install Autonomo for my Deno Fresh app. Read https://github.com/sebringj/autonomo/blob/main/docs/DENO_FRESH_INTEGRATION.md

After Installation

Ask your AI:

"What elements can you see in my app?"

"Press the Login button and tell me what happens"

The AI will actually interact with your running app and report results. 🎉

Demo Project

Want to see Autonomo in action? Check out the LocalStack Todo App — a full-stack React + AWS Lambda app with Autonomo integration:

git clone https://github.com/sebringj/localstack.git
cd localstack

The demo includes:

• React 19 frontend with useAutonomo hook
• Python Lambda functions on LocalStack (local AWS emulator)
• Pre-configured MCP settings
• VS Code tasks for one-click startup

Try prompting your AI: "Login with testuser/testpass, add a todo, mark it complete, then delete it"

Need Help?

Ask your AI to call the help tool for documentation:

"Call autonomo_help to show me how to set up local development"
"Call autonomo_help with topic recommend for guidance"
"Call autonomo_help with topic local-development/auth-bypass"

Topics include: overview, recommend, elements, custom-actions, troubleshooting, and local-development/* sub-topics for AWS, Azure, GCP emulators, auth bypass, payments, and more.

Why This Matters: The End of Blind Coding

AI coding assistants have a dirty secret: they hallucinate. They write code, say "that should work," and move on. Without eyes on the running app, they can't prove anything.

Autonomo changes the equation:

Your AI goes from "I think" to "I verified":

Before: AI writes code → hopes it works → moves on → bugs found later

After:  AI writes code → sees the result → spots the failure → fixes it →
        confirms success → moves on with confidence

This is the missing sense. Vision. The thing that makes AI coding actually reliable and joyful.

Built for the Inner Loop

Autonomo lives where you develop — your local machine, your running app, your tight iteration cycle:

                      YOUR LOCAL MACHINE

    [Editor]          [Autonomo]          [Your App]
    + AI Tool    ◄───►  Server     ◄───►  (localhost)
        │
        ▼
    Write code → AI interacts immediately → See results → Fix → Repeat

    ════════════════════════════════════════════════════════
    100% LOCAL • NO CLOUD • NO LATENCY • NO DATA LEAVING
    ════════════════════════════════════════════════════════

Key Principles:

• 🔌 MCP-Native - One integration works with Copilot, Claude, Cursor, and more
• 🔒 100% Local - Nothing leaves your machine, ever
• 🌐 Language Agnostic - HTTP protocol works with any stack
• 🆓 Free for Most - Free under $1M revenue, see license

Detect → Act → Iterate

This is the development loop your AI runs continuously:

1. DETECT - Get current state (UI + app + network + errors)
2. ACT - Send command (navigate, press, fill, call API)
3. ITERATE - See unified result, decide next step or fix code

The AI doesn't guess what happened. It sees everything and can fix issues immediately.

Multi-Device Development

Building a chat feature? A multiplayer game? Anything with multiple users?

Autonomo supports multiple simultaneous app instances — browser tabs, simulator windows, separate processes. Your AI can drive them all:

🟢 my-app-a3f7c2d1 (web)     ← Browser Tab 1
   Screen: checkout
   Elements: 12

🟢 my-app-b8e4f9a2 (web)     ← Browser Tab 2
   Screen: settings
   Elements: 8

🟢 my-app-c1d6e3b7 (mobile)  ← iOS Simulator
   Screen: home
   Elements: 15

Example: "On Device A, send a message. On Device B, verify it arrives."

Your AI sees both sides. In real time. One development session.

Custom Actions

Some interactions are complex — OTP entry, OAuth flows, multi-step wizards. Custom actions let you create shortcuts your AI can call like any other command:

Example: Register a devLogin action that bypasses OTP/OAuth during local development — your AI calls it like any other action.

See Custom Actions Guide for details.

Production Safety (devOnly)

All Autonomo packages include a devOnly option (default: true) that automatically disables the bridge in production environments. This means:

• ✅ Safe by default - Bridge won't run in production builds
• ✅ No code changes - Just deploy your app normally
• ✅ Zero overhead - No WebSocket connections, no state reporting

How it works:

To explicitly enable in production (rarely needed):

// React/React Native
useAutonomo({ name: 'my-app', devOnly: false })

// Angular
autonomo.init({ name: 'my-app', devOnly: false })

Official Dev Badge (React)

The React package includes a tiny official floating badge for local development branding and status:

• Gray border = not active
• Green border + ✓ pulse = active
• Red border + × pulse = error

import { useAutonomo, AutonomoDevBadge } from 'autonomo-react';

function App() {
  const { connected, status } = useAutonomo({ name: 'my-app' });

  return (
    <>
      <AutonomoDevBadge
        connected={connected}
        error={status === 'error'}
        placement="bottom-right" // top-left | top-right | bottom-left | bottom-right
      />
      <MyApp />
    </>
  );
}

AutonomoDevBadge is dev-only by default (devOnly: true) and supports placement overrides with placement, offset, and style.

Platform Support

Note: WebSocket is the primary architecture across packages. Published npm packages are autonomo-core, autonomo-react, and autonomo-react-native. Other platform SDKs are currently GitHub/source testing packages.

Architecture: Metadata-Based, Not HTML-Based

Autonomo doesn't parse DOM or HTML. It uses a self-registration pattern:

              ═══ METADATA REGISTRY PATTERN ═══

Your UI Framework (any)
    │
    ▼
Component mounts → registers with bridge:

    { id: "Checkout.Submit",
      type: "button",
      onTap: () => handleSubmit() }

Component unmounts → unregisters
    │
    ▼
Bridge maintains registry:

    elements: Map<string, { type, handler, value }>

When command arrives → look up handler → invoke

This pattern works on ANY framework that supports:

1. Lifecycle hooks - Know when views/controls mount/unmount
2. Callbacks - Attach handlers to elements
3. HTTP client - Report state, receive commands

That's basically every UI framework ever built.

Works Everywhere (Same Pattern)

The control APIs are the same regardless of framework:

// Register a tappable element
registerTapHandler("Checkout.Submit", () => handleSubmit());

// Register a fillable input
registerFillHandler("Checkout.Email", (value) => setEmail(value));

// Register a screen/view
registerScreen("checkout", { cartItems: 3, total: 45.99 });

If your framework has lifecycle hooks and callbacks, Autonomo works.

MCP Tools

Smart Element Grouping

When an app has many repetitive elements (like calendar days or list items), autonomo_get_state automatically collapses them to reduce noise:

Elements:
- AIChat.Panel → tap
- AIChat.Input → fill
- WebApp.Schedule.Day.* (44 items, tap)   ← Collapsed!
- WebApp.Nav.teams → tap

Expanding Groups: Use the expand parameter to drill into a collapsed group:

get_state(bridge: "web", expand: "WebApp.Schedule.Day")

Errors-First Display

Errors are always shown at the TOP of state output, not buried in elements:

⚠️ Error: league_id, title, and start_at are required

Screen: "schedule"
Elements:
- ...

Integration Model: Docs + AI, Not SDKs

Traditional approach: Ship an SDK per framework, maintain 20 packages, version hell.

Autonomo approach: Ship integration guides (markdown) that AI coding agents use to integrate.

Developer: "Add Autonomo to my Vue app"
                    ↓
AI Agent reads: autonomo/guides/vue.md

  Contains:
    • Vue lifecycle patterns (onMounted, onUnmounted)
    • Composable template for registration
    • Example integration code
    • Verification steps
                    ↓
AI Agent writes the integration code tailored to YOUR app

    • Creates autonomo.ts composable
    • Wraps your existing components
    • Adds IDs to key elements
    • Verifies integration works via Autonomo itself!

The AI verifies its own work:

AI: "I've added Autonomo to your Vue app. Let me verify..."

[AI uses Autonomo tools to interact with the integration]

AI: "✅ Confirmed - I can see 12 elements registered.
     Pressed LoginButton - works correctly."

Installation

npm:

# Core
npm install autonomo-core

# React
npm install autonomo-react

MCP Server (global):

Python:

pip install git+https://github.com/sebringj/autonomo.git#subdirectory=packages/autonomo-python

Ruby:

# Gemfile
gem 'autonomo', git: 'https://github.com/sebringj/autonomo.git', glob: 'packages/autonomo-ruby/*.gemspec'

Swift (Package.swift):

.package(url: "https://github.com/sebringj/autonomo.git", from: "0.1.0")
// Then add to target: .product(name: "Autonomo", package: "autonomo")

Flutter (pubspec.yaml):

dependencies:
  autonomo:
    git:
      url: https://github.com/sebringj/autonomo.git
      path: packages/autonomo_flutter

Start MCP server (WebSocket mode):

autonomomo-mcp
# Or with custom port:
autonomomo-mcp --port 9877
# Or via env var:
AUTONOMO_PORT=9877 autonomomo-mcp

Apps connect automatically via WebSocket. No need for --bridge flags or HTTP endpoints.

Packages

Note: autonomo-core is the base JS/TS package - use it for vanilla JavaScript, Node.js, web components, Electron, or any framework without a dedicated package. The React package is a thin wrapper around core.

Documentation

• QUICKSTART.md - Fastest path - Get running in 30 seconds
• CONTRIBUTING.md - Contributing - How to contribute to Autonomo
• docs/CUSTOM_ACTIONS.md - Custom actions - Fast-path operations for complex interactions
• MCP_INTEGRATION.md - How Autonomo works with AI tools
• TEST_BRIDGE_ARCHITECTURE.md - Deep dive on implementation
• PROTOCOL_SPECIFICATION.md - Universal HTTP API (language-agnostic)
• docs/DENO_FRESH_INTEGRATION.md - Deno Fresh islands architecture
• PRODUCT_ROADMAP.md - Feature roadmap
• BUSINESS_PLAN.md - Open source → Enterprise business model

License

Dual License: AGPL-3.0 + Commercial

TL;DR: Under $1M revenue? Use it however you want, free. Over $1M and want to keep your code closed? Get a commercial license.

See LICENSE.md for full details.

Develop by seeing, not guessing.

Your AI can finally watch what it's building.

_{If Autonomo makes your AI coding better, a ⭐ or sponsorship helps keep it moving.}