GitHub - jimmytoan/qa-agent: QA Agent is an AI-powered E2E testing platform for product, QA, and engineering teams. It lets you define browser tests in natural language, execute them with an LLM-driven browser agent, and inspect screenshots, GIFs, and run history when something fails.

4 min read Original article ↗

QA Agent

Run end-to-end browser tests using natural language.
No Selenium scripts. No brittle Playwright selectors. Just describe the test.

License FastAPI React 19 browser-use LangChain GitHub stars

QA Agent is an AI-powered E2E testing platform for product, QA, and engineering teams. It lets you define browser tests in natural language, execute them with an LLM-driven browser agent, and inspect screenshots, GIFs, and run history when something fails.

It supports Azure OpenAI, OpenAI, Anthropic Claude, and Google Gemini.

Why QA Agent?

Traditional E2E automation usually means:

  • writing automation scripts
  • maintaining brittle selectors
  • updating tests whenever the UI changes

QA Agent replaces that workflow with:

  • natural language test authoring
  • real browser execution
  • live streaming progress
  • screenshots + GIF artifacts
  • run reports and history

What a test looks like

Test: User can log in

Steps:
1. Go to https://example.com
2. Click the login button
3. Enter email test@example.com
4. Enter password
5. Submit the form

Evaluation:
The user dashboard is visible.

The agent opens a real browser, performs the flow, evaluates the result, and stores artifacts for review.

Key Features

Natural Language Test Authoring

Create tests with plain-language steps and evaluation criteria.

Real Browser Execution

Runs in a real browser using browser-use.

Real-Time Streaming

Watch each run live through the SSE event stream.

Products, Suites, and Tests

Organize test coverage by product and suite.

Artifacts

Every run can capture:

  • screenshots
  • step-by-step execution logs
  • GIF recordings

Run Reports and History

Track:

  • pass / fail status
  • execution details
  • error context
  • linked artifacts

Import / Export

Import suites from Excel and export suite definitions.

Multi-Provider LLM Support

Use the provider that fits your workflow:

  • Azure OpenAI
  • OpenAI
  • Anthropic Claude
  • Google Gemini

Example Run

Run Status: running

Step 1
Navigate to homepage

Step 2
Click login button

Step 3
Enter credentials

Step 4
Verify dashboard is visible

Result: PASSED

Artifacts generated:
- screenshots
- GIF recording
- step evaluation logs

Architecture

React + Vite frontend
        |
        | REST + SSE
        v
FastAPI backend
        |
        | run orchestration
        v
browser-use + LangChain chat model
        |
        v
real browser automation

PostgreSQL
run history + artifacts metadata

Supported Providers

QA Agent chooses the provider automatically from the selected model name:

  • gemini-* → Google Gemini via GOOGLE_API_KEY
  • claude-* → Anthropic via ANTHROPIC_API_KEY
  • everything else → Azure OpenAI if AZURE_OPENAI_API_KEY is set, otherwise OpenAI via OPENAI_API_KEY

Example models already supported in the UI:

  • OpenAI: gpt-5.4, gpt-5-mini, gpt-4.1, gpt-4o
  • Anthropic: claude-opus-4-6, claude-sonnet-4-6, claude-haiku-4-5-20251001
  • Gemini: gemini-3.1-pro-preview, gemini-2.5-pro, gemini-2.5-flash
  • Azure OpenAI: any deployment mapped through AZURE_OPENAI_DEPLOYMENT_NAME or matching the model name

See backend/.env.example for the full environment template.

Tech Stack

Backend

  • FastAPI
  • SQLAlchemy
  • Alembic
  • PostgreSQL

Agent Execution

  • browser-use
  • LangChain provider adapters
  • Azure OpenAI / OpenAI / Anthropic / Gemini

Frontend

  • React 19
  • Vite 6
  • TypeScript
  • Tailwind CSS
  • shadcn/ui

Infrastructure

  • Docker
  • pnpm workspaces
  • uv

Quick Start

1. Clone the repo

git clone https://github.com/jimmytoan/qa-agent.git
cd qa-agent

2. Install dependencies

Prerequisites:

  • Node.js 18+
  • pnpm
  • Python 3.12+
  • uv
  • PostgreSQL 15+

3. Configure environment

cp backend/.env.example backend/.env

At minimum, set:

DATABASE_URL=postgresql+psycopg://user:password@localhost:5432/qa_agent

# choose one or more providers
OPENAI_API_KEY=
AZURE_OPENAI_API_KEY=
AZURE_OPENAI_ENDPOINT=
ANTHROPIC_API_KEY=
GOOGLE_API_KEY=

4. Run database migrations

cd backend
uv run alembic upgrade head
cd ..

5. Start the app in development

Open:

In local mode, the Vite dev server proxies API and artifact requests to the backend.

Run with Docker

docker build -t qa-agent .

docker run --rm -p 8000:8000 \
  -e DATABASE_URL=postgresql+psycopg://user:password@host:5432/qa_agent \
  -e OPENAI_API_KEY=your_key_here \
  qa-agent

The container:

  • runs Alembic migrations on startup
  • serves the backend API
  • serves the built frontend on port 8000

API Examples

Create a browser run

POST /api/browser-use/runs

Response:

{
  "id": "...",
  "status": "queued"
}

Stream progress

GET /api/browser-use/runs/{run_id}/stream

Useful endpoints

  • GET /api/health
  • POST /api/tests/{test_id}/runs
  • POST /api/suites/{suite_id}/runs
  • GET /api/reports/runs/{run_id}
  • GET /api/suites/{suite_id}/export
  • POST /api/suites/import

Documentation

Roadmap

Planned improvements:

  • scheduled runs
  • Slack / Jira notifications
  • webhooks
  • API tokens and auth
  • object storage for artifacts
  • CI integrations
  • richer evaluation strategies

Why Open Source?

AI agents are changing how QA gets done.

We believe testing should be:

  • accessible to product teams
  • easier to debug
  • faster to write

Open sourcing QA Agent makes it easier for teams to experiment with AI-native testing workflows and adapt the stack to their own environments.

Contributing

PRs are welcome.

High-impact areas for contribution:

  • new evaluation strategies
  • additional model/provider support
  • CI and notification integrations
  • artifact storage backends
  • reporting UX improvements

License

This project is licensed under the GNU Affero General Public License v3.0. See LICENSE for details.

Support the Project

If QA Agent is useful to your team, give it a ⭐

It helps the project grow and makes the repo easier to discover.