GitHub - openqa-labs/openqa: Agentic Testing Harness

The open-source agentic testing harness. Write tests in plain English — the agent figures out the selectors.

Quick Start

Prerequisites — log in with Claude Code or opencode using your model provider (Anthropic, GitHub Copilot, Google Gemini, OpenAI, Amazon Bedrock, and more). No API key needed locally.

1. Scaffold the harness:

2. Write a feature file (.openqa/features/my-app.feature):

Feature: My App

  Scenario: User can log in
    * Navigate to "https://myapp.com"
    * Enter credentials and submit the login form
    * Should see the dashboard

3. Run:

No step definitions. No selectors. No code.

Features

• No selectors. Ever. — Agent navigates by intent. Survives any UI refactor automatically.
• CI-grade evidence — HTML report, trace viewer, and screenshots on every run.
• No API key locally — Uses your claude login or opencode auth login session.
• 2-minute setup — npx openqa init scaffolds the complete harness into your project.
• Dual-engine — opencode (70+ providers) or Claude Code SDK. Pick one.
• BDD & YAML — Playwright-BDD, Cucumber.js, or YAML.

Powered by: Claude Code SDK • opencode • Playwright MCP • Playwright-BDD • Cucumber.js

How It Works

1. Your BDD step definitions call runAgent(claudeCode('model'), 'natural language step', page).
2. OpenQA creates a Playwright MCP server in-process and exposes it over HTTP/SSE on a random localhost port.
3. The chosen AI provider SDK connects to that MCP URL and receives your natural language instruction.
4. The agent drives the real browser using Playwright MCP tools (browser_navigate, browser_click, etc.).
5. The step passes or fails based on what the agent reports back.

• True browser sharing — the agent drives the exact same page object your test holds.
• Parallel-safe — each test worker gets its own HTTP port. No shared config files.
• Session resumption — within a scenario, the agent resumes its conversation across steps.
• Multi-provider — swap claudeCode for openCode to use any model from OpenAI, Google, Anthropic, etc.

Environment Variables

The .openqa/ directory uses varlock for environment variable management. Variables are defined in .env.schema (committed to git) and values go in .env (gitignored). Secrets are automatically redacted from logs.

Adding your own variables — edit .openqa/.env.schema to declare them, then add values to .env:

# .openqa/.env.schema (add to the bottom)

# @sensitive=false
ENVIRONMENT = staging

# Your test account credentials for the staging environment
STAGING_USER =
# @sensitive
STAGING_PASSWORD =

Then use them in your steps or anywhere in the test process via process.env.ENVIRONMENT, etc.

Authentication

No API key needed for local development — just log in with the CLI once:

# Claude Code
claude login

# OpenCode (supports GitLab Duo, GitHub Copilot, Anthropic, OpenAI, Google, …)
opencode auth login

For CI (or if you prefer an API key), set the relevant key in .openqa/.env:

# Claude Code
ANTHROPIC_API_KEY=your_key

# OpenCode — use whichever provider you're connecting to
ANTHROPIC_API_KEY=your_key
# OPENAI_API_KEY=your_key
# GOOGLE_API_KEY=your_key

Customizing Your Setup

openqa init creates a working starting point — everything in .openqa/ is yours to edit. Common customizations:

Playwright config — .openqa/playwright.config.ts is a standard Playwright config. Add projects, change timeouts, add reporters, enable retries for CI:

// .openqa/playwright.config.ts
export default defineConfig({
  timeout: 120000,
  retries: process.env.CI ? 2 : 0,
  use: {
    baseURL: process.env.BASE_URL,
    locale: 'en-US',
    timezoneId: 'America/New_York',
  },
});

Step definitions — .openqa/steps/steps.ts is a regular Playwright-BDD or Cucumber.js step file. Add custom (non-AI) steps alongside the AI step, or add Before/After hooks:

// .openqa/steps/steps.ts — add a manual step alongside the AI one
import { createBdd } from 'playwright-bdd';
const { Given } = createBdd();

Given('I am on the home page', async ({ page }) => {
  await page.goto(process.env.BASE_URL!);
});

Writing Feature Files

openqa init places two example feature files in .openqa/features/ — todomvc.feature (2 scenarios) and getting-started.feature (1 scenario). Edit or replace them with your own.

Feature files use standard Gherkin syntax. We recommend using * (asterisk) for steps instead of Given/When/Then — it reads more naturally for AI-driven tests:

Feature: TodoMVC

  Scenario: Add a todo item
    * Navigate to "https://demo.playwright.dev/todomvc/"
    * Add a new todo item "Buy groceries"
    * Should see "Buy groceries" in the todo list

  Scenario: Filter completed todos
    * Navigate to "https://demo.playwright.dev/todomvc/"
    * Add three todo items: "Task 1", "Task 2", and "Task 3"
    * Mark the first todo as completed
    * Click the Active filter
    * Should see 2 active todos

You can still use Given/When/Then — both work identically.

Moving feature files elsewhere — if your feature files live outside .openqa/ (e.g. features/ in the project root), update the path in your config:

For Playwright-BDD, edit .openqa/playwright.config.ts:

const testDir = defineBddConfig({
  featuresRoot: '../features',
  features: '../features/**/*.feature',
  steps: 'steps/*.ts',
});

For Cucumber.js, edit .openqa/cucumber.js:

paths: ['../features/**/*.feature'],

Changing Model or Provider

After running openqa init, your model is set in one line inside .openqa/steps/steps.ts (or steps.js for Cucumber.js). Open that file and edit the provider call:

Change the Claude Code model:

// .openqa/steps/steps.ts
import { runAgent, claudeCode } from 'openqa';

// Before
await runAgent(claudeCode('claude-haiku-4-5'), action, page);

// After — switch to a more capable model
await runAgent(claudeCode('claude-sonnet-4-6'), action, page);

Switch from Claude Code to OpenCode (GitLab Duo, GitHub Copilot, etc.):

// .openqa/steps/steps.ts
import { runAgent, openCode } from 'openqa';  // swap the import

// GitLab Duo
await runAgent(openCode('gitlab/duo-chat-haiku-4-5'), action, page);

// GitHub Copilot
await runAgent(openCode('github-copilot/gpt-5.4'), action, page);

// Anthropic via OpenCode
await runAgent(openCode('anthropic/claude-sonnet-4-6'), action, page);

// OpenAI
await runAgent(openCode('openai/gpt-4o'), action, page);

// Google
await runAgent(openCode('google/gemini-2.0-flash'), action, page);

That's the only change needed — one import swap and one string update.

API Reference

runAgent(provider, prompt, pageOrContext, options?)

Runs the AI agent with a natural language instruction.

Returns: Promise<string> — the agent's final response.

claudeCode(model?)

import { claudeCode } from 'openqa';
const provider = claudeCode('claude-haiku-4-5'); // default

Requires @anthropic-ai/claude-agent-sdk to be installed.

openCode(model?)

import { openCode } from 'openqa';
const provider = openCode('gitlab/duo-chat-haiku-4-5'); // GitLab Duo (default in init)
// or: openCode('github-copilot/gpt-5.4')
// or: openCode('anthropic/claude-haiku-4-5'), openCode('openai/gpt-4o'), openCode('google/gemini-2.0-flash')

Model format: provider/model. Supports any provider configured in your OpenCode installation.

Requires @opencode-ai/sdk to be installed.

runAgent.resetSession(browserContext)

Resets the Claude Code conversation session for a specific browser context. Useful when you want to start a fresh conversation mid-test.

Examples

• examples/playwright-bdd/ — Playwright-BDD with natural language steps
• examples/playwright-yaml/ — YAML-based tests
• examples/cucumberjs/ — Cucumber.js integration

Requirements

• openqa library: Node.js 18+
• Scaffolded .openqa/ project: Node.js 22+ (required by varlock)
• @playwright/test ^1.57.0
• One of: @anthropic-ai/claude-agent-sdk (for claudeCode) or @opencode-ai/sdk (for openCode)

Links

• Website: https://openqa.io/
• NPM: https://www.npmjs.com/package/openqa
• GitHub: https://github.com/openqa-labs/openqa

License

MIT