e2e: The open source AI testing framework

TesterArmy

7 min read Original article ↗

Open Source AI
testing framework

Customizable TypeScript testing framework for web, mobile apps and more.

import { test, expect } from 'e2e';

test('a member upgrades to Pro', async ({ app, agent, screen }) => {

await app.open('/settings/billing');

await agent.act('upgrade the workspace to the Pro plan');

await agent.assert('the invoice preview shows a prorated amount');

await expect(screen.getByRole('status')).toContainText('Pro');

});

[01] Why e2e

Most AI tests start from scratch.
Every single time.

e2e remembers your agent's actions
and replays them when possible,
without calling the model again.

Less repeated reasoning.
Fewer tokens. Lower costs.

01/05

Connect on your terms.

Use your preferred gateway and keep control of how your AI connects.

import { gateway } from 'ai';

model: gateway('openai/gpt-6-luna-fast'),

02/05

Your favorite AI? Bring it along.

Choose the model that works for you. Your testing framework shouldn't decide for you.

import { gateway } from 'ai';

agents: {

default: {

model: gateway('openai/gpt-6-luna-fast'),

},

},

03/05

Your agent. Your rules.

Give your agent a purpose. Define how it should approach testing, what to look for, and what matters to you.

agents: {

default: {

system: 'You are a QA testing agent.',

},

},

04/05

Your app. Your environment.

Test on the web, go mobile, or bring your own engine. One framework, built to adapt.

import { web } from '@e2e-dev/web';

export default {

targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],

agents: {

default: {

model: gateway('openai/gpt-6-luna-fast'),

},

},

} satisfies E2EConfig;

05/05

Your stack. Ready to test.

Your model, your agent, your engine. All connected through one testing framework.

import type { E2EConfig } from 'e2e';

import { web } from '@e2e-dev/web';

import { gateway } from 'ai';

export default {

targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],

agents: {

default: {

model: gateway('openai/gpt-6-luna-fast'),

system: 'You are a QA testing agent.',

},

},

} satisfies E2EConfig;

[03] Core features

AI where it helps. Code where it counts.

Don't make AI think twice.

Replay compatible cached actions without unnecessary model calls.

Goals, not scripts.

Describe your testing goals in plain language. Let your agent handle the actions.

Exact when it matters.

Combine AI-driven actions with precise locators and deterministic assertions.

Your AI. Your choice.

Choose your preferred AI model and customize your testing agent.

One API. Multiple environments.

Test web and mobile applications through one shared API.

Bring your own engine.

Extend the framework with your own execution backend.

[04] Testimonials

Loved by developers.

[05] FAQ

Questions, answered.

The short version. When in doubt, the docs win.

Read the docs