MCPJam — MCP server testing, debugging & evals

· MCPJam

2 min read Original article ↗

From your first prompt to a continuous gate on every release, MCPJam shows what breaks across every AI client, and how to fix it.

Test in 16+ major AI clients

Inspector

Call a tool, read the trace, and compare how your product appears in different clients.

Swarms

Agent personas run them across every client. Every session is captured and scored.

Create a swarm of your users

Simulated users run through your server so you can see what breaks.

Choose the clients and servers your users will interact with. *

This will shape the swarm's behavior. You will be able to compare behavior across clients side by side.

ChatGPTClaudeCursor+ Add client

acme-payments· v2.15.0-rc.1 · 5 tools

Describe your users and their behavior. *

We build the user goals from your input. Your inputs are not final. You can edit personas and goals on the next screen.

Support agents refunding duplicate charges, finance ops reconciling Tuesday's payouts, and developers wiring a failed-payment webhook.

Generated personas

  • MCMaya Chen · Support agentFind the duplicate charge on order A-2291 and refund it.8 × 3 clients
  • JHJordan Hale · Finance opsMatch payout po_7Hd2 to its charges and flag anything missing.8 × 3 clients
  • PSPriya Shah · DeveloperCreate an invoice, fail its payment, and confirm the webhook fires.8 × 3 clients

User Testing

Share a sandboxed session. Measure user sentiment and uncover usability gaps.

Refund & reconcile · beta

Access settings

Anyone with the link (guests included)

Preview

Previewing as you, signed in. A tester opening this link gets the guest experience, which can differ.

Evaluate

Build durable testing and ensure reliability on every deployment.

Suite health

98%average across 12 runs

acme-paymentsPayments operations

100%95%

9/29/39/49/59/69/79/89/99/109/119/129/13

Runs

PlatformSuiteClientServer

CI/CD

Run the same suite on every pull request. A failing check blocks the merge.

acme/acme-paymentsPrivate

CodeIssuesPull requestsActions

list_charges: return full charge objects #482

Openm.okafor wants to merge 1 commit into release/2.15

ConversationCommits 1Checks 5Files changed

Some checks were not successful

1 failing and 4 successful checks

  • BuildSuccessfulRequired
  • Unit testsSuccessfulRequired
  • MCPJam / Protocol conformanceSuccessfulRequired
  • MCPJam / OAuth & securitySuccessfulRequired
  • MCPJam / Evals89% pass rate · 95% requiredRequired

Merging is blocked

MCPJam Evals must pass before this pull request can be merged.