Same model, different agent: 17 points apart on 21 MCP servers

The metrics has been designed keeping MCP server evaluation in mind, can you share numbers on how the models and harnesses perform on the final absolute answer

As a final outcome, most developers using the MCP servers still care only about the final result

i always thought model companies would make better harness ( atleast for their own model ) but here we are!?

btw im cofounder of sineframe… your agent driven mcp testing/CI tool. happy to answer any question