Models Engineering Index

2 min read Original article ↗

Mei—or, the Models Engineering Index—calculates language models' practical performance on engineering tasks. Mei calculates language models' engineering performance.

Each model's score is a composite index across 14 coding benchmarks, adjusted for benchmark difficulty and missing results so models average 50. Cost and token use are adjusted the same way and shown relative to the typical model (1×).

.

30405060704 Neon68.7Fable 5.267.26.2 Astra67.1Opus 5.561.96.2 Sol59.8Sonnet 5.559.64 Carbon58.44 Hydrogen55.14-Max52.0Wonder50.16.1 Terra49.7Wonder Full49.5K3.1 Pro49.4Ember46.9V4.1 Flash45.4Haiku 5.544.6Grok 4.743.2GLM 5.342.2Hy441.4Inkling 2 Small39.2V2.6-Pro39.16 Luna37.1M3.1 Flash36.8Large 435.5 8×4×2×1×0.5×0.25×0.125×0.0625×Ember0.044×6 Luna0.067×V4.1 Flash0.15×V2.6-Pro0.15×M3.1 Flash0.26×Hy40.4×6.2 Sol0.46×Inkling 2 Small0.54×Haiku 5.50.58×6.1 Terra1×Large 41×K3.1 Pro1.4×4 Hydrogen1.5×6.2 Astra1.9×4 Neon2.1×Wonder2.2×Wonder Full2.2×4 Carbon2.3×GLM 5.32.4×Sonnet 5.53.7×Opus 5.54.5×Grok 4.74.6×Fable 5.26.2×4-Max9.5× 2×0.5×6.2 Sol0.28×6.2 Astra0.38×6 Luna0.49×Ember0.52×K3.1 Pro0.66×Wonder Full0.7×Grok 4.70.76×4 Carbon0.77×4 Neon0.8×Wonder0.81×V2.6-Pro0.86×Large 40.93×4 Hydrogen0.93×Hy41.1×4-Max1.2×GLM 5.31.3×V4.1 Flash1.3×Fable 5.21.4×6.1 Terra1.6×Opus 5.51.6×Haiku 5.51.7×M3.1 Flash2.4×Inkling 2 Small2.8×Sonnet 5.53.2×

Performance versus .

30405060708×4×2×1×0.5×0.25×0.125×0.0625×4 NeonFable 5.26.2 AstraOpus 5.56.2 SolSonnet 5.54 Carbon4 Hydrogen4-MaxWonder6.1 TerraWonder FullK3.1 ProEmberV4.1 FlashHaiku 5.5Grok 4.7GLM 5.3Hy4Inkling 2 SmallV2.6-Pro6 LunaM3.1 FlashLarge 4 30405060702×0.5×4 NeonFable 5.26.2 AstraOpus 5.56.2 SolSonnet 5.54 Carbon4 Hydrogen4-MaxWonder6.1 TerraWonder FullK3.1 ProEmberV4.1 FlashHaiku 5.5Grok 4.7GLM 5.3Hy4Inkling 2 SmallV2.6-Pro6 LunaM3.1 FlashLarge 4
Help spread the word with a GitHub star (opens in a new tab)

Rankings

Index bar
14 Neon Google68.72.1×0.8×
2Fable 5.2 Anthropic67.26.2×1.4×
36.2 Astra OpenAI67.11.9×0.38×
4Opus 5.5 Anthropic61.94.5×1.6×
56.2 Sol OpenAI59.80.46×0.28×
6Sonnet 5.5 Anthropic59.63.7×3.2×
74 Carbon Google58.42.3×0.77×
84 Hydrogen Google55.11.5×0.93×
94-Max Alibaba52.09.5×1.2×
10Wonder Meta50.12.2×0.81×
116.1 Terra OpenAI49.71×1.6×
12Wonder Full Meta49.52.2×0.7×
13K3.1 Pro Moonshot49.41.4×0.66×
14Ember Meta46.90.044×0.52×
15V4.1 Flash DeepSeek45.40.15×1.3×
16Haiku 5.5 Anthropic44.60.58×1.7×
17Grok 4.7 xAI43.24.6×0.76×
18GLM 5.3 Z.ai42.22.4×1.3×
19Hy4 Tencent41.40.4×1.1×
20Inkling 2 Small Thinking Machines39.20.54×2.8×
21V2.6-Pro Xiaomi39.10.15×0.86×
226 Luna OpenAI37.10.067×0.49×
23M3.1 Flash MiniMax36.80.26×2.4×
24Large 4 Mistral35.51×0.93×

Comparing and

255075100CursorBenchFrontierCodeFrontierSWESWE-AtlasCode MigrationIOIProgramBenchTerminal-BenchVibe Code 1–100Vibe CodeTerminal-Bench-ScienceSWE-bench MultimodalOpenWorldEngineerHiL-Bench 74.471.879.874.961.978.474.286.474.285.572.080.069.674.873.074.1