Best Self-Hosted LLM Leaderboard 2026 | Open-Weight Model Rankings for Enterprise

2 min read Original article ↗

Self-Hosted LLMs — 2026 Rankings

The definitive ranking of self-hostable LLMs for enterprise — compared across quality, speed, hardware requirements, and cost. Find the best open-weight model for your infrastructure.

Roshan Desai

Roshan Desai · Last updated: 2026-07-20

Best Self-Hosted LLMs by Task — Benchmark Rankings

Which self-hosted model is best for coding, reasoning, or agentic tasks? See how every open-weight model stacks up — hover any bar for details.

Best Advanced Knowledge

Advanced knowledge with harder 10-option format (MMLU-Pro)

Best in Graduate Reasoning

PhD-level science reasoning (GPQA Diamond)

Best at Instruction Following

Instruction following accuracy (IFEval)

Chatbot Arena Rankings

Crowdsourced Elo from human preference votes (LMArena)

Self-Hosted LLM Benchmark Scores & Hardware Requirements

Complete benchmark results, VRAM requirements, and licensing for every major self-hostable LLM. Click any column header to sort and rank.

Command A+

Cohere

218B

128K

Apache 2.0

109 GB

436 GB

N/A

76.1

N/A

N/A

N/A

N/A

N/A

N/A

N/A

DeepSeek R1

DeepSeek

671B

128K

MIT

351 GB

1340 GB

84.0

71.5

83.3

1398

49.2

90.2

65.9

87.5

97.3

DeepSeek V3.2

DeepSeek

685B

130K

N/A

351 GB

1367 GB

85.0

79.9

N/A

1423

67.8

N/A

74.1

89.3

N/A

DeepSeek-V4-Flash

DeepSeek

284B

1M

MIT

142 GB

568 GB

86.2

88.1

N/A

N/A

79.0

N/A

91.6

N/A

N/A

DeepSeek-V4-Pro

DeepSeek

1.6T

1M

MIT

800 GB

3200 GB

87.5

90.1

N/A

N/A

80.6

N/A

93.5

N/A

N/A

Devstral-2-123B

Mistral

123B

256K

Modified MIT

65 GB

246 GB

N/A

N/A

N/A

N/A

72.2

N/A

N/A

N/A

N/A

DS-R1-Distill-Llama-70B

DeepSeek

70B

128K

MIT

36 GB

140 GB

N/A

65.2

N/A

N/A

N/A

86.0

57.5

70.0

94.5

DS-R1-Distill-Qwen-14B

DeepSeek

14B

128K

MIT

8 GB

28 GB

N/A

59.1

N/A

N/A

N/A

N/A

53.1

N/A

93.9

DS-R1-Distill-Qwen-32B

DeepSeek

32B

128K

MIT

17 GB

64 GB

N/A

62.1

N/A

N/A

N/A

85.4

53.1

72.0

94.3

Gemma 3 12B

Google

12B

128K

Gemma License

8 GB

24 GB

60.0

40.9

N/A

1342

N/A

85.4

N/A

N/A

N/A

Gemma 3 27B

Google

27B

128K

Gemma License

14 GB

54 GB

67.5

42.4

N/A

1366

N/A

N/A

29.7

N/A

89.0

Gemma 4 12B

Google

12B

262K

Gemma

6 GB

24 GB

77.2

78.8

97.2

N/A

N/A

N/A

72.0

N/A

N/A

Gemma 4 31B

Google

31B

262K

Gemma

16 GB

62 GB

85.2

84.3

N/A

1451

N/A

N/A

80.0

N/A

N/A

GLM-4.7

Zhipu AI

355B

200K

MIT

180 GB

710 GB

84.3

85.7

88.0

1441

73.8

94.2

84.9

95.7

N/A

GLM-5.2

Zhipu AI

753B

1M

MIT

376 GB

1506 GB

N/A

91.2

N/A

1468

N/A

N/A

N/A

N/A

N/A

GPT-oss 120B

OpenAI

117B

128K

Apache 2.0

62 GB

234 GB

90.0

80.9

N/A

1355

62.4

88.3

60.0

97.9

N/A

GPT-oss 20B

OpenAI

20B

128K

Apache 2.0

11 GB

40 GB

85.3

71.5

N/A

1318

N/A

N/A

N/A

98.7

N/A

Hunyuan 2.0

Tencent

406B

256K

Tencent License

215 GB

812 GB

N/A

N/A

N/A

N/A

53.0

N/A

N/A

N/A

N/A

Hunyuan Hy3

Tencent

295B

262K

Apache 2.0

148 GB

590 GB

N/A

90.4

N/A

1412

78.0

N/A

N/A

N/A

N/A

Kimi K2.6

Moonshot

1T

262K

Modified MIT

500 GB

2000 GB

N/A

90.5

N/A

N/A

80.2

N/A

89.6

N/A

N/A

Ling-2.6-1T

Ant Group

1T

262K

MIT

500 GB

2000 GB

N/A

76.2

N/A

N/A

72.2

N/A

65.6

N/A

N/A

Llama 3.3 70B

Meta

70B

131K

Llama License

38 GB

140 GB

68.9

50.7

92.1

1319

N/A

88.4

N/A

N/A

77.0

Llama 4 Maverick

Meta

400B

1M

Llama License

206 GB

800 GB

80.5

69.8

N/A

1328

N/A

62.0

43.4

N/A

N/A

Llama 4 Scout

Meta

109B

10M

Llama License

58 GB

218 GB

74.3

58.2

N/A

1323

N/A

N/A

N/A

N/A

N/A

MiMo-V2-Flash

Xiaomi

309B

262K

MIT

159 GB

618 GB

84.9

83.7

N/A

1393

73.4

84.8

80.6

94.1

N/A

MiMo-V2.5-Pro

Xiaomi

1.02T

1M

MIT

510 GB

2040 GB

68.5

66.7

N/A

N/A

N/A

N/A

39.6

N/A

N/A

MiniMax M2.7

MiniMax

230B

205K

Non-commercial

115 GB

460 GB

N/A

N/A

N/A

1417

N/A

N/A

N/A

N/A

N/A

MiniMax M3

MiniMax

428B

1M

MiniMax Community

214 GB

856 GB

N/A

N/A

N/A

1445

80.5

N/A

N/A

N/A

N/A

Mistral Large 3

Mistral

675B

256K

Apache 2.0

355 GB

1350 GB

N/A

43.9

N/A

1416

N/A

92.0

82.8

88.0

93.6

Mistral Medium 3.5

Mistral

128B

256K

Modified MIT

64 GB

256 GB

N/A

74.8

N/A

1427

77.6

N/A

N/A

N/A

N/A

Mistral Small 3.1

Mistral

24B

131K

Apache 2.0

14 GB

48 GB

66.8

40.7

79.8

1304

N/A

87.2

N/A

N/A

N/A

Mistral Small 4

Mistral

119B

262K

Apache 2.0

60 GB

238 GB

N/A

71.2

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Nemotron 3 Nano

Nvidia

30B

1M

NVIDIA Open Model

15 GB

60 GB

78.3

73.0

N/A

N/A

38.8

N/A

68.3

89.1

N/A

Nemotron 3 Super

Nvidia

120B

1M

NVIDIA Open Model

60 GB

240 GB

83.7

79.2

N/A

N/A

60.5

N/A

81.2

90.2

N/A

Nemotron 3 Ultra

Nvidia

550B

1M

OpenMDW-1.1

275 GB

1100 GB

86.8

87.0

81.7

N/A

70.7

N/A

89.0

N/A

N/A

Nemotron Ultra 253B

Nvidia

253B

128K

Open Weight

135 GB

506 GB

N/A

76.0

89.5

1348

N/A

N/A

66.3

72.5

97.0

Phi-4

Microsoft

14B

16K

MIT

9 GB

28 GB

70.4

56.1

64.6

1256

N/A

82.6

N/A

N/A

80.4

Phi-4-mini

Microsoft

3.8B

131K

MIT

3 GB

8 GB

52.8

30.4

N/A

N/A

N/A

72.0

N/A

N/A

N/A

Qwen 3.5

Qwen

397B

262K

Apache 2.0

207 GB

794 GB

87.8

88.4

92.6

1450

76.4

N/A

83.6

N/A

N/A

Qwen3-Coder-Next

Qwen

80B

256K

Apache 2.0

42 GB

160 GB

78.4

53.4

89.1

N/A

70.6

94.1

74.5

89.2

83.5

Qwen3.5-4B

Qwen

4B

262K

Apache 2.0

2 GB

8 GB

79.1

76.2

N/A

N/A

N/A

N/A

N/A

N/A

N/A

Qwen3.5-9B

Qwen

9B

262K

Apache 2.0

5 GB

18 GB

82.5

81.7

N/A

N/A

N/A

N/A

65.6

N/A

N/A

Qwen3.6-27B

Qwen

27B

262K

Apache 2.0

14 GB

54 GB

86.2

87.8

N/A

N/A

77.2

N/A

83.9

N/A

N/A

Qwen3.6-35B-A3B

Qwen

35B

262K

Apache 2.0

18 GB

70 GB

85.2

86.0

N/A

N/A

73.4

N/A

80.4

N/A

N/A

Step-3.5-Flash

Stepfun

196B

262K

Apache 2.0

102 GB

392 GB

85.8

N/A

N/A

1389

74.4

81.1

86.4

99.8

N/A

Step-3.7-Flash

Stepfun

198B

262K

Apache 2.0

99 GB

396 GB

N/A

N/A

N/A

N/A

76.5

N/A

N/A

N/A

N/A

VRAM estimates are based on model weight size only: FP16 uses 2 bytes per parameter (e.g. 70B model = 140 GB), INT4 uses 0.5 bytes per parameter (e.g. 70B model = 35 GB). Actual usage is typically 10–20% higher due to KV cache, activations, and framework overhead. Tools like Ollama default to 4-bit quantization, so real-world usage is often closer to the INT4 figure.

Compare Self-Hosted LLMs Head-to-Head

Select two models to see how they stack up across all benchmarks.

Deploy These Models with Onyx

Onyx is the open-source AI platform that lets you self-host any of these LLMs and connect them to your team's docs, apps, and people.