All data sourced from the official Kimi K3 Technical Report (47 pages, July 2026)View PDF → · Full Model Specs →
2.8T parameters · 104B activated per token · 93 layers · MXFP4 weights / MXFP8 activations. All data from the official 47-page technical report.
~1,750 GB
Min Total VRAM
MXFP4 native (released format)
22× H100
Recommended GPUs
or 8× H100 with tensor+expert parallel
No GPU needed
API Alternative
platform.moonshot.cn
Why Does Kimi K3 Need So Much VRAM?
Kimi K3 uses a Mixture-of-Experts (MoE) architecture with 896 experts. During inference, only 16 experts are active per token — but all 896 experts must be loaded into VRAM simultaneously. This is different from dense models where only the active layers need to be in memory.
Important: K3 weights are stored in MXFP4 format (Microscaling Floating Point 4-bit). The model was trained with quantization-aware training from the SFT stage onward — meaning the 4-bit precision is native, not post-training compression. You cannot apply standard INT4 quantization to an already-4-bit model.
The math: 2.8 trillion parameters × bytes per param × overhead factor.
- MXFP4 Native (as released): 2.8T × 0.5 bytes × 1.25 (KDA + KV cache) = ~1,750 GB
- MXFP8 Activation: 2.8T × 1.0 bytes × 1.25 = ~3,500 GB
- FP16 (decompressed): 2.8T × 2.0 bytes × 1.25 = ~7,000 GB (theoretical — weights don't exist in FP16)
With expert parallelism (each GPU holds a subset of the 896 experts): 22 GPUs = each holds ~41 experts × 3.1B params × 0.5 bytes ≈ 64 GB weights + 16 GB overhead ≈ 80 GB per GPU. This fits in 22× H100 80GB. For 8× H100, combined tensor+expert parallelism is required.
Where the 1,750 GB Goes
896 Expert Weights
~1,400 GB
2.8T params × 0.5 bytes (MXFP4)
KDA KV Cache
~210 GB
1M context at 15% overhead
Expert Routing Tables
~70 GB
896→16 mapping + load balancing
Framework Overhead
~66 GB
CUDA contexts · NCCL buffers · activation memory
MoonViT-V2 Vision
~0.8 GB
401M params · text+image modality
How K3 Compares to Running Other Models
| Model | Parameters | Min GPU Setup | Min VRAM | Layers |
|---|---|---|---|---|
| ★ Kimi K3 | 2.8T MoE | 22× H100 80GB | ~1,750 GB | 93 |
| DeepSeek-V3 | 671B MoE | 4× A100 80GB | ~260 GB | 60 |
| Llama 4 Scout | 400B Dense | 8× H100 80GB | ~640 GB | 126 |
| Qwen3-235B | 235B Dense | 2× A100 80GB | ~120 GB | 94 |
| Claude Opus 4.8 | Unknown | API only | — | — |
| GPT-5.6 Sol | Unknown | API only | — | — |
K3 requires 7× more VRAM than DeepSeek-V3 and 15× more than Qwen3-235B, primarily due to the 896-expert MoE architecture (all experts must be loaded simultaneously) and the 1M token KDA KV cache. Hardware data from official sources and public benchmarks.
Network Requirements
GPU-to-GPU
≥ 400 GB/sNVLink / NVSwitch
Node-to-Node
≥ 200 GB/sInfiniBand NDR400
Recommended
8× H100/node × 3 nodes24 GPUs total
Consumer GPUs (RTX 4090 PCIe 5.0 ×16 = 64 GB/s) are 6× too slow for expert parallelism — even if VRAM were sufficient, communication would bottleneck.
Disk & Download
Model Size
~1.4 TB97 safetensors (MXFP4)
Download (1 Gbps)
~3.2 hours125 MB/s
Download (10 Gbps)
~20 minutes1.25 GB/s
hf-mirror.com
50-200 MB/s国内推荐
Free Disk Space
≥ 2 TB权重 + 临时解压 + KV cache
Storage Type
NVMe SSD避免 SATA (加载 30+ min)
How 22 GPUs Work Together — Expert Parallelism Layout
Each GPU holds ~41 experts. Per token, the router selects 16 experts — if the selected expert lives on GPU 5, the token data is sent to GPU 5 for computation, then returned. This routing happens for every token. The 2 shared experts (activated for every token) and KV cache fragments are replicated across all GPUs. GPU 22 holds the remaining spare experts and overhead, using only ~13 GB — which is why 21× H100 is the practical minimum, with the 22nd providing headroom.
Recommended GPU Configurations
| Quantization | Total VRAM | GPU Setup | Est. Cost | Notes |
|---|---|---|---|---|
| FP16 (uncompressed) | 7,000 GB | 44× H100 80GB or 88× A100 | $1.1M-1.4M | Theoretical. K3 weights were never stored in FP16 — decompressing MXFP4 to FP16 inflates 4×. Only relevant for training. |
| MXFP8 (activation) | 3,500 GB | 22× H100 80GB or 44× A100 | $550K-700K | MXFP8 activations with MXFP4 weights. The format used during K3 training and recommended for fine-tuning. |
| MXFP4 Native (as released) | 1,750 GB | 22× H100 80GB or 8× H100 (tensor+expert parallel) | $550K-700K | The released format. Weights were trained at 4-bit — no quality loss from compression. Requires expert parallelism across 22 GPUs or combined tensor+expert parallel on 8× H100. |
| Q2_K (post-training) | 1,225 GB | 16× H100 80GB or 31× A100 | $400K-500K | Further compression possible but may degrade MXFP4 quality gains. Use only if VRAM is severely constrained. Not recommended over native MXFP4. |
Cloud GPU Rental Options
Prices are approximate and subject to availability. Spot/preemptible instances offer significant discounts.
Cost per Token — Self-Hosted vs API
Estimates based on public cloud GPU pricing
| Option | Setup | Per 1M Tokens | Monthly (1B tokens) |
|---|---|---|---|
| Kimi API (official) | No GPU | $0.50 | $500 |
| ★ Self-host (22× H100) | 22× H100 80GB | ~$0.12 | ~$120 |
| Self-host (8× H100, TP+EP) | 8× H100 80GB | ~$0.15 | ~$150 |
| Claude Opus 4.8 API | No GPU | $15.00 | $15,000 |
| GPT-5.6 API | No GPU | $10.00 | $10,000 |
API prices from public documentation. Self-host estimates: 22× H100 @ $20/hr ÷ estimated 170 tokens/sec = ~$0.12/1M tokens. At scale (1B+ tokens/month), self-hosting K3 is approximately 100× cheaper than Claude or GPT API equivalents. Source: public cloud GPU pricing (RunPod, Lambda Labs), not from the K3 tech report.
📖
From the Official Technical Report
Every hardware number on this page — and 47 pages more — distilled into a practical guide. Decide if self-hosting K3 makes financial sense. Deploy with official vLLM/SGLang/TokenSpeed recipes. Understandwhy MXFP4 means you can't INT4-quantize this model.
📄 $1.99 · Instant PDF DownloadGumroad →
~2.5× scaling efficiency improvement over K2.5
Despite 3× the parameters, K3 achieves better performance per unit of compute through KDA attention, Stable LatentMoE routing, and MXFP4 quantization-aware training. Source: Technical Report §2.3.
What You Can Run on This Hardware
Source: Official Tech ReportPerformance data from the official Kimi K3 Technical Report. K3 (max) vs leading closed-source models.
| Benchmark | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol | Claude Opus 4.8 |
|---|---|---|---|---|
| GPQA Diamond | 93.5 | 92.6 | 94.1 | 91.0 |
| DeepSWE (Coding) | 67.5 | 70.0 | 73.0 | 59.0 |
| Terminal-Bench 2.1 | 88.3 | 88.0 | 88.8 | 84.6 |
| FrontierSWE | 81.2 | 86.6 | 71.3 | 66.7 |
| AA-LCR (Reasoning) | 74.7 | 70.0 | 73.7 | 67.7 |
| ProgramBench | 77.8 | 76.8 | 77.6 | 71.9 |
Official Deployment Recipes
Moonshot AI recommends these inference engines with tested configurations. API is OpenAI/Anthropic-compatible at platform.kimi.ai — select kimi-k3 model.
Recommended models are selected to fit RTX 5090 (32GB VRAM) and run normally on it. The JD.com link below may earn OMM a small commission at no extra cost to you.
🛒 JD.com — 华硕 RTX5090 夜神 32GB ☁️ Rent cloud GPU first →
⚠️ Above links point to RTX 5090 32GB. Hardware recommendations are for reference only. Verify specs before purchase. OMM is not responsible for purchase decisions.
Further Reading — All Based on the Official Technical Report
FAQ
What is the minimum GPU to run Kimi K3?
K3 uses native MXFP4 weights (~4 bits/param). With Q2_K further compression, minimum is ~16× H100 80GB (~1,225 GB total). For the released MXFP4 format: 22× H100 80GB or 8× H100 with combined tensor+expert parallelism. A single consumer GPU like RTX 4090 (24GB) cannot run even 1/50th of the model.
Can I run Kimi K3 on consumer GPUs like RTX 4090?
No. The model in MXFP4 native format needs ~1,750 GB total VRAM. Each GPU in an expert-parallel setup holds ~41 experts requiring ~80 GB — which exceeds a 4090's 24 GB by 3×. Even with aggressive Q2_K compression (~1,225 GB), a single 4090 can't run it. For consumer hardware, use the Kimi API instead.
How much does the hardware cost to run Kimi K3?
H100 80GB cards currently cost $25,000-30,000 each. A 22× H100 cluster runs $550,000-700,000. Cloud rental: ~$20-30/hour per H100 × 22 = $440-660/hour. For most users, the Kimi API (pay-per-token) is dramatically more cost-effective unless running inference 24/7.
What is MXFP4 and why does K3 use it?
MXFP4 (Microscaling Floating Point 4-bit) is a native 4-bit floating point format. Unlike post-training INT4 quantization (which takes an FP16 model and compresses it), MXFP4 means K3 was trained at 4-bit precision from the SFT stage — there is no quality loss from compression. It also means you cannot 'quantize to INT4' from MXFP4 — it's already at 4-bit. This is a newer approach that delivers better quality than post-training quantization at the same bit width.
💛 Support OpenModelMap
If this hardware guide helped you, consider supporting us. Every contribution keeps OMM free and independent.
⚠️ 免责声明 / Disclaimer
本文内容仅供参考，不构成任何购买建议。GPU 价格和云服务费用为估算值，实际价格因地区、供需关系和经销商而异。硬件需求基于 MoE 架构理论分析，实际 VRAM 使用取决于上下文长度、批次大小和框架优化。购买 GPU 或云服务前请根据自身需求评估。所列云服务商（Lambda Labs、RunPod、Vast.ai）与 OpenModelMap 无商业合作关系。OpenModelMap 不对因使用本文信息而产生的任何直接或间接损失承担责任。
This content is for informational purposes only and does not constitute purchasing advice. GPU prices and cloud costs are estimates — actual prices vary by region, availability, and vendor. Hardware requirements are based on MoE architecture analysis. Cloud providers listed are not affiliated with OpenModelMap. Always evaluate based on your own needs before purchasing.