Blog / Model guides
Kimi K3 hardware requirements: plan beyond the weight size
Kimi K3 is a cluster deployment decision. Its public checkpoint is about 1.56 TB before allowing for runtime memory and request state. The first question is whether the proposed machines can hold it; the next is whether the serving engine can use those machines efficiently.
Moonshot's model card describes 2.8 trillion total parameters, about 104 billion active per token, native MXFP4 weights and a 1,048,576-token context window. The active parameter count describes computation per token. It does not reduce storage requirements to those of a 104B model.
Use the actual checkpoint as the starting point
The published repository contains 96 weight shards and reports about 1.56 TB. Use the files at a fixed revision when budgeting disk and transfer size. Downloaded file size is useful evidence, but actual GPU allocations can differ with the engine and representation.
| Illustrative hardware layout | Nominal aggregate memory | Capacity conclusion |
|---|---|---|
| 8 × H200 at 141 GB | 1,128 GB | Below the released checkpoint's size |
| 16 × H200 at 141 GB | 2,256 GB | Above the file-size floor; runtime compatibility remains unproven by this arithmetic |
| 8 × B300 at 288 GB | 2,304 GB | Above the file-size floor; still needs a matching serving recipe |
Memory inputs come from NVIDIA's H200 specifications and its DGX B300 specifications. These totals do not include a measured cache budget or prove that a particular cloud offering is available.
Start from a supported topology
The vLLM Kimi K3 recipe specifies at least eight GB300 GPUs on its NVIDIA path and recommends multiple nodes for production traffic. It also documents an AMD path. Its NVIDIA image requires a compatible CUDA 13 driver environment.
A B300 server and a GB300 system should not be treated as interchangeable because their memory totals look similar. Confirm the exact GPU, host architecture, interconnect, driver and container image with the recipe and provider. At this scale, a networking mismatch can prevent startup or dominate inference time.
For a multi-node layout, verify how weights are read, how ranks communicate and how failures are detected. If each node downloads a complete copy, disk capacity and transfer work multiply. Shared storage changes that calculation but introduces its own bandwidth and availability requirements.
Budget context and concurrency separately
A million-token advertised context is a model capability, not a promise that a minimum-sized server can serve it at useful concurrency. Set a context target based on actual documents or coding sessions, then leave memory for simultaneous requests and runtime buffers.
Start the pilot with a shorter context and one request. Increase input length, output length and concurrency one at a time. Track time to first token, completed-task latency, successful tool calls and peak memory. Keep image-input tests separate from text-only tests so the resource difference is visible.
Price the whole cluster
Ask for the full node rate and confirm what it includes: CPU, RAM, interconnect, storage, network transfer and any minimum term. Multiplying a marketplace's lowest per-GPU price by eight can produce a configuration nobody actually sells.
For an illustrative cluster quoted at $64/hour, 176 serving hours cost $11,264; 720 hours cost $46,080. These are arithmetic examples, not Kimi hosting quotes. Add startup, evaluation, storage and operating effort. A four-hour unsuccessful setup attempt at that rate is already $256.
Before reserving capacity, compare Kimi with a smaller model on tasks where you expect the larger model to help. Measure accepted results and human correction time. A noticeable quality gain on your workload can justify the cluster; a benchmark score alone cannot establish that gain.
Read the license for your actual use
The Kimi K3 License has conditions beyond a standard permissive license. Section 2 requires a separate agreement when the licensee or an affiliate operates a Model-as-a-Service business and aggregate licensee-and-affiliate revenue exceeds $20 million over any consecutive 12 months. Section 3 requires UI attribution for covered products exceeding 100 million monthly active users or $20 million monthly revenue.
Section 4 exempts defined internal use and access through official products or certified inference partners from those two sections. Internal use excludes making the model, its outputs or its capabilities available to third parties. Have the responsible legal reviewer map the license's definitions to your service; owning the cloud account does not settle that question.
Questions people ask
Can Kimi K3 run on a single eight-H200 node?
The released checkpoint is about 1.56 TB, above the 1,128 GB nominal memory of eight 141 GB H200s. A different representation or offloading strategy would need its own supported recipe and performance evaluation.
Does enough aggregate VRAM prove that Kimi will run?
No. The quantization format, engine, drivers, topology, per-rank allocations and request memory must also be compatible. Treat a memory total as an initial filter.
Should I rent the cluster before evaluating model quality?
Where your data policy permits, evaluate representative tasks through an available service first. Use a cluster pilot to verify the exact self-hosted artifact, latency, cost and operational behavior before committing to capacity.