Serving Kimi K3 on vLLM was hard. Here is what we measured.

On 2026-07-29, no released vLLM build could serve Kimi K3. We checked the installable options, then worked through several distinct failures before the model produced a first token on an 8x B300 node. The result was hard won, and the proof is publishable even though the implementation recipe is not.

That work is now a catalog package. This post explains the result, the measurement protocol, the operating limits, what the buyer receives, and what we still refuse to claim.

The catalog

Why this was hard

The failures were independent and surfaced at different stages of bring-up. Fixing one exposed another, and no public installable build provided a working starting point. That is the useful public fact: this was not a missing toggle or a one-line launch command.

We are not publishing the failure sequence, runtime internals, or launch settings. Those are the paid work product. We are publishing the measurements and limitations because buyers should be able to judge the result before they buy it.

What we measured

Initial validation used repo-scale coding prompts of 33 to 36K tokens as reported by the server, 2048 output tokens per request, and 16 measured requests after 1 warmup. Every published result is scoped to text generation on a single 8x B300 node.

The headline comparison was a one-variable test on 2026-07-30. Both arms ran back to back in one measurement window on the same eight cards and the same vLLM installation. Each arm used 8 timed requests at concurrency 1, a 1024-token prompt target, 1024 requested output tokens, temperature 0.0, streaming, a fixed seed, and generation through the requested output target. All 8 requests completed successfully in each arm. The only changed variable was the selected operating profile.

The serving curve

The repo-scale serving curve used 32768 prompt tokens and 2048 requested output tokens per request, temperature 0.0, streaming, a fixed seed, and one warmup request per point. It measured concurrency 1 with 4 requests, concurrency 8 with 16 requests, and concurrency 32 with 64 requests. Every request completed successfully. Concurrency 128 is not characterized on this version.

• Concurrency 1: 11.13 output tokens per second per GPU, 89.04 aggregate, time to first token p50 5707.65 ms, time per output token p50 26.46 ms, end-to-end latency p50 22629.34 ms, 4 of 4 requests completed.
• Concurrency 8: 28.74 output tokens per second per GPU, 229.93 aggregate, time to first token p50 6900.96 ms, time per output token p50 99.59 ms, end-to-end latency p50 69479.02 ms, 16 of 16 requests completed.
• Concurrency 32: 34.10 output tokens per second per GPU, 272.77 aggregate, time to first token p50 114549.13 ms, time per output token p50 175.66 ms, end-to-end latency p50 212831.87 ms, 64 of 64 requests completed. Request rate was 0.133 requests per second.

The profile advantage changes with load. The interactive profile measured +83.07 percent per GPU at concurrency 1 and +20.25 percent at concurrency 8, then minus 9.46 percent at concurrency 32. At concurrency 32, the batch profile measured 37.66 output tokens per second per GPU and 301.28 aggregate, with time to first token p50 of 7.4 seconds, versus 34.10 per GPU, 272.77 aggregate, and 114.5 seconds for the interactive profile.

What we did not measure

Accuracy recovery was NOT measured. Output parity is by construction because the package changes no weight byte. We do not claim token-level identity across operating profiles, and we publish no cross-server accuracy score. A control between two identical server instances measured 78.12 percent top-1 agreement, so that method could not distinguish a server from itself and certifies nothing.

• We did not measure image input. Every number here is text only.
• We did not evaluate refusal, toxicity, or jailbreak behaviour. The weights are unmodified, so we do not expect a shift, but we did not check.
• We did not measure anywhere near the 1 million token context the model supports.
• Peak VRAM is not published. The measured hardware was 8x B300 with 288 GB per GPU and 2304 GB across the node.
• The 8-request single-stream sample supports a median, but it is not a strong tail sample. Its p95 and p99 sit between the two slowest requests.

Why a catalog and not just an agent

Our optimization agent takes your model and your workload and finds a faster way to serve it. That is a process. It runs, it measures, it costs GPU time, and the answer is specific to you.

A catalog package is the opposite shape. The process already ran. What you buy is the finished artifact, the measured proof, and the verifier. No optimization session is required. You deploy it on your own hardware, subject to the named prerequisite below, and the receipt states exactly what was measured.

Both exist because they answer different questions. The agent answers what should I do with my model. The catalog answers someone already solved this model on this hardware, can I buy the measured work product. K3 is a strong example because the difficulty was the independent failure chain and the validation burden, not one isolated setting.

What a package contains

• Deploy files for the measured 8x B300 target and the interactive and batch operating profiles, with the prerequisite below stated plainly.
• The exact runtime build record and measured serve command in the signed buyer receipt delivered after purchase.
• A benchmark receipt signed with Ed25519, so the numbers you were sold are the numbers in the artifact.
• A verifier that checks the bytes you downloaded against what we measured.
• Zero weight bytes. The kit pins an immutable upstream revision and your node pulls the model from its source. We do not store, mirror, or redistribute the weights.

The K3 kit is 2.5 MB. The model behind it is about 1.56 TB across 96 shards. The live package card above carries the current byte size, kit checksum, weight checksum, and $2,300 price directly from the package definition. That asymmetry is the point. You are buying the engineering, not a copy of the weights.

We do not quantize this model. The upstream weights arrive untouched, byte for byte. Claiming otherwise would be false, so the package states that no weight byte changes and ties the download to measured checksums.

The enterprise tier

This $2,300 package, plus $200 per month for maintenance and version updates, requires a signed enterprise licence before purchase. The buyer names the licensed entity, the signer, and their authority to bind it, then signs four separate affirmations covering redistribution, resale, and traceability. The signature is recorded append-only with the exact terms text and its hash, and it is bound to the purchase inside the same database transaction that moves the money. An entitlement cannot exist without a signature.

The upstream weights remain governed by the Kimi K3 License. The package licence governs the paid serving work product. There is no technical measure that stops a determined buyer from copying configuration and documentation, and we will not pretend otherwise. What the gate provides is a specific, evidenced agreement rather than an implied one.

Where this goes

Frontier open models are getting larger faster than the tooling around them. K3 is 2.8 trillion parameters total with about 104 billion active per token, across 92 mixture-of-experts layers. Day-zero support in a released inference engine is now the exception, not the rule. The window between a model release and a measured production path is where the work lives.

We think that window is a product. Not a consulting engagement, not a hosted endpoint you rent forever, but a verified artifact you own and run on your own hardware. Enterprises that cannot send weights to a third party still need someone to have already done the difficult bring-up and measurement work.

Buyers receive subsequent package versions through the maintenance subscription. Each version remains tied to its own signed receipt, measurements, checksums, known limits, and deployment prerequisites rather than silently inheriting claims from an earlier run.