Serving Kimi K3 on vLLM was hard. Here is what we measured.

Status note, September 26, 2026: package sales are closed, and kimi-k3-b300x8-k3turbo is no longer sold. For hosted inference, see Model APIs. The July 2026 measurements and package details below are historical. Version 4 retired the earlier batch-profile switching rule.

On 2026-07-29, no released vLLM build could serve Kimi K3. We checked the installable options, then worked through several distinct failures before the model produced a first token on an 8x B300 node. The result was hard won, and the proof is publishable even though the implementation recipe is not.

That work was a catalog package. This post explains the result, the measurement protocol, the operating limits, what the buyer received, and what we still refuse to claim.

The package, as offered in July

Why this was hard

The failures were independent and surfaced at different stages of bring-up. Fixing one exposed another, and no public installable build provided a working starting point. That is the useful public fact: this was not a missing toggle or a one-line launch command.

We are not publishing the failure sequence, runtime internals, or launch settings. Those were the paid work product. We are publishing the measurements and limitations because buyers could judge the result before they bought it.

What we measured

Initial validation used repo-scale coding prompts of 33 to 36K tokens as reported by the server, 2048 output tokens per request, and 16 measured requests after 1 warmup. Every published result is scoped to text generation on a single 8x B300 node.

The headline comparison was a one-variable test on 2026-07-30. Both arms ran back to back in one measurement window on the same eight cards and the same vLLM installation. Each arm used 8 timed requests at concurrency 1, a 1024-token prompt target, 1024 requested output tokens, temperature 0.0, streaming, a fixed seed, and generation through the requested output target. All 8 requests completed successfully in each arm. The only changed variable was the selected operating profile.

The serving curve

The repo-scale serving curve used 32768 prompt tokens and 2048 requested output tokens per request, temperature 0.0, streaming, a fixed seed, and one warmup request per point. It measured concurrency 1 with 4 requests, concurrency 8 with 16 requests, and concurrency 32 with 64 requests. Every request completed successfully. Concurrency 128 is not characterized on this version.

• Concurrency 1: 11.13 output tokens per second per GPU, 89.04 aggregate, time to first token p50 5707.65 ms, time per output token p50 26.46 ms, end-to-end latency p50 22629.34 ms, 4 of 4 requests completed.
• Concurrency 8: 28.74 output tokens per second per GPU, 229.93 aggregate, time to first token p50 6900.96 ms, time per output token p50 99.59 ms, end-to-end latency p50 69479.02 ms, 16 of 16 requests completed.
• Concurrency 32: 34.10 output tokens per second per GPU, 272.77 aggregate, time to first token p50 114549.13 ms, time per output token p50 175.66 ms, end-to-end latency p50 212831.87 ms, 64 of 64 requests completed. Request rate was 0.133 requests per second.

The profile advantage changes with load. The interactive profile measured +83.07 percent per GPU at concurrency 1 and +20.25 percent at concurrency 8, then minus 9.46 percent at concurrency 32. At concurrency 32, the batch profile measured 37.66 output tokens per second per GPU and 301.28 aggregate, with time to first token p50 of 7.4 seconds, versus 34.10 per GPU, 272.77 aggregate, and 114.5 seconds for the interactive profile.

What we did not measure

Accuracy recovery was NOT measured. Output parity is by construction because the package changes no weight byte. We do not claim token-level identity across operating profiles, and we publish no cross-server accuracy score. A control between two identical server instances measured 78.12 percent top-1 agreement, so that method could not distinguish a server from itself and certifies nothing.

• We did not measure image input. Every number here is text only.
• We did not evaluate refusal, toxicity, or jailbreak behaviour. The weights are unmodified, so we do not expect a shift, but we did not check.
• We did not measure anywhere near the 1 million token context the model supports.
• Peak VRAM is not published. The measured hardware was 8x B300 with 288 GB per GPU and 2304 GB across the node.
• The 8-request single-stream sample supports a median, but it is not a strong tail sample. Its p95 and p99 sit between the two slowest requests.

Why it was offered as a package

With a catalog package, the work already ran. What buyers received was the finished artifact, the measured proof, and the verifier. Buyers deployed it on their own hardware, subject to the named prerequisite below, and the receipt states exactly what was measured.

The catalog answered one question: someone already solved this model on this hardware, could a buyer purchase the measured work product? K3 is a strong example because the difficulty was the independent failure chain and the validation burden, not one isolated setting.

What the package contained

• Deploy files for the measured 8x B300 target and the interactive and batch operating profiles, with the prerequisite below stated plainly.
• The exact runtime build record and measured serve command in the signed buyer receipt delivered after purchase.
• A benchmark receipt signed with Ed25519, so the numbers you were sold are the numbers in the artifact.
• A verifier that checks the bytes you downloaded against what we measured.
• Zero weight bytes. The kit pins an immutable upstream revision and your node pulls the model from its source. We do not store, mirror, or redistribute the weights.

The K3 serving recipe was 2.5 MB. The model behind it was about 1.56 TB across 96 shards. That asymmetry mattered: the artifact captured serving engineering rather than duplicating the weights.

We did not quantize this model. The upstream weights arrived untouched, byte for byte. Claiming otherwise would be false, so the package stated that no weight byte changed and tied the download to measured checksums.

The original enterprise tier

The July 2026 package was offered at $2,300, plus $200 per month for maintenance and version updates, and required a signed enterprise licence before purchase. The buyer named the licensed entity, the signer, and their authority to bind it, then signed four separate affirmations covering redistribution, resale, and traceability. The signature was recorded append-only with the exact terms text and its hash, and it was bound to the purchase inside the same database transaction that moved the money. An entitlement under those terms could not exist without a signature.

The upstream weights remain governed by the Kimi K3 License. The package licence governs the paid serving work product. There is no technical measure that stops a determined buyer from copying configuration and documentation, and we will not pretend otherwise. The original gate provided a specific, evidenced agreement rather than an implied one.

Where this goes

Frontier open models are getting larger faster than the tooling around them. K3 is 2.8 trillion parameters total with about 104 billion active per token, across 92 mixture-of-experts layers. Day-zero support in a released inference engine is now the exception, not the rule. The window between a model release and a measured production path is where the work lives.

We saw that window as a product: a verified artifact buyers owned and ran on their own hardware, rather than a consulting engagement or an ongoing hosted rental. Enterprises that could not send weights to a third party needed someone to have already done the difficult bring-up and measurement work.

Buyers receive subsequent package versions through the maintenance subscription. Each version remains tied to its own signed receipt, measurements, checksums, known limits, and deployment prerequisites rather than silently inheriting claims from an earlier run.