Channelwise FP8 weights, dynamic per-token activations
llm-compressor 0.12.0, data-free: no calibration corpus is read at any point
← All optimized models
Qwen3.6 27B
Quantized and serving-tuned for H100 on vLLM 0.25.1, with the benchmark receipt included.
You get a self-contained kit: the optimized weights, the exact serving configuration, and the measured proof.
$40
one-time, yours forever
Get this modelCore plan required
$20/mo
We keep re-optimizing and re-benchmarking as engines move, and you always download the latest verified version. Cancel anytime. Nothing is charged until you confirm at checkout.
- Speed result
- 1.28x fasterMedian latency measured 2857 ms at baseline and 2215 ms optimized. Median latency (p50), concurrency 8.The full benchmark methodology and receipt ship inside the kit.
- Throughput
- 465tokens/s, concurrency 8
- Latency p50
- 2215ms
- Latency p99
- 2230ms
- Accuracy
- 99.87%gsm8k exact_match strict (N=1319, completion protocol) recoveryPassed accuracy gate
- GPU target
- H100
- Base model license
- Apache-2.0
- Proof verified
- Jul 25, 2026
- Published
- Jul 25, 2026
Request profile
chat-128
Latency p50lower is better↓ 22%
Baseline2857 ms
Optimized2215 ms
Latency p99lower is better↓ 23%
Baseline2881 ms
Optimized2230 ms
Throughputhigher is better↑ 29%
Baseline361 tokens/s
Optimized465 tokens/s
Runs on any GPU provider
Docker Compose
Kubernetes
RunPod
Modal
Your own hardware
This is what it means to own your AI
You control the intelligence
Run the model on your infrastructure, without deprecations, silent swaps, or provider restrictions.
You keep the economics
Buy once instead of paying per token; your deployment cost follows the GPUs you run.
You keep the data
Prompts and outputs stay on your network, with nothing retained elsewhere.
Technical implementation
Delivery contract
What is in the kit
Download
./models/kit.zipArchive root
qwen3-6-27b-fp8cd-v3-mlponly-h100-vllm/Model directory
qwen3-6-27b-fp8cd-v3-mlponly-h100-vllm/weights/Included
Kit contains
- Serving configuration
- Benchmark receipt
- Verifier
- Optimized weights
- Kit size
- 41.7 GB
Kit SHA-256
bcf523e6013b...871131ca2865Technical specifications
Will it run here?
- GPU target
- H100
- Compute capability
- NVIDIA Hopper (SM 90)
- Parameters
- 27B
What is it built from?
- Engine
- vLLM 0.25.1
- Quantization
- Channelwise FP8, measured selective-layer recipe
- Modality
- Text
- Base model license
- Apache-2.0
- Package version
- v5
How was it measured?
- Measurement basis
- Measured
- Request profile
- chat-128
- Concurrency
- 8
- Proof verified
- Jul 25, 2026
- Published
- Jul 25, 2026
Optimization techniques
Gated-DeltaNet path, mlp.down_proj and vision tower held at BF16; self-attention quantized
found by a measured per-layer sensitivity search on this architecture. The conventional assumption, which the model author's own FP8 build follows, protects self-attention and quantizes down_proj; measurement shows that is backwards here
Pinned engine and measured serving configuration
the exact engine build the receipt numbers were produced on; speculative decoding is off because it measured slower at production concurrency
Measured accuracy
gsm8k exact_match strict
(N=1319, completion protocol)
Passed accuracy gate
- Baseline
- 0.5754
- Optimized
- 0.5747
- Recovery
- 99.87%
- Delta
- -0.12%
Measured difference, significance not published
The optimized score measured below the baseline, at 99.87% of it. Neither score published a standard error, so this difference cannot be separated from sampling noise. The difference is not stated as a regression.
Methodology
- STATISTICALLY LOSSLESS:
- 0.5747 +/- 0.0136 against a 0.5754 +/- 0.0136 BF16 baseline is a 0.04 sigma difference, so quantization cost no measurable capability. lm-eval 0.4.12, vLLM backend, gsm8k full set N=1319, completion protocol, seed 1234, identical settings on both sides.
- For comparison
- on that identical protocol, the model author's own Qwen3.6-27B-FP8 scores 0.4268 (74.2 percent recovery): the naive FP8 conversion of this architecture loses about a quarter of its reasoning, and this build does not, because a measured per-layer search found mlp.down_proj and the Gated-DeltaNet path to be the sensitive layers while self-attention quantizes safely.
License
Two licenses apply: the RunInfra package license you purchase under, and the base model's own open-source license.
Package license
RunInfra Package License v1.1 (2026-07-25)
Buy once. No meter, no expiry, nothing calling home. The full terms ship inside the kit.
What you are licensed to do
This package is licensed to the purchasing workspace, one time, for the exact version purchased. Everyone in that workspace may run it in production without limits: unlimited inference, on any hardware or cloud the workspace controls, for any lawful commercial purpose. You may modify the configuration and tooling for your own use.
Your outputs are yours
Everything the model produces for you belongs to you. You may use, sell, and build products on the model's outputs without restriction or royalty. Serving the model to your own customers as part of your product is use, not redistribution, and is fully allowed.
What you may not do
You may not redistribute, resell, sublicense, rent, publish, or otherwise make the package or its artifacts available to any third party. That covers the optimized weights, serving configuration, scripts, kit archive, and benchmark receipts, whole or in part, modified or not. The license belongs to the purchasing workspace and cannot be transferred separately from it.
The base model keeps its own license
The underlying model remains under its upstream open-source license, which is included in this kit with attribution and a statement of RunInfra's modifications. Nothing in this license restricts rights the upstream license grants you for the ORIGINAL model; the restrictions above apply to RunInfra's optimized package.
Version-pinned, as measured
You purchased this exact version, proven against the exact engine version named in the receipt. It stays downloadable to your workspace and never expires. The benchmark receipt describes measurements taken at verification time on the named hardware; RunInfra does not promise future updates to this version, and later package versions are separate purchases.
Breach ends the license
If the workspace redistributes the package or its artifacts, this license terminates for that workspace. Sections about your outputs survive termination for outputs already produced.
Base model license
Apache-2.0, as published by the model author on Hugging Face. The full license text ships inside the kit.
Company
Who you are buying from
RunInfra is a sub-product of RightNow Research Lab.
Y CombinatorBacked by Y Combinator.
NVIDIA InceptionMember of NVIDIA Inception.
If you need custom optimization for a specific model, describe what you need
End-to-end encryption
Isolated GPU infrastructure
No training on your data
SOC 2 Type II
RunInfraby RightNow© 2026 RunInfra. All rights reserved.
Ask AI about RunInfra
