Start with a Hugging Face model, GPU, and real traffic shape. The result keeps fit, utilization, idle capacity, provider dates, and every caveat visible.

First

Check memory fit

Then

Price real utilization

Compare

API and GPU economics

Trust

Choose the better value

gemma-2-9b-it on NVIDIA A100 PCIe 40 GB

1 GPU, FP16, 50,000 requests/day. 500 input and 500 output tokens per request.

Estimated memory fit

Fits in memory, cost withheld.

No measured aggregate serving capacity matches this configuration. Memory fit and dated hourly rates remain available; monthly rent, replica counts, and API comparisons are withheld.

Memory required: 26.4 GB
Usable GPU memory: 36 GB

See Model APIs pricing

Edit inputs

Configure the workload

Change the inputs, then calculate a shareable result.

Hugging Face model

GPUGPU countRequests per dayQuantizationAverage input tokensAverage output tokens

Maximum served contextMaximum combined prompt and generated tokens accepted by the server. This is capacity, not average request length.

API comparison settings

A matching API price and serving cost are required before this assumption changes a comparison.

Model

google/gemma-2-9b-it

Hardware

1 x NVIDIA A100 PCIe 40 GB, FP16

Memory fit

Weights and KV cache stay separate so the fit is reproducible.

18.5 GB
Model weights: 4.4 GB
KV cache: 26.4 GB
Total required: 36 GB
Usable (90% of 40 GB)

8,192 tokens
Served context used: 3.4 GB
Runtime overhead: Parameter-only heuristic
KV cache method

Decode bandwidth ceiling

Single-stream physics only. Batched serving capacity is not established.

84 output tok/sCeiling

Ceiling, not an expected throughput result.

Decode and prefill ceilings derived from GPU memory bandwidth, dense compute, and model parameters, as of 2026-08-08

Provider price, NVIDIA A100 PCIe 40 GB

One dated GPU-hour rental price is available. It is not a monthly serving total because throughput evidence is missing.

Each bar compares the labelled value on the same scale.

Source:

Lambda Cloud, on demand, 1 month ago, captured 2026-08-08

What this chart means

These are GPU-hour rental prices only. A monthly serving total requires throughput evidence and remains absent.

vLLM deployment starting point

This line starts the selected vLLM shape. It is not a throughput reproduction claim.

vllm serve 'google/gemma-2-9b-it' --dtype float16 --tensor-parallel-size 1 --max-model-len 8192 --gpu-memory-utilization 0.9

Optimization evidence

Not measured on this config yet

No site-wide multiplier is applied.

Sources and caveats11 sources, 6 caveatsShow evidence

Provenance

Hugging Face model API revision sha
As of 2026-08-08
Hugging Face model API safetensors.total
As of 2026-08-08
Hugging Face config.json
As of 2026-08-08
NVIDIA A100 40 GB PCIe product specification
As of 2026-08-08
RunInfra Engine parameter-only KV-cache heuristic
As of 2026-08-08
RunInfra Engine feasibility runtime-overhead policy
As of 2026-08-09
Decode and prefill ceilings derived from GPU memory bandwidth, dense compute, and model parameters
As of 2026-08-08
vLLM serve CLI documentation
As of 2026-08-08
NVIDIA L40S product specification
As of 2026-08-08
NVIDIA A100 product specification
As of 2026-08-08
NVIDIA H100 Tensor Core GPU datasheet
As of 2026-08-08

Caveats

Served context is 8192 tokens. It is a serving-capacity decision and is not inferred from average input or output tokens.
Resolved layer count, KV-head count, or head dimension is absent. KV cache uses the parameter-only RunInfra Engine heuristic rather than an exact architecture calculation.
Runtime overhead adds 3.4 GB to the total memory requirement. This approximate 15% policy allowance covers CUDA context, activations, and vLLM overhead. Deployment-time profiling, including CUDA graph capture and allocator behavior, can differ.
Single-stream physics ceiling, not batched serving capacity. Batching changes aggregate throughput, so this concurrency-1 model cannot establish a minimum replica count or monthly rent. A measured serving benchmark matching the workload is required.
Compute prefill ceiling: 16,880 tokens/sec. Bandwidth decode ceiling: 84 tokens/sec.
The Hugging Face repository is gated. Running the generated command requires authorized access to the model weights.

Share this calculation

Embed code appears only when the calculator can publish a sourced cost for a fitting configuration.

Badge unavailable for this configuration

The calculator does not have a complete throughput source for this workload, so it withheld the cost and does not generate embed code.

Configuration details

Model

google/gemma-2-9b-it

GPU

NVIDIA A100 PCIe 40 GB

Requests/day

50,000

GPU count

Quantization

FP16

Tokens/request

500 in, 500 out

Charts use cited evidenceUtilization defaults to 30%

gemma-2-9b-it on NVIDIA A100 PCIe 40 GB

1 GPU, FP16, 50,000 requests/day. 500 input and 500 output tokens per request.

Estimated memory fit

Fits in memory, cost withheld.

No measured aggregate serving capacity matches this configuration. Memory fit and dated hourly rates remain available; monthly rent, replica counts, and API comparisons are withheld.

Memory required: 26.4 GB
Usable GPU memory: 36 GB

See Model APIs pricing

Edit inputs

Configure the workload

Change the inputs, then calculate a shareable result.

Hugging Face model

GPUGPU countRequests per dayQuantizationAverage input tokensAverage output tokens

Maximum served contextMaximum combined prompt and generated tokens accepted by the server. This is capacity, not average request length.

API comparison settings

A matching API price and serving cost are required before this assumption changes a comparison.

Model

google/gemma-2-9b-it

Hardware

1 x NVIDIA A100 PCIe 40 GB, FP16

Memory fit

Weights and KV cache stay separate so the fit is reproducible.

18.5 GB
Model weights: 4.4 GB
KV cache: 26.4 GB
Total required: 36 GB
Usable (90% of 40 GB)

8,192 tokens
Served context used: 3.4 GB
Runtime overhead: Parameter-only heuristic
KV cache method

Decode bandwidth ceiling

Single-stream physics only. Batched serving capacity is not established.

84 output tok/sCeiling

Ceiling, not an expected throughput result.

Decode and prefill ceilings derived from GPU memory bandwidth, dense compute, and model parameters, as of 2026-08-08

Provider price, NVIDIA A100 PCIe 40 GB

One dated GPU-hour rental price is available. It is not a monthly serving total because throughput evidence is missing.

Each bar compares the labelled value on the same scale.

Source:

Lambda Cloud, on demand, 1 month ago, captured 2026-08-08

What this chart means

These are GPU-hour rental prices only. A monthly serving total requires throughput evidence and remains absent.

vLLM deployment starting point

This line starts the selected vLLM shape. It is not a throughput reproduction claim.

vllm serve 'google/gemma-2-9b-it' --dtype float16 --tensor-parallel-size 1 --max-model-len 8192 --gpu-memory-utilization 0.9

Optimization evidence

Not measured on this config yet

No site-wide multiplier is applied.

Sources and caveats11 sources, 6 caveatsShow evidence

Provenance

Hugging Face model API revision sha
As of 2026-08-08
Hugging Face model API safetensors.total
As of 2026-08-08
Hugging Face config.json
As of 2026-08-08
NVIDIA A100 40 GB PCIe product specification
As of 2026-08-08
RunInfra Engine parameter-only KV-cache heuristic
As of 2026-08-08
RunInfra Engine feasibility runtime-overhead policy
As of 2026-08-09
Decode and prefill ceilings derived from GPU memory bandwidth, dense compute, and model parameters
As of 2026-08-08
vLLM serve CLI documentation
As of 2026-08-08
NVIDIA L40S product specification
As of 2026-08-08
NVIDIA A100 product specification
As of 2026-08-08
NVIDIA H100 Tensor Core GPU datasheet
As of 2026-08-08

Caveats

Served context is 8192 tokens. It is a serving-capacity decision and is not inferred from average input or output tokens.
Resolved layer count, KV-head count, or head dimension is absent. KV cache uses the parameter-only RunInfra Engine heuristic rather than an exact architecture calculation.
Runtime overhead adds 3.4 GB to the total memory requirement. This approximate 15% policy allowance covers CUDA context, activations, and vLLM overhead. Deployment-time profiling, including CUDA graph capture and allocator behavior, can differ.
Single-stream physics ceiling, not batched serving capacity. Batching changes aggregate throughput, so this concurrency-1 model cannot establish a minimum replica count or monthly rent. A measured serving benchmark matching the workload is required.
Compute prefill ceiling: 16,880 tokens/sec. Bandwidth decode ceiling: 84 tokens/sec.
The Hugging Face repository is gated. Running the generated command requires authorized access to the model weights.

Share this calculation

Embed code appears only when the calculator can publish a sourced cost for a fitting configuration.

Badge unavailable for this configuration

The calculator does not have a complete throughput source for this workload, so it withheld the cost and does not generate embed code.

Configuration details

Model

google/gemma-2-9b-it

GPU

NVIDIA A100 PCIe 40 GB

Requests/day

50,000

GPU count

Quantization

FP16

Tokens/request

500 in, 500 out

Charts use cited evidenceUtilization defaults to 30%