Tokens to GPUs Calculator | Cedana

2 min read Original article ↗

» GPU count

1 trillion tokens in one month (30 days) on Llama 3.3 70B needs approximately 367 H100 GPUs. Confidence is moderate.

Base estimatePlausible rangeLog axis. Select a row to see its calculation.

One model copy needs 3 V100 GPUs (77 GB against 32 GB). Treat the V100 result as theoretical.

» Calculation

How the calculator gets the H100 number

Each step changes one quantity. Read from left to right.

  1. Start

    1 trillion

    tokens in one month (30 days)

  2. ÷ 2,592,000 seconds

    385,802

    tokens per second, average

  3. × 0.475 for the token mix

    183,256

    output-equivalent tokens per second

  4. ÷ 50% utilization

    366,512

    tokens per second of installed capacity

  5. ÷ 1,000 tok/s for each H100

    367

    H100 GPUs

gpus = tokens ÷ seconds × [output share + input share × input cost] ÷ utilization ÷ tok/s for each gpu

» Sensitivity

Which assumption moves the H100 result most

Each bar shows the GPU count when one assumption moves from its best case to its worst case. The other assumptions stay at the base value. The longest bar is the assumption to measure first.

H100 speed for this modelmeasurement uncertainty

244 at 1,500 tok/s

814 at 450 tok/s

Average utilizationyour operating choice

229 at 80%

611 at 30%

Cost of an input tokenmeasurement uncertainty

251 at 0.1 of output

482 at 0.5 of output

Measurement uncertaintyYour operating choiceVertical line: base, 367 H100 GPUs before rounding

» Per-GPU detail

» Model presets and evidence

Reference speed is tokens per second for one GPU, as low, base, and high. "Measured" values come from public benchmarks across loose and strict latency targets. "Estimated" values have no direct benchmark.

» Other assumptions

Values read on 2 October 2026.

» Limits of this model

  • The result is a planning estimate. Expect an error of 2x in each direction for measured presets, and more for estimated presets. Measure your model on your hardware before you buy.
  • Speed changes with the latency target. The low and high values of each preset come from strict and loose latency targets.
  • For large mixture-of-experts models, the B200 lead over the H100 is 4x to 21x. FP4 support and memory size cause this. The ratio is specific to each model.
  • A100 and V100 results for the large models are theoretical. One model copy needs more GPUs than one node contains.
  • The V100 value has no measured LLM serving data. It comes from hardware specifications.
  • The range combines independent errors as a root sum of squares in log space. It is not a statistical confidence interval.
  • The calculator does not include prompt caching, speculative decoding, reasoning-token overhead, failures, or regional duplication.