Methodology

How each calculator turns inputs into a result, which constants are published facts and which are this site's assumptions, and how the numbers are checked. Last reviewed 2026-11-08.

Principles

LLM GPU memory

Formula. Total = weights + KV cache + 1.8 GB runtime overhead, in decimal gigabytes.

Published facts: parameter counts, layer counts, KV heads and head dimensions of the eight presets, from each model's configuration file on Hugging Face (checked 2026-11-08). Assumptions: 1.8 GB runtime overhead and the 92% usable-memory rule are planning values; actual overhead depends on the serving software. Not modelled: prefill activation peaks, memory pools, CUDA graphs, tensor-parallel duplication, and the extra storage of quantization scales.

PC power supply

Formula. Steady draw = GPU + CPU + other components. Peak estimate = (GPU × 2.0) + CPU + other components, rounded up to the next 100 W.

Published facts: GPU total graphics power and the manufacturer's stated minimum system power, quoted on the tool page with source and date. Assumption: the 2.0× GPU multiplier is a conservative margin chosen by this site, not a datasheet value. Not modelled: CPU spikes, multi-GPU rail balancing, efficiency curves and connector limits.

GPU server electricity cost

Formula. Annual cost = nodes × kW per node × PUE × 8,760 hours × price per kWh. Heat load = facility kW × 3,412.142 BTU/hr; refrigeration tons = BTU/hr ÷ 12,000.

Published facts: the unit conversions; reference tariff (U.S. Energy Information Administration) and reference PUE (Uptime Institute survey) on the tool page. Assumption: constant full load for the whole year. Not modelled: idle periods, demand charges, time-of-use tariffs, and colocation pricing per reserved kW.

Cloud versus on-premises cost

Formula. Monthly cloud cost = hourly rate × hours per day × 30.4. Breakeven months = hardware price ÷ (monthly cloud cost − monthly on-premises cost).

Published facts: none are built in; prices are inputs you supply from current quotes and price lists, because they change often. Not modelled: staff time, support contracts, egress, tax, depreciation, resale value, financing.

Jetson battery runtime

Formula. Energy (Wh) = mAh × volts ÷ 1,000. Usable energy = energy × 0.82. Runtime (minutes) = usable energy ÷ (motor watts + compute watts) × 60. Current = total watts ÷ volts; C-rate = current ÷ capacity in Ah.

Published facts: the Jetson Orin module power modes quoted on the tool page, from NVIDIA's documentation (checked 2026-11-08). Assumption: the 82% usable fraction is a reserve chosen by this site, not a battery rating. Not modelled: voltage sag, converter losses, temperature, battery ageing, and load variation across a mission.

How the numbers are checked

Our own measurements (Tesla T4, 15 November 2026 UTC)

To test the memory formula against a real runtime, we ran Qwen2.5 models on a Kaggle Tesla T4 (PyTorch 2.11.0+cu130, Transformers 5.20.2, bitsandbytes 0.50.2, CUDA 13.0). For each model and prompt length we loaded the weights, ran one forward pass over a prompt of that many tokens with caching enabled, and read PyTorch's allocator counters: memory after loading (weights), memory retained by the key-value cache after the pass, and the peak allocation during the pass. The prediction columns are the calculator formula with the model's own layer count and key/value dimension from its configuration. Values are decimal GB. The raw results file is published here. The run took place at 06:32 UTC on 15 November 2026 (the date_utc field in the file is in UTC); that is the evening of 14 November in US Pacific time, which is why the file name carries 2026-11-14.

Predicted / measured memory, one sequence, batch size 1, plain PyTorch, no serving software
ModelWeightsPrompt tokensWeights GB (pred / measured)KV cache GB (pred / measured)Peak allocated GB
Qwen2.5 0.5Bfp161,0240.99 / 0.990.013 / 0.0131.17
Qwen2.5 0.5Bfp164,0960.99 / 0.990.050 / 0.0503.32
Qwen2.5 0.5Bfp168,1920.99 / 0.990.101 / 0.10110.01
Qwen2.5 1.5Bfp161,0243.09 / 3.100.029 / 0.0293.29
Qwen2.5 1.5Bfp164,0963.09 / 3.100.117 / 0.1175.26
Qwen2.5 1.5Bfp168,1923.09 / 3.100.235 / 0.23511.18
Qwen2.5 3Bfp161,0246.17 / 6.190.038 / 0.0386.44
Qwen2.5 3Bfp164,0966.17 / 6.190.151 / 0.1519.03
Qwen2.5 3Bfp168,192out of memory on the 16 GB T4 during the full-prompt pass
Qwen2.5 1.5Bint41,0240.77 / 1.230.029 / 0.0291.42
Qwen2.5 1.5Bint44,0960.77 / 1.230.117 / 0.1173.38
Qwen2.5 1.5Bint48,1920.77 / 1.230.235 / 0.2359.30
Qwen2.5 7Bint41,0243.81 / 5.870.059 / 0.0596.29
Qwen2.5 7Bint44,0963.81 / 5.870.235 / 0.23510.77
Qwen2.5 7Bint48,192out of memory on the 16 GB T4 during the full-prompt pass

What matched. The key-value cache prediction agrees with the measurement to the rounding shown in every successful run, so the layer × KV-dimension × tokens formula is right for these models. The 16-bit weight prediction (parameters × 2 bytes) is within about 0.5% of what was measured.

What did not. Four-bit weights were larger than the nominal parameters × 0.5 bytes: 1.23 GB measured against 0.77 GB predicted for the 1.5B model and 5.87 GB against 3.81 GB for the 7B model. In this setup the embedding and output layers stay in 16-bit and the quantisation scales add overhead, so “4-bit” weights averaged well above 4 bits per parameter. Use a higher effective bits-per-weight when you size quantised models; the calculator lets you enter it.

The peak is not weights plus cache. During the full-prompt pass, the peak allocation was far above weights plus cache at long prompts (for example 10.0 GB for a 0.5B model at 8,192 tokens, whose weights and cache total 1.1 GB). In this plain PyTorch harness the model computes output scores for every prompt position over the whole vocabulary, which dominates at long prompts. Production serving software avoids this and manages memory differently, so the peak column is specific to this harness and should not be read as the requirement for production serving. It does show why a model that “should fit” can still run out of memory: the 3B model at 8,192 tokens and the 7B 4-bit model at 8,192 tokens ran out of memory on the 16 GB card.

Limits. One GPU type, one model family, one framework, one run per row; no throughput or latency was measured in this run. Treat these as a check on the memory arithmetic, not as benchmarks of speed.

A separate speed run. On 16 November 2026 (17:46 UTC) we ran a second, separate test on the same kind of Kaggle Tesla T4 to compare single-stream decode speed with the bandwidth ceiling. The method, table and limits are in the guide on predicting tokens per second; the raw file is published here. The same day we compared Transformers with vLLM on that card; see the inference servers guide and its raw results.

Sources used across the site

Corrections and updates

When a specification changes or a reader reports an error, the page is checked against its source and corrected, and the update date on the page is changed. Changes to formulas or constants are described here. Report a problem on the contact page. See also the editorial policy and about page.

Change log