LLM GPU Memory (VRAM) Calculator
Estimate how much GPU memory a model needs for inference or training, from its parameter count, numeric precision and optimizer. Useful for choosing a GPU, picking a quantization level, or checking whether a model will fit. Everything runs in your browser.
How is this calculated? Read the guide →
Activations, CUDA context and framework workspace. ~15–25% is typical.
Optional: KV cache (advanced)
Fill these to add the attention KV cache. Leave layers or hidden size at 0 to skip.
Optimizer states and the master copy are stored in fp32. Activation memory depends on batch and sequence length and is not included — add headroom.
Estimated total VRAM
—
These are estimates. Real usage depends on the framework, kernels, attention implementation, fragmentation and activations. Use the result for planning and leave headroom — don't size a GPU to the exact number.
How to use the VRAM calculator
- Enter the model size in billions of parameters (for example 7B, 13B, 70B).
- Choose a quantization level and a context length.
- Read the estimated VRAM, and compare it against your GPU's memory to see if it will fit.
Where the VRAM goes
Three things consume memory when you run a model locally:
- Weights — the model itself, the biggest chunk, and what quantization shrinks.
- The KV cache — grows with context length and can rival the weights for long prompts.
- Overhead — activations and framework buffers.
What quantization saves
Quantization stores the weights at lower precision. FP16 uses about 2 bytes per parameter, 8-bit halves that, and 4-bit halves it again, so a 4-bit model needs roughly a quarter of the memory of FP16 with only a modest quality trade-off. That is often the difference between a model fitting on your card or not. If it does not fit, some layers can spill to system RAM, but that slows generation considerably.
For the full breakdown and a sizing table by GPU, read how much VRAM you need to run an LLM locally.
Where you'd use this
Deciding whether a model will run on the hardware you have before downloading 40GB to find out, and choosing a quantisation level that fits.
For example: You have a 24GB card and want a 70B model. The calculator shows fp16 needs about 140GB — impossible — while 4-bit lands near 35GB, still too big, so the practical options are a smaller model or CPU offload.
Frequently asked questions
What does the VRAM calculator estimate?
The GPU memory needed to run or fine-tune a large language model, based on parameter count, quantisation (FP16, 8-bit, 4-bit) and overhead such as the KV cache.
Why does my model need more memory than its raw parameter count?
Beyond weights, inference needs memory for activations and the KV cache, and training adds optimiser states and gradients — often several times the weight size. The calculator accounts for these.
Is anything uploaded?
No. The estimate is computed in your browser.