LLM VRAM Calculator
Estimate the VRAM a local LLM needs: weights by quantization level, KV cache for your context length and batch size, and overhead, checked against common GPUs and unified memory.
Loading tool...
What is LLM VRAM Calculator?
The LLM VRAM Calculator estimates the GPU memory a decoder-only language model needs at inference. It separates the decoder weights (parameter count times bits per weight), the KV cache (2 x layers x KV heads x head dimension x context length x batch x bytes per KV element), and a configurable overhead reserve, then compares the total against common GPUs and unified memory.
Key Benefits
- Know whether a model fits before downloading tens of gigabytes
- See which part drives the footprint: weights, KV cache, or overhead
- GQA and MQA aware, so modern models are not overstated
- States the quantization overhead it assumes and the limits of the formula
Common Use Cases
- •Deciding whether a 70B model runs on a 48 GB card at 32k context
- •Sizing the context window for a fixed GPU memory budget
- •Comparing quantizations by their real memory cost, not just the label
- •Checking whether KV cache quantization is needed for long-context work
How to Estimate LLM VRAM
- Pick a model: Choose a preset to fill in layers, attention heads, KV heads and head dimension, or enter the architecture yourself.
- Choose the quantization: Select the weight format. Every option shows its bits per weight, including the block-scale overhead that GGUF formats pay.
- Set context and batch: Enter the context length you will actually serve and how many sequences run at once, then pick the KV cache dtype.
- Compare with your card: Select your GPU and read the fit verdict, the remaining margin, and any warning about sparse models or sliding windows.
Key Features
- Weights, KV cache and overhead split into separate numbers
- GQA and MQA KV head counts, not just attention heads
- FP16, FP8, Q8_0, Q4_K_M and Q4_0 with the bits per weight stated
- KV cache dtype from FP16 down to 4-bit
- Fit check against 8 GB to 128 GB cards and unified memory
- States what the estimate does not cover