GPU / VRAM Calculator for Local LLMs
Estimate VRAM for quantized weights plus KV cache, and see which RTX cards and Apple Silicon tiers fit your model.
It checks whether an AI model will fit in your computer’s graphics memory.
What this tool does
A local model needs memory for its weights and for the conversation it is processing. Model size, quantization, and context length all change that number. This calculator estimates the total and compares it with common hardware.
- Check a model before downloading a large file
- Choose a quantization that fits your GPU or Apple Silicon Mac
- See how longer context increases memory use
KV cache grows with context. Long chats and RAG can dominate VRAM on smaller cards.
KV factor: per 1k context at this model size. Simplified — real usage depends on architecture (GQA, MLA) and runtime.
| Component | Formula | GB |
|---|---|---|
| Weights (raw) | ||
| Loading overhead | ||
| KV cache | ||
| Total |
snapshot. Apple tiers assume ~72% of unified memory is usable for GPU.
| Device | Usable | Verdict | Headroom |
|---|---|---|---|
Q4_K_M is the usual quality/size sweet spot for local models. Offloading layers to CPU works but is slow. On Apple Silicon, unified memory bandwidth often matters more than the headline GB number. Partial GPU offload and smaller context are the first knobs when you are tight on VRAM.
Running Llama, Mistral, or Qwen locally means fitting quantized weights plus a growing KV cache into VRAM (or unified memory). This calculator uses transparent approximations — not vendor-specific profilers — so you can sanity-check before you download a 70B GGUF.
Pair it with the local vs API break-even calculator if you are deciding whether to buy hardware at all.