What can my machine run?

Q: How much memory does a model need to run locally?

The weights plus the KV cache for your context length. Weights are the parameter count times the bytes per parameter: about 2 at 16-bit, 1 at 8-bit and 0.55 at 4-bit including the quantisation tables, or the published checkpoint size where one exists. The KV cache grows with the context and the model's attention layout. The page adds 15% headroom before calling a fit comfortable.

Q: Why does unified memory count for less than its size?

The system, the display and the runtime take a share of a shared pool, and macOS caps how much the GPU may wire. The page keeps 28% back by default on unified memory and 8% on a discrete card; both are editable because the right number depends on what else the machine runs.

Q: Why are mixture-of-experts models faster than their size suggests?

Every parameter has to be in memory, but each token only reads the active experts. A model with 120B total and 5B active parameters may read fewer weights per token than a dense 120B model. Actual speed also depends on routing, attention, memory access and engine support; it should not be assumed to match a dense 5B model.

Updated 2 September 2026. Renting or buying instead? Cost calculator.

Pick your machine or type its memory and bandwidth. The table estimates memory fit, precision and single-user decode speed. Check your engine supports the exact model and format before downloading it. A model that does not fit shows the cheapest current rental rate from the price index.

Pick your machine

Your budget for weights

Machine

Usable for a model

Biggest that fits

Models it can run

Model	Weights¹	Size	Speed²

comfortable: at least 15% headroom tight: fits with almost no room left does not fit at 4-bit

¹ The highest precision that fits: 16-bit is bf16 or fp16 (MLX 16-bit), 8-bit is Q8_0 or MLX 8-bit, 4-bit is Q4_K_M or MLX 4-bit, in llama.cpp, Ollama, LM Studio or MLX. Models shipped in a low precision (gpt-oss in MXFP4, DeepSeek V4 Flash in fp8) show as shipped. ² Estimates: decode only, one user, bandwidth x efficiency divided by the bytes read per token; a mixture of experts reads only its active experts. Prefill, batching and thermal throttling are not modelled. No local speed on this page is a measurement.

size = weights + KV cache weights = parameters x bytes per parameter (4-bit 0.55, 8-bit 1.05, 16-bit 2.0), or the published checkpoint size KV cache = 50 KB x sqrt(active parameters in billions) x context tokens, or the model's own bytes per token where its config was read usable = memory x (1 - reserve); reserve 28% on unified memory, 8% on a graphics card fit = comfortable when size x 1.15 <= usable; tight when size <= usable; the highest precision that fits wins speed = bandwidth x efficiency / (active parameters x bytes per parameter)

Where the numbers come from

Machines: vendor specification pages for the Apple M5 family (Mac mini, Mac Studio, MacBook Pro), NVIDIA GeForce RTX 50 desktop and laptop GPUs, RTX PRO 6000 Blackwell and DGX Spark, and AMD Ryzen AI Max+ 395, as listed on 2 September 2026. Memory and bandwidth are the vendor's figures, not measurements; where bandwidth depends on the GPU tier, the memory chip carries the bandwidth for that configuration.
Models: the same list and sources as the cost calculator (model cards on Hugging Face and vendor documentation), plus the sub-10B Qwen3.5 and Gemma 4 sizes from their family cards. Both pages load one sizing module, so the same model at the same context shows the same gigabytes on both.
Speed: bandwidth x 72% by default, editable. The efficiency is the design's working figure; it has not yet been calibrated against a published benchmark table, which is why every speed is an estimate.
Prices: the LLM Hangar price index, latest check first, the 28 August 2026 snapshot when the index is unreachable. Your provider bills you at its own rates and none of this is a quotation.

Questions

How much memory does a model need to run locally?