Local Model Guide
Guides

Quickstart: what runs where

See which popular models fit on popular hardware. Pick a provider, then read the grid: models run down the side and machines across the top.

How to read it

Models go from small (top) to large (bottom). Machines go from least memory (left) to most (right). A green cell means the model fits fully in that machine's memory; red means it does not. Where someone has benchmarked the pair, the cell shows measured tokens/sec instead.

Fit is estimated at a 4096-token context. Tokens/sec is measured, and it only appears where someone has benchmarked that model and machine. How we calculate this

All hardware →

Apple Silicon Macs with integrated GPU and neural engine. Great for on-device inference with low power consumption.

Estimated fit of popular models (rows) on popular Apple hardware (columns), with measured tokens/sec where available
Model ↓ · Machine →Mac Mini M4MacBook Pro M4 14-inchMac Mini M4 ProMac Mini M4 32GBMac Studio M5 MaxMacBook Pro M4 Max 16-inchMac Studio M5 Ultra
Llama 3.2 1B1B✓ FP32✓ FP32✓ FP32✓ FP32✓ FP32✓ FP32✓ FP32
Qwen 2.5 7B7B✓ FP8✓ FP8✓ FP16✓ FP16✓ 40 tok/s✓ 75 tok/s✓ FP32
Llama 3.3 8B8B✓ FP8✓ FP8✓ FP8✓ FP16✓ FP16✓ FP16✓ FP32
Qwen 3 30B-A3B (MoE)30B✓ INT2✓ INT2✓ FP4✓ FP4✓ FP4✓ FP8✓ FP16
Llama 3 70B70.6B✕ Too big✕ Too big✕ Too big✓ INT2✓ INT2✓ INT2✓ FP4
DeepSeek-V4-Pro (MoE)1600B✕ Too big✕ Too big✕ Too big✕ Too big✕ Too big✕ Too big✕ Too big

Tok/s is the fastest measured result for each pair. Hover a green cell to see its best-fitting quantization. Results vary with runtime and settings.

Want more options? See every device · Browse all models