Compare popular local LLM hardware
82 of 144 popular model × device pairs run fully in memory. Each cell shows the highest-precision quantization that fits; measured tokens/sec appear where the community has published a benchmark. Estimated at a 4096-token context. How fit is calculated
- ✓ Runs at the shown quantization
- ◐ Partial offload only
- ✕ Does not fit
| Model ↓ · Device → |  Intel Arc A770 8GB8 GB · $299 |  Qualcomm Snapdragon 8 Gen 3 Leading Version12 GB |  Qualcomm Snapdragon 8 Elite12 GB |  Mac Mini M416 GB · $599 |  MacBook Pro M4 14-inch16 GB · $1599 |  AMD Radeon RX 7800 XT16 GB · $599 | Intel Arc A770 16GB16 GB · $329 |  Qualcomm Snapdragon X Elite16 GB |  AMD Radeon RX 7900 XT20 GB · $899 | Mac Mini M4 Pro24 GB · $1399 |  NVIDIA RTX 409024 GB · $1599 |  AMD Radeon RX 7900 XTX24 GB · $999 | Mac Mini M4 32GB32 GB · $999 |  NVIDIA RTX 509032 GB · $1999 |  Mac Studio M5 Max36 GB · $2500 | MacBook Pro M4 Max 16-inch48 GB · $3999 |  NVIDIA RTX 6000 Ada48 GB · $6500 | Mac Studio M5 Ultra96 GB · $5500 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
 Mistral 7B7B · 8K ctx | ✓ FP4 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP16 | ✓ FP16 | ✓ FP16110 tok/s measured | ✓ FP16 | ✓ FP16 | ✓ FP16 | ✓ FP16 | ✓ FP32 | ✓ FP32 | ✓ FP32 |
 Llama 3.1 8B8B · 128K ctx | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP16131 tok/s measured | ✓ FP16 | ✓ FP16 | ✓ FP16101 tok/s measured | ✓ FP16 | ✓ FP16 | ✓ FP32 | ✓ FP32 |
 Gemma 2 9B9B · 8K ctx | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP8 | ✓ FP16 | ✓ FP16 | ✓ FP16 | ✓ FP1678 tok/s measured | ✓ FP16 | ✓ FP16 | ✓ FP32 | ✓ FP32 |
 Qwen 3.8 27B27B · 262K ctx | ✕ No fit | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP4 | ✓ FP8 | ✓ FP8 | ✓ FP16 |
 Llama 3.3 70B70.6B · 8K ctx | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✓ INT225 tok/s measured | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT2 | ✓ INT215 tok/s measured | ✓ FP4 | ✓ FP414 tok/s measured |
 Kimi K3104B · 1M ctx | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✓ INT2 | ✓ INT2 | ✓ FP4 |
 Llama 3.1 405B405B · 128K ctx | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit |
 DeepSeek-V3 (MoE)671B · 128K ctx | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit | ✕ No fit |
Fit verdicts are estimates; tokens/sec figures are measured and labeled as such. Quantization names like FP4 are explained in the quantization guide.