Discrete GPU★ Popular pick

NVIDIA RTX 4090

24 GB$1599~21.6 GB usable

Type: Discrete GPU
Provider: NVIDIA ★
VRAM: 24 GB
Price: $1599

Usable for models (est.): 21.6 GB · Price per GB: $67

Popular models that run here

The largest models this device holds fully in memory, each at its best-fitting quantization.

Every model: does it fit?

35 of 41 models run fully in memory, 1 more with partial offload. Estimated at a 4096-token context. How fit is calculated

Model	Params	Best fit	Memory needed (est.)	Fit
DeepSeek-V4-Pro (MoE)	1600B	—	~480.0 GB	✕ Does not fit
DeepSeek-V3 (MoE)	671B	—	~201.3 GB	✕ Does not fit
Llama 3.1 405B	405B	—	~121.5 GB	✕ Does not fit
DeepSeek-V4-Flash (MoE)	284B	—	~85.2 GB	✕ Does not fit
Qwen 3 235B-A22B (MoE)	235B	—	~70.5 GB	✕ Does not fit
Mistral Large 2 (MoE)	141B	—	~42.3 GB	◐ Partial offload
Qwen 2.5 72B	72B	INT2	~21.6 GB	✓ Runs
Llama 3 70B	70.6B	INT2	~21.2 GB	✓ Runs
Llama 3.3 70B	70.6B	INT2	~21.2 GB	✓ Runs
Llama 3.1 70B	70B	INT2	~21.0 GB	✓ Runs
Llama 2 70B	70B	INT2	~21.0 GB	✓ Runs
Falcon 40B	40B	INT2	~12.0 GB	✓ Runs
Qwen 2.5 32B	32B	FP4	~19.2 GB	✓ Runs
Qwen 3 32B	32B	FP4	~19.2 GB	✓ Runs
OLMo 3 32B	32B	FP4	~19.2 GB	✓ Runs
Gemma 4 31B	31B	FP4	~18.6 GB	✓ Runs
Qwen 3 30B-A3B (MoE)	30B	FP4	~18.0 GB	✓ Runs
Gemma 3 27B	27B	FP4	~16.2 GB	✓ Runs
Phi-4	14B	FP8	~16.8 GB	✓ Runs
Llama 2 13B	13B	FP8	~15.6 GB	✓ Runs
OLMo 2 13B	13B	FP8	~15.6 GB	✓ Runs
Mistral Nemo 12B	12B	FP8	~14.4 GB	✓ Runs
Gemma 3 12B	12B	FP8	~14.4 GB	✓ Runs
Gemma 2 9B	9B	FP16	~21.6 GB	✓ Runs
Llama 3 8B	8B	FP16	~19.2 GB	✓ Runs
Llama 3.1 8B	8B	FP16	~19.2 GB	✓ Runs
Qwen 3 8B	8B	FP16	~19.2 GB	✓ Runs
Llama 3.3 8B	8B	FP16	~19.2 GB	✓ Runs
Llama 2 7B	7B	FP16	~16.8 GB	✓ Runs
Mistral 7B	7B	FP16	~16.8 GB	✓ Runs
Qwen 2.5 7B	7B	FP16	~16.8 GB	✓ Runs
Falcon 7B	7B	FP16	~16.8 GB	✓ Runs
OLMo 2 7B	7B	FP16	~16.8 GB	✓ Runs
OLMo-7B	7B	FP16	~16.8 GB	✓ Runs
Mistral Small 3 7B	7B	FP16	~16.8 GB	✓ Runs
Gemma 3 4B	4B	FP32	~19.2 GB	✓ Runs
Phi-3	3.8B	FP32	~18.2 GB	✓ Runs
Phi-3-mini-128K	3.8B	FP32	~18.2 GB	✓ Runs
Llama 3.2 3B	3B	FP32	~14.4 GB	✓ Runs
Llama 3.2 1B	1B	FP32	~4.8 GB	✓ Runs
Gemma 3 1B	1B	FP32	~4.8 GB	✓ Runs

Measured throughput

Qwen 3 30B-A3B (MoE) Q4_K_XL195.78 tok/s
Qwen 3 8B Q4_K_M141 tok/s
Qwen 3 8B Q4_K_M141 tok/s
Qwen 3 8B Q4_K_M141 tok/s
Qwen 3 8B Q4_K_M141 tok/s
Qwen 2.5 7B Instruct Q4_K_M135 tok/s
Llama 3.1 8B Instruct Q4_K_M131 tok/s
Phi-4 Q4_K_M130 tok/s
Mistral 7B Q4_K_M110 tok/s
Mistral Small 3 7B Q4_K_M85 tok/s
OLMo 3 32B Q4_K_M32 tok/s
Llama 3.3 70B Q4_K_M25 tok/s
Falcon 40B Q4_K_M21 tok/s

Model	Quantization	Tokens/sec (measured)
Qwen 3 30B-A3B (MoE)	Q4_K_XL	195.78
Qwen 3 8B	Q4_K_M	141
Qwen 3 8B	Q4_K_M	141
Qwen 3 8B	Q4_K_M	141
Qwen 3 8B	Q4_K_M	141
Qwen 2.5 7B Instruct	Q4_K_M	135
Llama 3.1 8B Instruct	Q4_K_M	131
Phi-4	Q4_K_M	130
Mistral 7B	Q4_K_M	110
Mistral Small 3 7B	Q4_K_M	85
OLMo 3 32B	Q4_K_M	32
Llama 3.3 70B	Q4_K_M	25
Falcon 40B	Q4_K_M	21

Similar GPUs

Alternatives with similar memory

See what runs where on popular hardware