DeepSeek · open weights

DeepSeek-V3 (MoE)

671B params128,000 ctx

Family

DeepSeek

Lab

DeepSeek

Parameters

671B

Max context

128,000 tokens

Quick answer: what should I run it on?

Estimated at a 4096-token context. New to quantization? Read the plain-words guide.

How much memory it needs

A quantization is the same model stored with fewer bits per number. Fewer bits use less memory but lose some output quality. Each card shows the estimated memory for DeepSeek-V3 (MoE), including KV-cache headroom at a 4096-token context.

• FP3232-bit

  ~3220.8 GB est.

  full FP32 size · full quality, largest size

  Full precision with 32 bits per number. Uses maximum memory but provides highest accuracy. A 7B model needs ~28GB RAM.

  Good for Training, research, or when maximum accuracy is critical. Requires high-end hardware like A100 GPUs or professional workstations.

  Runs on 0 of 18 popular devices

• FP1616-bit

  ~1610.4 GB est.

  50% of the FP32 size · full quality, largest size

  Half precision with 16 bits per number. Uses 50% less memory than FP32 with minimal quality loss. A 7B model needs ~14GB RAM.

  Good for Standard choice for GPU users. Works well on RTX 4090, H100, or other modern GPUs with good speed and reasonable memory requirements.

  Runs on 0 of 18 popular devices

• BFLOAT1616-bit

  ~1610.4 GB est.

  50% of the FP32 size · full quality, largest size

  Brain floating point with 16 bits per number. Uses 50% less memory than FP32, optimized for modern AI hardware. A 7B model needs ~14GB RAM.

  Good for Excellent for TPUs and newer GPUs (H100, RTX 6000 Ada). Better training stability than FP16 while maintaining small memory footprint.

  Runs on 0 of 18 popular devices

• FP88-bit

  ~805.2 GB est.

  25% of the FP32 size · near-full quality, half the size of FP16

  8-bit precision using ~25% of FP32 memory. Aggressive compression with good quality trade-off. A 7B model needs ~7GB RAM.

  Good for For GPU users with limited VRAM (RTX 3090, RTX 4060 Ti). Good quality maintained, but watch for slight accuracy reduction on specialized tasks.

  Runs on 0 of 18 popular devices

• INT88-bit

  ~805.2 GB est.

  25% of the FP32 size · near-full quality, half the size of FP16

  Integer-based 8-bit quantization using ~25% of FP32 memory. Stable and efficient. A 7B model needs ~7GB RAM.

  Good for CPU inference on servers, edge devices, or GPUs with space constraints. Slightly faster than FP8 for integer operations.

  Runs on 0 of 18 popular devices

• FP44-bit

  ~402.6 GB est.

  13% of the FP32 size · the usual sweet spot of size and quality

  Ultra-aggressive 4-bit compression using only ~12.5% of FP32 memory. May see minor quality loss. A 7B model needs ~3.5GB RAM.

  Good for MacBook Pro users or laptops with 8-16GB unified memory. Budget gaming GPUs like RTX 2060. Popular for experimentation at extreme compression.

  Runs on 0 of 18 popular devices

• INT44-bit

  ~402.6 GB est.

  13% of the FP32 size · the usual sweet spot of size and quality

  Integer 4-bit quantization using ~12.5% of FP32 memory. Very efficient for inference. A 7B model needs ~3.5GB RAM.

  Good for Cost-effective inference at scale. Works on mobile devices, edge hardware, and consumer laptops. Most popular for local LLM use.

  Runs on 0 of 18 popular devices

• NF44-bit

  ~402.6 GB est.

  13% of the FP32 size · the usual sweet spot of size and quality

  Normalized 4-bit optimized for neural networks using ~12.5% of FP32 memory. Better quality than INT4 at same size. A 7B model needs ~3.5GB RAM.

  Good for Best 4-bit choice for quality. Good for fine-tuning on consumer hardware. Popular with tools like llama.cpp and Ollama on Mac/Windows.

  Runs on 0 of 18 popular devices

• INT22-bit

  ~201.3 GB est.

  6% of the FP32 size · very compressed, noticeable quality loss

  Extreme 2-bit compression using only ~6.25% of FP32 memory. Experimental and lossy. A 7B model needs ~1.75GB RAM.

  Good for Research and experimentation only. May produce lower quality outputs. Only use if memory is extremely limited or for non-critical tasks.

  Runs on 0 of 18 popular devices

Hatched bars are estimates. For comparison, a 24 GB GPU has about 21.6 GB usable.

Fit on popular hardware at FP4

Each bar shows how much of a device's usable memory this model takes at FP4, the format most people download first. The tick marks the usable limit, and a bar that reaches past it means the model does not fit.

• Intel Arc A770 8GB✕ Does not fit402.6 GB / 7.2 GB usable
• Qualcomm Snapdragon 8 Gen 3 Leading Version✕ Does not fit402.6 GB / 9.0 GB usable
• Qualcomm Snapdragon 8 Elite✕ Does not fit402.6 GB / 9.0 GB usable
• Mac Mini M4✕ Does not fit402.6 GB / 12.0 GB usable
• MacBook Pro M4 14-inch✕ Does not fit402.6 GB / 12.0 GB usable
• AMD Radeon RX 7800 XT✕ Does not fit402.6 GB / 14.4 GB usable
• Intel Arc A770 16GB✕ Does not fit402.6 GB / 14.4 GB usable
• Qualcomm Snapdragon X Elite✕ Does not fit402.6 GB / 12.0 GB usable
• AMD Radeon RX 7900 XT✕ Does not fit402.6 GB / 18.0 GB usable
• Mac Mini M4 Pro✕ Does not fit402.6 GB / 18.0 GB usable
• NVIDIA RTX 4090✕ Does not fit402.6 GB / 21.6 GB usable
• AMD Radeon RX 7900 XTX✕ Does not fit402.6 GB / 21.6 GB usable
• Mac Mini M4 32GB✕ Does not fit402.6 GB / 24.0 GB usable
• NVIDIA RTX 5090✕ Does not fit402.6 GB / 28.8 GB usable
• Mac Studio M5 Max✕ Does not fit402.6 GB / 27.0 GB usable
• MacBook Pro M4 Max 16-inch✕ Does not fit402.6 GB / 36.0 GB usable
• NVIDIA RTX 6000 Ada✕ Does not fit402.6 GB / 43.2 GB usable
• Mac Studio M5 Ultra✕ Does not fit402.6 GB / 72.0 GB usable

Every quantization × popular device

These verdicts are estimated at a 4096-token context. How fit is calculated

Not seeing your device? Check every device we track

Measured throughput

No one has submitted a measured benchmark for this model yet. The fit table above is an estimate.

Other DeepSeek models

• DeepSeek-V4-Flash (MoE)
• DeepSeek-V4-Pro (MoE)

Similar-sized models from other families

• Llama 3.1 405B (405B)
• Qwen 3 235B-A22B (MoE) (235B)
• Mistral Large 2 (MoE) (141B)
• Qwen 2.5 72B (72B)
• Llama 3 70B (70.6B)
• Llama 3.3 70B (70.6B)