Open-weight local model
DeepSeek R1 Distill Qwen 32B
Demanding dense reasoning model. On 24GB cards, lower quantization or short context may be necessary.
- Parameters
- 32.764B
- Recommended quantization
- Q4_K_M
- Weight estimate
- 20.4GB
- Maximum context
- 128K
Quantization memory at the default context
| Quantization | Weights | KV cache | Total VRAM |
|---|---|---|---|
| FP16 | 66.00GB | 2.15GB | 74.75GB |
| Q8_0 | 36.00GB | 2.15GB | 41.75GB |
| Q6_K | 27.95GB | 2.15GB | 32.89GB |
| Q5_K_M | 23.65GB | 2.15GB | 28.16GB |
| Q4_K_M | 20.40GB | 2.15GB | 24.59GB |
| Q3_K_M | 16.90GB | 2.15GB | 20.74GB |
| Q2_K | 10.75GB | 2.15GB | 13.97GB |
GPUs to check for this model
- Can RTX 4090 24GB run it?
- Can RTX 5090 32GB run it?
- Can RTX 5090 D 32GB run it?
- Can RTX 6090 32GB run it?
- Can Nvidia TITAN V CEO Edition 32GB run it?
- Can Nvidia RTX 5000 Ada Generation 32GB run it?
- Can Apple MacBook Pro M4 Max (40-core GPU) 48GB run it?
- Can Apple MacBook Pro M3 Max (40-core GPU) 48GB run it?
- Can Apple MacBook Pro M4 Pro (16-core GPU) 48GB run it?