开源权重本地模型
Mistral Small 3.1 24B Instruct
High-quality local multimodal model. Text estimates exclude vision overhead; 24GB is the practical Q4 target.
- 参数量
- 24B
- 推荐量化
- Q4_K_M
- 权重估算
- 15GB
- 最大上下文
- 128K
默认上下文下的量化显存
| 量化 | 权重 | KV cache | 总显存 |
|---|---|---|---|
| FP16 | 48.50GB | 1.34GB | 54.69GB |
| Q8_0 | 26.50GB | 1.34GB | 30.49GB |
| Q6_K | 20.48GB | 1.34GB | 23.86GB |
| Q5_K_M | 17.32GB | 1.34GB | 20.40GB |
| Q4_K_M | 15.00GB | 1.34GB | 17.84GB |
| Q3_K_M | 12.40GB | 1.34GB | 14.98GB |
| Q2_K | 7.88GB | 1.34GB | 10.02GB |