开源权重本地模型
Qwen 2.5 Coder 7B Instruct
Popular code-specialized model that is practical on an 8GB GPU at 4-bit with modest context.
- 参数量
- 7.616B
- 推荐量化
- Q4_K_M
- 权重估算
- 5.1GB
- 最大上下文
- 32K
默认上下文下的量化显存
| 量化 | 权重 | KV cache | 总显存 |
|---|---|---|---|
| FP16 | 15.60GB | 0.47GB | 17.63GB |
| Q8_0 | 8.70GB | 0.47GB | 10.04GB |
| Q6_K | 6.50GB | 0.47GB | 7.77GB |
| Q5_K_M | 5.50GB | 0.47GB | 6.77GB |
| Q4_K_M | 5.10GB | 0.47GB | 6.37GB |
| Q3_K_M | 4.30GB | 0.47GB | 5.57GB |