Open-weight local model
Google Gemma 3 4B IT
Compact multimodal option. Text estimates exclude image encoder and vision activation peaks.
- Parameters
- 4.3B
- Recommended quantization
- Q4_K_M
- Weight estimate
- 3.3GB
- Maximum context
- 128K
Quantization memory at the default context
| Quantization | Weights | KV cache | Total VRAM |
|---|---|---|---|
| FP16 | 8.80GB | 1.14GB | 10.82GB |
| Q8_0 | 5.10GB | 1.14GB | 7.04GB |
| Q6_K | 3.67GB | 1.14GB | 5.61GB |
| Q5_K_M | 3.10GB | 1.14GB | 5.04GB |
| Q4_K_M | 3.30GB | 1.14GB | 5.24GB |
| Q3_K_M | 2.80GB | 1.14GB | 4.74GB |
GPUs to check for this model
- Can RTX 4060 8GB run it?
- Can Nvidia RTX A5000-8Q 8GB run it?
- Can RTX 3060 Ti GDDR6X 8GB run it?
- Can RTX 3070 Ti 8GB run it?
- Can RTX 3070 Ti 8 GB GA102 run it?
- Can Intel Arc A580 8GB run it?
- Can Intel Arc A750 8GB run it?
- Can Nvidia Quadro RTX 4000 Max-Q 8GB run it?
- Can Nvidia Quadro RTX 4000 Mobile 8GB run it?