DeepSeek-V2.5
by deepseek-ai
236B params · text-generation · 733 likes · 85.3k downloads
DeepSeek-V2.5 is a 236B parameter model. At Q4 quantization it requires 118GB of VRAM. It fits on H200 SXM 141 GB (29 tok/s), M2 Ultra 192GB (2 tok/s) but performance may be limited.
GPU compatibility
| GPU | VRAM | Q4 Decode | Verdict |
|---|---|---|---|
| H200 SXM 141 GB | 141GB | 29 tok/s | tight |
| M2 Ultra 192GB | 192GB | 2 tok/s | tight |