DeepSeek-Coder-V2-Instruct
by deepseek-ai
236B params · text-generation · 681 likes · 5.7k downloads
DeepSeek-Coder-V2-Instruct is a 236B parameter model. At Q4 quantization it requires 118GB of VRAM. It fits on H200 SXM 141 GB (29 tok/s), M2 Ultra 192GB (2 tok/s) but performance may be limited.
GPU compatibility
| GPU | VRAM | Q4 Decode | Verdict |
|---|---|---|---|
| H200 SXM 141 GB | 141GB | 29 tok/s | tight |
| M2 Ultra 192GB | 192GB | 2 tok/s | tight |