Overview
- DeepSeek published DeepInfer on GitHub under the Apache‑2.0 license on Tuesday as an open, high‑performance inference framework built to run DeepSeek‑V4.1‑Flash locally or on private clusters.
- The code bundles low‑level optimizations such as custom kernels, distributed parallelism including MoE expert parallelism, speculative decoding to speed generation, and structured output to produce predictable formats.
- DeepInfer officially targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs and requires modern system prerequisites like Linux, GCC 13, Python 3.12, CUDA 13 or CANN 9.2 for building and running.
- DeepSeek explicitly says DeepInfer is not a turnkey service: deployers must add a KV cache, disaggregated prefill/decode components, and load balancing for a complete production stack, and the lab’s throughput and latency figures are provider‑reported and not independently validated.
- A companion tool, deepseek‑recipe, aligns tokenization with DeepSeek’s API to reduce output drift, and the release follows the lab’s Oct. 17 V4.1‑Flash weight drop as part of a broader strategy to lower operating costs and encourage self‑hosting.