DeepSeek Open-Sources DeepInfer, Its Production Inference Engine

Overview

• DeepSeek published DeepInfer on Tuesday, Oct. 20, 2026, releasing the code and a companion deepseek‑recipe tool on GitHub under the Apache‑2.0 license.
• The open source package bundles low‑level optimized kernels, speculative (draft‑token) decoding, MoE‑aware parallelism, asynchronous scheduling, and structured output support that DeepSeek uses to run DeepSeek‑V4.1‑Flash in production.
• DeepInfer ships day‑one support for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs and includes reproducible benchmark scripts, but DeepSeek’s published performance figures are self‑reported and tied to specific SM100 test setups.
• The release targets offline/batch inference and deliberately omits full online‑serving subsystems — notably a KV cache, VE/prefill/decode disaggregation, and replica load balancing — so adopters must integrate those pieces themselves.
• The repo drew rapid community interest and could lower hosting costs for enterprises that invest the engineering effort to validate and integrate it, while also sharpening DeepSeek’s hardware diversification between NVIDIA and Huawei.