Overview
- DeepSeek published DeepInfer under the Apache‑2.0 license on Tuesday, October 20, 2026, and simultaneously released deepseek‑recipe so community deployments can match the company’s token encoding and API behavior.
- The open release bundles DeepSeek’s production optimizations, including custom kernels, speculative (draft‑token) decoding, asynchronous scheduling, MoE parallelism and structured output controls.
- DeepInfer ships with first‑class support for NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs and notes that batch‑invariant inference and deterministic sampling are available only on NVIDIA in this release.
- DeepSeek published high‑throughput performance figures for V4.1‑Flash—155 trillion tokens per day per cluster, a 96% input cache hit rate, ~400 ms time‑to‑first‑token and roughly 230 tokens/sec generation—and provided reproducible sweep scripts while noting those numbers come from its own runs.
- The package targets offline and batch inference and deliberately omits key online service pieces such as a KV cache, VE/prefill/decode disaggregation and replica load balancing, and the release has drawn rapid community interest and extra scrutiny from recent U.S. agency restrictions that could shape enterprise adoption.