DeepSeek open-sources AI engine it says cuts running costs

On Tuesday, October 20, Chinese AI developer DeepSeek released DeepInfer, the in-house software it uses to serve its models at scale, as open source. The company presented the release as a way to lower the cost of running large AI systems. Bloomberg reported that the move could make DeepSeek's technology cheaper and easier for businesses to run. DeepInfer is software that makes the company's models run faster and use computing resources more efficiently.bloomberg.com

DeepSeek said DeepInfer powers DeepSeek-V4.1-Flash "at scale with high throughput and low latency." It also said it is sharing its production stack "to help the community lower compute costs, reduce energy use, and make AI more affordable," according to its announcement on X. The company added that inference code for future models and later optimizations will be published in the same repository.alphasignal.ai+1

What is in the release

The GitHub repository calls DeepInfer "the high-performance inference framework that powers DeepSeek's large-scale production services." The first release offers offline inference and supports DeepSeek-V4.1-Flash. The code is under the Apache 2.0 license. It includes DeepSeek's custom kernels, such as FlashMLA, DeepGEMM and DeepEP, along with DSpark speculative decoding, batch-invariant inference and structured output. The framework is built on the open-source vLLM project.alphasignal.ai+1

DeepSeek also released deepseek-recipe, a component that converts API requests into model prompts and parses the output. It matches the processing logic of DeepSeek's official API, so self-hosted deployments behave like the company's own service.tech.ifeng.com

Nvidia and Huawei hardware

DeepInfer supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs. Last week, DeepSeek said it plans to pre-train models on that same Huawei chip by the end of 2026. Chinese media coverage of the release said a single DeepSeek cluster processes 155 trillion tokens a day. That figure has not been independently verified.aiweekly.co+2

The announcement came three days after DeepSeek updated the model to V4.1-Flash-1017, with API pricing unchanged. Hualong Securities said DeepSeek's fast updates support the case for falling costs and rising demand for inference computing in China.stcn.com

Pieces left out

The release is not a full serving system. According to the README, it does not include a KV cache system, prefill and decode disaggregation, or load balancing across replicas. Companies that want to run a production service will need to build that infrastructure themselves. That makes DeepInfer most useful for teams that already operate distributed inference systems.fourweekmba.com+1

In its Chinese-language announcement, DeepSeek said it hopes the release will help developers and enterprises build private and public cloud deployments efficiently, "so that intelligence can reach a wider population at a lower price".sina.cn