DeepSeek Open-Sources Inference Framework; Social Media Estimates Inference Gross Margin Could Exceed 96% at Maximum
4 hour ago / Read about 0 minute
Author:小编   

DeepSeek has open-sourced its internal inference framework, DeepInfer, along with a Token encoding/decoding tool called deepseek-recipe. Compatible with NVIDIA GPUs and Huawei Ascend NPUs, the framework operates under the Apache-2.0 license and is designed to enhance the efficiency of large-scale inference for DeepSeek V4.1-Flash, reducing latency and increasing throughput. According to DeepSeek, V4.1-Flash can process 155 trillion Tokens per day in a single cluster, generating up to $38 in revenue per GPU hour during off-peak electricity periods. Industry experts, based on parameters such as cache rates and throughput, estimate that its inference gross margin could reach as high as 84.2%-96.1%. Extrapolating from this, the gross margins for API services of mainstream models like OpenAI and Anthropic could reach 95%, while Anthropic's Claude Max subscription service could still maintain a gross margin of approximately 41% even when users fully utilize their quotas. The industry has engaged in discussions about technical approaches, comparing the cost-reduction methods of Groq's custom ASICs with DeepInfer's approach of leveraging general-purpose chips to boost throughput through software scheduling. However, some argue that the aforementioned high-margin estimates only consider the difference between Token pricing and single-card hardware consumption, without accounting for costs such as model training, networking, and labor. Therefore, their accuracy still requires validation with more real-world operational data.