Search
Huawei Officially Open-Sources Its UCM Inference Memory Management Technology
2025-11-05 /
Read about 0 minute
Author：小编
On November 5, 2025, Huawei made an announcement regarding the official open-sourcing of a pivotal technology for AI inference acceleration: UCM (Unified Cache Manager) inference memory data management. At the heart of UCM lies KV Cache, and it boasts four core capabilities, namely sparse attention, prefix caching, prefill offloading, and heterogeneous PD decoupling. By fostering a three-layer collaboration among the inference framework, computing power, and storage, this technology can significantly enhance performance. Specifically, it can reduce the first-token latency by up to 90%, boost system throughput by a maximum of 22 times, and expand the context window by 10 times, ultimately driving down the inference cost per token.
-
C114 Communication Network -
Communication Home
7 X 24 Track global technological trends
