According to the official announcement from KIMI, the company has unveiled an open-source, lightweight middleware solution named Checkpoint-Engine. This middleware is specifically tailored for inference engines of large language models (LLMs). Leveraging an innovative technical architecture, Checkpoint-Engine is capable of executing weight updates for trillion-parameter models in just around 20 seconds within a cluster environment comprising thousands of GPUs.

It offers two distinct update modes: broadcast and point-to-point. Additionally, the middleware incorporates pipeline optimization techniques to ensure maximum hardware utilization. With its lightweight design and flexible deployment capabilities, Checkpoint-Engine significantly reduces resource consumption and operational costs in large-scale clusters. This provides a standardized and efficient solution for the deployment of enterprise-level LLM inference systems. The project is now officially available as open-source software.