On July 29, the Tencent Hunyuan team formally announced the open-sourcing of AngelSpec, a cutting-edge speculative decoding framework. This framework encompasses a comprehensive suite of tools and methodologies, spanning from the training of drafters (a component responsible for generating initial text sequences), architectural design, to the seamless deployment of the system online. In a parallel move, the team also made public the MTP (a key component for enhancing decoding efficiency) and DFly drafter weights (pre-trained parameters for the drafter model), along with their corresponding training code, specifically tailored for the Hy3-A21B model.

AngelSpec is designed with flexibility in mind, supporting the integration of any Transformer-based architecture, which is a widely adopted neural network structure in natural language processing. It comes equipped with an MTP scheduler, a sophisticated mechanism for optimizing the decoding process, and a dual-mode inference engine for the DFly drafter. This dual-mode engine allows for both efficient and accurate decoding, depending on the specific requirements of the task at hand. The overarching goal of AngelSpec is to significantly boost both the efficiency and precision of speculative decoding technology, thereby pushing the boundaries of what's possible in natural language generation and understanding.