NVIDIA has officially published a guide for large-scale model inference, named 'AI Model Co-Design with Speculative Decoding.' The crux of this guide lies in a curated selection table of mainstream inference acceleration solutions, where a striking majority of the pivotal solutions are helmed by Chinese teams. Speculative decoding technology harnesses the power of a smaller model for front-end prediction, which is then validated by a larger model, achieving lossless acceleration. The extent of this acceleration hinges on the anticipated acceptance length and the time consumed in the process.

The table delineates four generations of solutions that trace the main trajectory of technological evolution:

• EAGLE-3, developed by teams from Peking University and collaborators, which once reigned supreme in the field but has since been outstripped in terms of acceptance length.
• MTP, which NVIDIA has lauded as the optimal choice for large models on GPUs and has been refined by DeepSeek-V3.
• DFlash, a parallel architecture crafted by a team that includes scholars from NVIDIA Research, achieving a remarkable 6x lossless acceleration.
• DSpark, developed by teams from DeepSeek and Peking University, which boasts support for a higher acceptance length.

Furthermore, the guide underscores that hardware physical constants will impose constraints on the evolution of model architectures. Consequently, the current competitive landscape in large-scale models has pivoted towards exploring the physical limits of silicon. In this endeavor, Chinese teams have emerged as pivotal forces, pushing the boundaries and breaking through these limitations.