Recently, artificial intelligence firm Caiyun Technology revealed that its foundational model algorithm team has consecutively broken the world record for large-model training speed twice in the NanoGPT Speedrun, a globally recognized open-source AI technology speed competition. The team managed to slash the standard training time down to a mere 1 minute and 16 seconds. The pertinent code has now been seamlessly integrated into the official open-source repository. It has been reported that the Caiyun Technology team has pioneered two groundbreaking innovations in model architecture: DCMHA (Dynamically Combinable Multi-Head Attention) and MUDD (Multi-path Dynamic Dense Connection). These innovations were presented at the International Conference on Machine Learning (ICML), a premier international machine learning conference, in 2024 and 2025, respectively, and both were chosen for oral presentations, highlighting their significance and impact in the field.
