Tencent has recently unveiled its Hunyuan Simultaneous Interpretation Large Model, Hy-SimulMT, and has made the groundbreaking decision to release it as open-source software. This innovative model is designed for low-latency, high-accuracy performance, enabling real-time speech-to-speech translation between Chinese and English. It operates by translating speech as it is being spoken, delivering immediate translation outputs along with corresponding speech. Furthermore, the model supports real-time voice cloning, accurately replicating the speaker's unique voice and tone. Tests have demonstrated that the model boasts a response time of approximately 3 seconds, outpacing the 4-5 second delay typically observed with human professional interpreters. Moreover, its translation quality exceeds that of mainstream open-source simultaneous interpretation models and surpasses most commercial simultaneous interpretation APIs currently available in the market. On the RealSI, a widely recognized open-source evaluation set for long-audio in the industry, Hy-SimulMT achieved top-tier performance among open-source models, boasting the lowest latency and outperforming the majority of commercial APIs. It also delivered exceptional results on short-audio test sets. To evaluate the model, the Tencent team compiled a dataset of newly released Chinese-English audio content, spanning ten domains and multiple scenarios, all in compliance with relevant regulations. This dataset was then compared against the work of six human simultaneous interpretation experts, all of whom held CATTI Level 1 and Level 2 interpretation certificates and possessed extensive practical experience. The findings revealed that the model's translation quality matched, and in some cases, even surpassed that of the human experts, all while maintaining lower latency.
