Search
AMD Unveils Open-Source Instella-MoE Language Model
1 day ago /
Read about 0 minute
Author：小编
On July 24, AMD officially launched the Instella-MoE series, a suite of open-source mixture-of-experts (MoE) language models. These models boast an impressive total of 16 billion parameters, with 2.8 billion of them being actively utilized. Constructed upon a sophisticated 27-layer decoder architecture, the Instella-MoE series is available in six distinct versions. Notably, the pre-training speed of these models has witnessed a significant boost of 12.7%. Moreover, during the inference phase, the response time for the initial token has been substantially slashed by up to 39.2%, ensuring a more efficient and streamlined user experience.
-
C114 Communication Network -
Communication Home
7 X 24 Track global technological trends
