Search
Xiaohongshu Unveils Its First Large Model, Pioneering Pre-training Without Synthetic Data
6 day ago /
Read about 0 minute
Author：小编
Xiaohongshu has recently released its groundbreaking first large model, named dots.llm1. This model represents a Mixture-of-Experts (MoE) architecture, boasting an impressive 142 billion parameters. Notably, during inference, it efficiently activates only 14 billion parameters, ensuring high performance while drastically cutting down on both training and inference expenses. In its pre-training phase, dots.llm1.ins leveraged an extensive 11.2 trillion tokens of non-synthetic data, achieving performance comparable to Alibaba's Qwen3-32b across Chinese and English language tasks, mathematics, and alignment challenges.
-
C114 Communication Network -
Communication Home
7 X 24 Track global technological trends
