Step 5 Preview of JieYue: Large Model Weights Unveiled, Enabling a 1 Million Token Context Window
10 hour ago / Read about 0 minute
Author:小编   

JieYue has officially rolled out the weights for its Step 5 Preview, simultaneously making BF16 and FP8 model weights, technical blog posts, and open-source development tools accessible to the public. This model is tailored for long-range Agent tasks, utilizing a sparse Mixture-of-Experts (MoE) architecture. With a staggering total of 600 billion parameters, it activates only 27 billion parameters per token, ensuring efficiency. It boasts a context window that can accommodate up to 1 million tokens and is inherently compatible with both textual and visual inputs. The model has showcased outstanding performance across a variety of professional scenarios, recently securing the second position globally on both the weekly and daily invocation lists of OpenRouter. Presently, the model weights can be found on the Hugging Face and ModelScope platforms. They are compatible with various chip platforms and inference frameworks, allowing for versatile deployment and offline inference. This flexibility makes it easier for developers to incorporate the model into their systems and construct intelligent applications. Additionally, the official technical blog offers an in-depth exploration of the model architecture's key design elements and collaborative training optimizations, providing crucial support for the execution of long-range, complex Agent tasks.