On October 20, the Shanghai Memory Tensor team officially announced the release and open-sourcing of the MemT-1.0 series, a groundbreaking set of memory-native models. In contrast to traditional large models, which commence each conversation from a blank slate, the MemT-1.0 series integrates a sustainable and updatable internal memory mechanism into its computational framework. This innovative approach allows the model to directly leverage historical information and task experience in subsequent related tasks, eliminating the necessity for repeatedly inputting original historical texts.

The open-sourced models within this series, namely MemT-1.0-Flash and MemT-1.0-Nano, boast parameter counts of 28.8 billion and 3.6 billion, respectively. The research team has ingeniously incorporated native memory modules into specific Transformer components, enabling the original attention mechanism and the new memory attention mechanism to function in tandem. Consequently, memory becomes an indispensable element of the model's forward propagation process, enhancing its overall efficiency and performance.