Shanghai Academy of Science and Intelligence Open-Sources REMORY, a 1.9B Memory Module That Enables Small Models to Recursively Self-Improve Over Multiple Days
4 hour ago / Read about 0 minute
Author:小编   

Researcher Xia Hanchen (first author) of the Shanghai Academy of Science and Intelligence and Professor Zhu Siyu (corresponding author) of Fudan University have open-sourced REMORY, a model built on the RSI framework. The model uses only 1.9B trainable parameters and two H800 GPUs to achieve fully self-supervised training: by fitting the memory residuals produced by context compression, it generates soft memory residual tokens that are appended after the summary for use by the main model. Throughout training, the base model weights are frozen, and only the memory module parameters are updated in two stages. Evaluations show that this technique greatly reduces tool errors and repetitive outputs, lowers generation overhead, and improves task performance across main models of different architectures. It can also enable automated scientific research agents to carry memory across context boundaries in long-cycle research tasks spanning multiple days, support recursive self-improvement, and produce results approaching those of GPT-6.1-Terra.