DataFlex-RL: An Open-Source RL Data Strategy Framework Developed by Peking University Team, Claims Second Spot on HuggingFace's Daily Leaderboard
13 hour ago / Read about 0 minute
Author:小编   

To tackle the challenges of incorporating dynamic data strategies into the post-training reinforcement learning (RL) process for large-scale models and to enable a fair comparison of the effectiveness of diverse strategies, the DCAI team from Peking University, in collaboration with other researchers, has introduced the open-source framework, DataFlex-RL. This framework leverages the existing training process, introducing three dynamic data strategies—data selection, sample reweighting, and domain ratio adjustment—as customizable components. It disseminates signals such as rewards and advantages generated during training, separating scoring from execution. This design empowers users to seamlessly switch and validate different data strategies within the same RLVR/GRPO process, eliminating the need for repeated trainer modifications. The research team conducted 591 experiments, encompassing a variety of models and tasks in mathematics, logic, and science, to empirically validate the practical effectiveness of various data strategies. DataFlex-RL represents an extension of the previously open-sourced DataFlex project into the realm of reinforcement learning training. The two frameworks synergize to comprehensively address data scheduling needs, spanning from data preparation to reinforcement learning updates.