Teams from Zhejiang University and Peking University Introduce SPADER: Enhancing Long-Tail Exploration in Multi-Answer Question Answering Through Step-Wise GRPO and Diversity-Sensitive Rewards
1 day ago / Read about 0 minute
Author:小编   

The SPADER reinforcement learning framework for multi-answer question answering, developed by research teams from Zhejiang University and Peking University, has been selected for presentation at EMNLP 2026. This innovative framework tackles two key challenges in existing reinforcement learning approaches: the imprecise credit assignment stemming from trajectory-level advantage estimation, and the insufficient incentive for long-tail exploration due to the F1 reward's uniform treatment of both common and rare entities. SPADER refines the trajectory-level advantage calculation in GRPO into a step-wise peer advantage mechanism, facilitating fine-grained credit assignment without the need for a parameterized value network or external evaluator. Additionally, it introduces a dual-axis, diversity-aware exploration reward system, which combines horizontal information gain and vertical novelty premium. When integrated with tool cost and terminal answering rewards, this system motivates the agent to uncover long-tail answers and autonomously determine optimal search and termination times. Experimental results demonstrate that SPADER significantly enhances recall and F1 scores across two base models and four multi-answer question answering benchmarks, achieving superior coverage without resorting to excessive search attempts. It effectively guides the agent to persistently discover valid entities until all targets are exhausted. The relevant code and models are now openly accessible.