带有过程奖励的截断步级采样用于检索增强推理

Training large language models to reason with search engines via reinforcement learning faces a fundamental credit assignment problem: existing methods like Search-R1 provide only sparse outcome rewards after an entire multi-step trajectory, making it challenging to attribute success or failure to individual reasoning and retrieval decisions. Process-reward methods such as StepSearch alleviate this issue by introducing step-level supervision but rely on heuristic rewards like TF-IDF overlap with gold documents and still sample k complete trajectories per example, leading to high gradient variance. The proposed SLATE framework is built on two complementary ideas: truncated step-level sampling that generates k trajectories sharing common elements and process rewards that optimize the reasoning process, thereby enhancing the model's reasoning capabilities and stability.