可控探索的混合策略强化学习与可验证奖励在多模态推理中的应用

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a primary learning paradigm for enhancing the reasoning capabilities of Multi-Modal Large Language Models (MLLMs). However, during RL training, the vast state space of MLLMs and sparse rewards often lead to entropy collapse, policy degradation, or over-exploitation of suboptimal behaviors. This necessitates an exploration strategy that maintains productive stochasticity while avoiding the inefficiencies of uncontrolled random sampling. CalibRL is proposed as a hybrid-policy RLVR framework that supports controllable exploration with expert guidance, enabled by two key mechanisms. First, it employs distribution-aware advantage weighting scaling to enhance the efficiency of the exploration process.