📄 中文摘要
Q学习和策略梯度是两种流行的计算机学习方法，通过试错来进行学习。尽管这两种方法在很长一段时间内看起来截然不同，但最新研究表明，当引入一定的随机性时，这两种方法实际上变成了同一思想的软（熵正则化）版本。这一发现令人惊讶，因为计算机学习到的 Q 值往往看起来不正确，但系统仍然能够学习到良好的行为。关键在于这两种方法最终遵循相同的学习规则，只是表达方式不同，因此在引导学习的方式上也相似。研究者在简单游戏测试和大型街机游戏中进行了验证。
策略梯度与软 Q 学习之间的等价性
出处: Equivalence Between Policy Gradients and Soft Q-Learning
发布: 2026年2月17日
📄 English Summary
Equivalence Between Policy Gradients and Soft Q-Learning
Q-learning and policy gradients are two popular methods for teaching computers to learn from trial and error. For a long time, they appeared to be fundamentally different approaches. However, recent research reveals that by introducing a degree of randomness, these two methods actually converge into a soft (entropy-regularized) version of the same underlying principle. This finding is surprising because the Q-values learned by the computer often seem incorrect, yet the system still manages to learn effective behaviors. The key insight is that both methods ultimately follow the same learning rules, albeit expressed differently, which leads to similar guidance in the learning process. Researchers have tested this equivalence on simple game scenarios as well as larger arcade games.