经验蒙特卡洛树搜索：通过双经验蒙特卡洛树搜索实现连续智能体演化

In continuous control tasks, agents often rely on extensive trial-and-error learning, which can be inefficient in complex environments. This paper introduces a novel framework called Empirical-MCTS (Empirical Monte Carlo Tree Search) to address this challenge. Empirical-MCTS significantly accelerates the learning process by incorporating a "dual-experience" mechanism, combining the agent's own experiences with those derived from external experts or pre-collected data. Specifically, it leverages the powerful planning capabilities of Monte Carlo Tree Search (MCTS) and integrates them with the adaptability of reinforcement learning. This allows the agent to effectively utilize existing knowledge while exploring unknown states. The dual-experience approach enables the agent to learn not only from its own trial-and-error but also to gain guidance from high-quality experiences, whether pre-collected or generated by other agents, thereby avoiding redundant and inefficient exploration. Experimental results demonstrate that Empirical-MCTS achieves faster convergence and superior final performance compared to traditional MCTS and pure reinforcement learning methods across various continuous control benchmarks. This framework offers a promising avenue for efficient agent learning in complex continuous control tasks, particularly in scenarios where data is scarce or exploration costs are high, paving the way for more robust and adaptable intelligent systems.