PILOT：大型语言模型通过内化潜在优化轨迹进行规划

Strategic planning is critical for multi-step reasoning, yet compact Large Language Models (LLMs) often lack the capacity to formulate global strategies, leading to error propagation in long-horizon tasks. Analysis reveals that LLMs possess latent reasoning capabilities that can be unlocked when conditioned on explicit plans from a teacher model. However, runtime reliance on external guidance is often impractical due to latency and availability constraints. PILOT (Planning via Internalized Latent Optimization Trajectories) addresses this limitation by internalizing the planning capabilities of a teacher model into smaller LLMs. The core idea is that instead of directly mimicking the teacher model's outputs, the student model learns the latent optimization trajectories that the teacher model employs to generate high-quality plans, achieved through a novel training paradigm. Specifically, PILOT introduces a two-stage training methodology: In the first stage, the student model learns to generate intermediate states and decision sequences similar to the teacher model’s plans through imitation learning. This involves employing contrastive learning and reinforcement learning techniques to guide the student model in approximating the teacher's planning paths within the latent space. In the second stage, the student model further refines its planning capabilities through self-supervision and environmental feedback, without direct guidance from the teacher model, making the internalized planning trajectories more robust and adaptive. This approach allows compact LLMs to acquire planning abilities comparable to larger teacher models without incurring additional inference costs or external dependencies.