📄 中文摘要
大型语言模型（LLMs）在各种任务中展现出卓越能力，但其行为控制和对齐仍是挑战。现有方法常依赖人工反馈，难以扩展且效率低下。本文提出一种新颖的交互式监督框架，旨在通过可扩展的人机协作来有效引导LLMs。该框架的核心思想是利用人类的少量、高价值反馈，结合自动化机制，逐步将LLMs的行为塑造成符合预期。研究人员设计了一套新颖的交互协议，允许人类以更高效的方式提供指导，例如通过选择、排序或提供少量修正。同时，该框架引入了基于模型自身能力和不确定性的自适应反馈请求策略，以最小化人类干预成本。实验结果表明，该方法在多个任务上显著提升了LLMs的对齐效果和性能，同时大幅降低了所需的人工监督量。这为未来构建更安全、更可控、更符合人类价值观的LLMs提供了新的途径，尤其在需要持续迭代和微调的实际应
📄 English Summary
Steering LLMs via Scalable Interactive Oversight
Large Language Models (LLMs) demonstrate remarkable capabilities across various tasks, yet controlling their behavior and aligning them with human intent remains a significant challenge. Existing methods often rely heavily on human feedback, which is typically difficult to scale and inefficient. This paper introduces a novel interactive oversight framework designed to effectively steer LLMs through scalable human-AI collaboration. The core idea is to leverage sparse, high-value human feedback combined with automated mechanisms to progressively shape LLM behavior towards desired outcomes. Researchers have devised a new set of interactive protocols that enable humans to provide guidance more efficiently, for instance, by selecting, ranking, or offering minimal corrections. Concurrently, the framework incorporates an adaptive feedback request strategy based on the model's capabilities and uncertainty, aiming to minimize human intervention costs. Experimental results demonstrate that this approach significantly improves LLM alignment and performance across multiple tasks, while substantially reducing the required amount of human supervision. This work offers a promising new avenue for building safer, more controllable LLMs that better align with human values, particularly relevant for real-world applications demanding continuous iteration and fine-tuning.