迈向可还原不确定性建模以实现可靠大型语言模型智能体

Uncertainty quantification (UQ) for large language models (LLMs) is a foundational element for ensuring the safety and reliability of daily LLM applications. Despite the increasing deployment of LLM agents in highly complex tasks, the majority of UQ research remains confined to single-turn question-answering scenarios. A critical shift is needed in UQ research towards realistic settings involving interactive agents, necessitating a principled framework for agent UQ. This paper introduces the first general formulation of agent UQ, which effectively subsumes a broad spectrum of existing UQ setups. Within this formulation, prior works are shown to implicitly treat LLM UQ as an uncertainty accumulation process. This perspective, however, proves inadequate and breaks down when applied to interactive agents operating in open-world environments. In contrast, a novel P-UQ framework is proposed, which conceptualizes agent uncertainty as reducible rather than merely accumulative. This framework offers a fresh perspective and innovative solutions for enhancing the reliability of LLM agents in complex, open-world contexts by introducing the concept of reducible uncertainty. The aim is to significantly improve the safety and robustness of LLM agents in practical applications, moving beyond the limitations of traditional UQ approaches.