从偏见聊天机器人到偏见代理:考察角色分配对大型语言模型代理鲁棒性的影响
📄 中文摘要
大型语言模型(LLMs)越来越多地被部署为能够进行实际操作的自主代理,超越了文本生成的范畴。尽管文本生成中的角色引发的偏见已被广泛记录,但这些偏见对代理任务表现的影响仍然鲜有研究,然而这些影响直接关系到操作风险。本研究首次系统性地展示了基于人口统计的角色分配如何改变LLM代理的行为,并在多个领域中降低其性能。通过评估广泛使用的模型在战略推理、规划和技术操作等代理基准上的表现,发现性能存在显著差异,最高可达26.2%的下降,这种下降主要由与任务无关的角色引发的偏见所驱动。
📄 English Summary
From Biased Chatbots to Biased Agents: Examining Role Assignment Effects on LLM Agent Robustness
Large Language Models (LLMs) are increasingly deployed as autonomous agents capable of actions with real-world impacts beyond text generation. While persona-induced biases in text generation are well documented, their effects on agent task performance remain largely unexplored, posing significant operational risks. This study presents the first systematic case study demonstrating that demographic-based persona assignments can alter LLM agents' behavior and degrade performance across diverse domains. Evaluating widely deployed models on agentic benchmarks spanning strategic reasoning, planning, and technical operations reveals substantial performance variations, with degradation reaching up to 26.2%, driven by task-irrelevant persona biases.
Powered by Cloudflare Workers + Payload CMS + Claude 3.5
数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等