📄 中文摘要
该研究提出了一种全面的评估框架，用于亚马逊的代理 AI 系统，旨在应对复杂的代理 AI 应用。框架包含两个核心组件：一个通用评估工作流程，标准化不同代理实现的评估程序；以及一个代理评估库，提供亚马逊 Bedrock AgentCore 评估中的系统测量和指标，结合亚马逊特定用例的评估方法和指标。这一框架为优化和提升代理 AI 系统的性能提供了系统化的支持。通过这些方法，亚马逊能够更有效地评估和改进其代理 AI 系统的实际应用效果。
评估 AI 代理：来自亚马逊构建代理系统的现实世界经验
📄 English Summary
Evaluating AI agents: Real-world lessons from building agentic systems at Amazon
A comprehensive evaluation framework for Amazon's agentic AI systems is presented, addressing the complexities of agentic AI applications. The framework consists of two core components: a generic evaluation workflow that standardizes assessment procedures across diverse agent implementations, and an agent evaluation library that offers systematic measurements and metrics in Amazon Bedrock AgentCore Evaluations. Additionally, it includes evaluation approaches and metrics specific to Amazon use cases. This framework provides systematic support for optimizing and enhancing the performance of agentic AI systems, enabling Amazon to more effectively evaluate and improve the real-world application outcomes of its agentic AI systems.