📄 中文摘要
在 AI 技术快速发展的背景下，如何为每次运行返回略有不同字符串的函数编写单元测试成为一大挑战。传统的手动检查输出方式难以扩展，现代解决方案是使用更大、更智能的模型（如 GPT-4.5 或 Claude 3.7）来评估较小的生产模型（如 Llama 3）的输出。实施过程包括定义评估标准，要求评估模型根据事实准确性、语气和 JSON 结构遵循性对输出进行评分，并构建黄金数据集以确保评估的有效性和一致性。
测试 AI 是困难的（但你必须这样做）
出处: Testing AI is Hard (But You Have To Do It)
发布: 2026年3月2日
📄 English Summary
Testing AI is Hard (But You Have To Do It)
Testing AI models poses unique challenges, particularly when functions return slightly different strings with each execution. The traditional method of manually reviewing outputs is not scalable. A modern approach involves using a larger, more sophisticated model (like GPT-4.5 or Claude 3.7) to evaluate the outputs of a smaller production model (such as Llama 3). The implementation process includes defining a strict rubric for the judge model, which scores responses based on factual accuracy, tone, and adherence to a JSON schema, as well as building a golden dataset to ensure the effectiveness and consistency of evaluations.