📄 中文摘要
经过三周的时间和约340美元的投入，对三种大型语言模型（LLM）进行了基准测试，重点关注这些模型在实际生产环境中自主代理所执行的任务。这些任务并非演示任务或简单的摘要，而是一些单调、重复且偶尔奇怪的工作，这些工作支撑着一个由六个代理组成的系统。与大多数基准测试不同，本次测试关注的是模型在实际应用中的表现，尤其是在需要频繁执行特定任务的情况下。测试涵盖了内容生成、数据处理等四类任务，结果揭示了不同模型的优劣和一些意外发现。
Claude、GPT-4 和 Gemini 在自主代理任务中的比较：我的生产基准测试
📄 English Summary
Claude vs GPT-4 vs Gemini for Autonomous Agent Tasks: My Production Benchmark
Over three weeks and with an expenditure of about $340, a benchmarking study was conducted on three large language models (LLMs) focusing on the actual tasks performed by autonomous agents in a production environment. These tasks are not demo tasks or simple summaries but involve monotonous, repetitive, and occasionally peculiar activities that keep a six-agent system operational. Unlike most benchmarks that assess general reasoning on standardized tasks, this study emphasizes the models' performance in real-world applications, particularly when specific tasks need to be executed frequently. The evaluation covered four categories of tasks, including content generation and data processing, revealing the strengths and weaknesses of different models along with some surprising findings.