⚡️SWE-Bench 验证的终结 — Mia Glaese & Olivia Watkins, OpenAI Frontier Evals & Human Data

The OpenAI Frontier Evals team has announced the end of SWE-Bench verification, marking a significant shift in frontier agent evaluations. The new evaluation methods will focus more on agents' performance in complex tasks, particularly their adaptability and flexibility in real-world scenarios. By incorporating human data, the evaluations will become more comprehensive, better reflecting the practical application potential of agents. This change aims to advance artificial intelligence technology, enhancing the utility and reliability of agents to ensure their effectiveness in diverse environments.