当人工智能基准测试达到饱和时：基准饱和的系统研究

AI benchmarks play a crucial role in measuring progress in model development and guiding deployment decisions. However, many benchmarks quickly reach saturation, failing to differentiate between the best-performing models, which diminishes their long-term value. This study analyzes benchmark saturation across 60 Large Language Model (LLM) benchmarks selected from technical reports by major model developers. By characterizing benchmarks along 14 properties related to task design, data construction, and evaluation format, factors driving saturation are identified. Five hypotheses are tested to examine how each property contributes to saturation rates. The analysis reveals that nearly half of the benchmarks have reached saturation, impacting their effectiveness and future applicability.