ImageNet：基准测试的终结还是新生？

Researchers revisited ImageNet by undertaking a rigorous re-labeling process for its validation images, aiming for cleaner and more accurate annotations. When recent AI models were evaluated against this newly labeled dataset, the previously reported substantial performance gains appeared significantly diminished. This suggests that some of the celebrated advancements might have stemmed from models overfitting to the original dataset's inherent quirks rather than demonstrating genuine, robust progress. The predictive power of the original ImageNet labels for the new annotations has decreased, indicating that traditional leaderboards might be less effective in accurately assessing the true intelligence of computer vision systems. Nevertheless, the updated labeling methodology successfully rectified numerous errors within the dataset, thereby reinforcing ImageNet's continued utility as a valuable benchmark for future research and development. In essence, the discrepancy between reported progress and actual generalized intelligence highlights the critical need for continuous re-evaluation of established benchmarks to ensure meaningful advancements in the field of artificial intelligence.