CARE：考虑混杂因素的聚合方法用于可靠的大型语言模型评估

The standard paradigm for scalable evaluation employs LLM-as-a-judge ensembles, yet their aggregation mechanisms suffer from a fundamental flaw: they implicitly assume that judges provide independent estimates of true quality. In reality, LLM judges exhibit correlated errors due to shared latent confounders, such as verbosity, stylistic preferences, or training artifacts. This correlation can lead standard aggregation rules, like majority voting or averaging, to yield minimal gains or even exacerbate systematic errors. To tackle this issue, a new framework called CARE is introduced, which explicitly models LLM judge scores as arising from both a latent true-quality signal and shared confounding factors. By modeling these influences, CARE aims to enhance the reliability and accuracy of evaluations.