📄 中文摘要
在涉及智能代理的众多应用中，代理生成的海量警报（大多数为虚假警报）可能使用户产生麻木感，从而忽视关键问题，导致所谓的“警报疲劳”。一种常见策略是训练反射模型作为过滤器，利用用户验证反馈收集的标注数据来拦截虚假警报。然而，收集的数据往往存在噪声，尤其是在生产环境中。由于手动标注清理噪声的成本高昂，研究提出了一种新方法ConceptRM，旨在构建高质量语料库，以训练能够有效拦截虚假警报的反射模型。该方法仅需少量专家注释即可实现高效的数据清理。
ConceptRM：通过基于共识的纯度驱动数据清理来减轻警报疲劳的探索
📄 English Summary
ConceptRM: The Quest to Mitigate Alert Fatigue through Consensus-Based Purity-Driven Data Cleaning for Reflection Modelling
The overwhelming volume of alerts generated by intelligent agents, predominantly false, can lead to user desensitization and critical issues being overlooked, a phenomenon known as 'alert fatigue.' A common approach involves training a reflection model to filter out false alerts using labeled data derived from user verification feedback. However, a significant challenge arises from the noisy nature of such data, often collected in production environments. Cleaning this noise through manual annotation incurs high costs. This research proposes a novel method, ConceptRM, for constructing a high-quality corpus aimed at training a reflection model that effectively intercepts false alerts. The method requires only a small amount of expert annotation to achieve efficient data cleaning.