📄 中文摘要
基础模型在处理涉及时间序列多模态事件的语言生成任务中扮演着重要角色。模型识别视频中最重要子事件的能力，是进行多模态事件叙述或总结的基础。本研究以足球比赛为例，评估模型区分重要与非重要子事件的能力。通过分析足球比赛视频数据，深入探究基础模型如何理解事件的上下文重要性，并识别出比赛中的关键转折点、高光时刻或具有决定性影响的瞬间。研究设计了一系列实验，利用标注数据训练和测试模型，以量化其在复杂动态场景中捕捉语义显著性信息的能力。评估指标涵盖了准确率、召回率、F1分数以及人类感知一致性等多个维度，旨在全面衡量模型在不同粒度下对“重要性”概念的理解。探讨了不同模型架构（例如，基于Transformer的视觉语言模型）和训练策略对识别性能的影响，并分析了模型在面对数据偏差、场景多样性及瞬时事件识别方面的局限性。研究结果为提升基础模型在复杂多模态数据中进行语义理解和事件摘要的能力提供了见解，并指出了未来在多模态事件分析领域中优化模型性能和泛化能力的方向，特别是在自动化视频内容分析、智能推荐系统和体育赛事解说生成等应用场景中具有潜在价值。
📄 English Summary
Where is the multimodal goal post? On the Ability of Foundation Models to Recognize Contextually Important Moments
Foundation models are extensively utilized in real-world applications involving language generation from temporally-ordered multimodal events. A fundamental prerequisite for narrating or summarizing such events is the model's ability to identify the most important sub-events within a video. This work specifically investigates the capacity of these models to distinguish between important and non-important sub-events, focusing on football games as a concrete example. Through the analysis of football match video data, the research delves into how foundation models comprehend contextual importance and pinpoint critical turning points, highlights, or decisive moments within a game. A series of experiments were designed, employing annotated datasets for model training and testing, to quantitatively assess their ability to capture semantically salient information in complex, dynamic scenarios. Evaluation metrics encompass accuracy, recall, F1-score, and human perception consistency, aiming to comprehensively measure the models' understanding of "importance" across various granularities. The study explores the impact of different model architectures, such as Transformer-based vision-language models, and training strategies on recognition performance. Furthermore, it analyzes the limitations of current models when confronted with data bias, scene diversity, and the identification of instantaneous events. The findings provide insights into enhancing the semantic understanding and event summarization capabilities of foundation models in complex multimodal data. It also outlines future directions for optimizing model performance and generalization in multimodal event analysis, with particular relevance to applications such as automated video content analysis, intelligent recommendation systems, and sports commentary generation.