📄 中文摘要
尽管最先进的视觉语言模型（VLMs）在推理能力上表现出色，但它们在多语言数学推理方面的性能尚未被充分探索，尤其是在与人类表现进行比较时。为了弥合这一差距，M3Kang数据集被引入，这是首个大规模多语言、多模态数学推理数据集，专为评估VLMs而设计。该数据集源自“袋鼠数学竞赛”，这是全球规模最大的数学竞赛，其题目涵盖了从小学到高中的各个难度级别和数学领域。M3Kang的构建旨在提供一个多样化且具有挑战性的测试平台，以全面评估VLMs在理解和解决复杂数学问题方面的能力。数据集中的问题不仅包含文本描述，还融入了丰富的视觉元素，例如几何图形、图表和公式，这要求模型能够同时处理和整合来自不同模态的信息。此外，M3Kang强调多语言特性，题目被翻译成多种主流语言，以测试VLMs在跨语言环境下进行数学推理的鲁棒性。数据集的构建过程严格遵循竞赛标准，确保了问题的质量和难度符合实际教育场景。通过M3Kang，研究人员可以深入分析VLMs在多语言、多模态环境下进行数学推理的优势与劣势，识别模型在理解数学概念、逻辑推理、符号操作以及视觉信息处理方面的潜在瓶颈。最终，M3Kang有望推动下一代能够更有效地进行跨语言、跨模态数学推理的AI模型的发展，使其在教育、科学研究等领域发挥更大作用。
📄 English Summary
M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
State-of-the-art vision-language models (VLMs) exhibit robust reasoning capabilities, yet their performance in multilingual mathematical reasoning remains largely underexplored, particularly when benchmarked against human proficiency. To address this critical gap, M3Kang is introduced, pioneering as the first massively multilingual, multimodal mathematical reasoning dataset specifically tailored for VLMs. This comprehensive dataset is meticulously derived from the Kangaroo Math Competition, recognized globally as the largest mathematics contest, thereby encompassing a broad spectrum of difficulty levels and mathematical domains from elementary to high school curricula. The design of M3Kang aims to furnish a diverse and challenging evaluation platform to thoroughly assess VLMs' aptitude in comprehending and resolving intricate mathematical problems. Beyond mere textual descriptions, the problems within the dataset integrate rich visual components, including geometric figures, graphs, and complex equations, necessitating that models process and synthesize information concurrently from disparate modalities. Furthermore, M3Kang emphasizes a profound multilingual dimension, with problems meticulously translated into multiple major languages to scrutinize the robustness of VLMs in cross-lingual mathematical reasoning contexts. The dataset's construction adheres strictly to competition standards, guaranteeing the quality and appropriate difficulty of the problems for realistic educational scenarios. Utilizing M3Kang, researchers can conduct in-depth analyses of VLMs' strengths and weaknesses in multilingual, multimodal mathematical reasoning, pinpointing potential bottlenecks in concept understanding, logical inference, symbolic manipulation, and visual information processing. Ultimately, M3Kang is poised to catalyze the advancement of next-generation AI models capable of more effective cross-lingual and cross-modal mathematical reasoning, thereby expanding their utility across educational, scientific research, and other critical domains.