有效的推理链降低内在维度

📄 中文摘要

链式思维(CoT)推理及其变体显著提升了语言模型在复杂推理任务上的表现,但不同策略如何促进泛化的具体机制仍不清楚。当前的解释通常指向增加的测试时计算或结构指导,但在这些因素与泛化之间建立一致、可量化的联系仍然具有挑战性。本研究确定了内在维度作为量化推理链有效性的度量标准。内在维度量化了在特定任务上达到给定准确率阈值所需的最小模型维度数量。通过保持模型架构的稳定性,研究揭示了推理链的有效性与内在维度之间的关系。

📄 English Summary

Effective Reasoning Chains Reduce Intrinsic Dimensionality

Chain-of-thought (CoT) reasoning and its variants have significantly enhanced the performance of language models on complex reasoning tasks, yet the precise mechanisms by which different strategies facilitate generalization remain unclear. Current explanations often attribute this to increased test-time computation or structural guidance, but establishing a consistent and quantifiable link between these factors and generalization is challenging. This study identifies intrinsic dimensionality as a quantitative measure for characterizing the effectiveness of reasoning chains. Intrinsic dimensionality quantifies the minimum number of model dimensions required to achieve a given accuracy threshold on a specific task. By maintaining the stability of the model architecture, the research reveals the relationship between the effectiveness of reasoning chains and intrinsic dimensionality.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等