📄 中文摘要
提出了一种名为AMOR（自适应元认知输出路由器）的混合架构，该架构结合了状态空间模型（SSMs）和稀疏注意力机制，旨在提高信息检索的效率。AMOR在SSM主干网络不确定时（通过预测熵衡量）动态地激活稀疏注意力。与标准变换器相比，AMOR通过从SSM隐藏状态中投影键和值（Ghost KV），重用SSM的O(n)计算，避免了每层都需要O(n^2)的注意力计算。在小规模合成检索任务中，AMOR的表现优于传统的SSM和变换器。
何时快速思考与慢速思考？AMOR：基于熵的元认知门用于动态状态空间模型注意力切换
📄 English Summary
When to Think Fast and Slow? AMOR: Entropy-Based Metacognitive Gate for Dynamic SSM-Attention Switching
The study proposes AMOR (Adaptive Metacognitive Output Router), a hybrid architecture that integrates State Space Models (SSMs) with a sparse attention mechanism to enhance retrieval efficiency. AMOR dynamically engages sparse attention when the SSM backbone exhibits uncertainty, measured by prediction entropy. Compared to standard transformers, AMOR gains computational efficiency by projecting keys and values from SSM hidden states (Ghost KV), reusing the SSM's O(n) computation and avoiding the O(n^2) attention requirement at each layer. In small-scale synthetic retrieval tasks, AMOR outperforms both traditional SSMs and transformers.