📄 中文摘要
深度神经网络表现出一种简单性偏见，即倾向于选择简单函数而非复杂函数。通过最小描述长度原则，研究将监督学习形式化为最佳双部分无损压缩问题。该理论解释了简单性偏见如何通过模型复杂性（描述假设的成本）与预测能力（描述数据的成本）之间的基本权衡来影响神经网络中的特征选择。随着可用训练数据量的增加，学习者在特征选择上经历从简单的虚假捷径到复杂特征的质变。
从压缩的视角看简单性偏见
出处: A Compression Perspective on Simplicity Bias
发布: 2026年3月30日
📄 English Summary
A Compression Perspective on Simplicity Bias
Deep neural networks exhibit a simplicity bias, a well-documented tendency to prefer simple functions over complex ones. This study formalizes supervised learning as an optimal two-part lossless compression problem through the Minimum Description Length principle. The theory elucidates how simplicity bias influences feature selection in neural networks via a fundamental trade-off between model complexity (the cost of describing the hypothesis) and predictive power (the cost of describing the data). The framework predicts that as the amount of available training data increases, learners transition through qualitatively different features, evolving from simple spurious shortcuts to more complex features.