异质时间常数提高平衡传播的稳定性

📄 中文摘要

平衡传播(EP)是一种生物上可行的替代反向传播的神经网络训练方法。然而,现有的EP模型使用统一的标量时间步长dt,这在生物学上对应于神经元之间的膜时间常数的异质性。研究提出了异质时间步长(HTS)的方法,通过为每个神经元分配来自生物学动机分布的特定时间常数,来改进EP。结果表明,HTS在保持竞争性任务性能的同时,提高了训练的稳定性。这些结果表明,纳入异质时间动态增强了平衡传播的生物现实性和鲁棒性。

📄 English Summary

Heterogeneous Time Constants Improve Stability in Equilibrium Propagation

Equilibrium propagation (EP) serves as a biologically plausible alternative to backpropagation for training neural networks. Existing EP models utilize a uniform scalar time step dt, which corresponds to a heterogeneous membrane time constant across neurons in biological systems. This research introduces heterogeneous time steps (HTS) for EP by assigning neuron-specific time constants drawn from biologically motivated distributions. The findings demonstrate that HTS enhances training stability while maintaining competitive task performance. These results indicate that incorporating heterogeneous temporal dynamics improves both the biological realism and robustness of equilibrium propagation.

Powered by Cloudflare Workers + Payload CMS + Claude 3.5

数据源: OpenAI, Google AI, DeepMind, AWS ML Blog, HuggingFace 等