使用对数数据表示的卷积神经网络

A novel approach utilizing logarithmic data representation in convolutional neural networks significantly reduces model size and enhances device efficiency. This technique allows large models to be compressed to merely 3-bit numbers with negligible performance degradation, a surprising outcome. The uneven distribution of most learned values makes logarithmic representation a better fit, thereby eliminating the need for bulky hardware multipliers, which in turn lowers device cost and power consumption. Training with logarithmic representation effectively addresses numerical overflow and precision loss issues commonly encountered with traditional fixed-point representations during quantization, especially when dealing with weights and activation values that have a wide dynamic range. This optimization not only boosts AI computing capabilities on edge devices but also offers a new pathway for deploying low-power, high-performance neural networks. It holds promise for enabling smarter and more efficient applications in resource-constrained environments such as mobile devices.