KBVQ-MoE：基于 KLT 引导的 SVD 和偏差修正向量量化的 MoE 大型语言模型

Mixture of Experts (MoE) models have achieved remarkable success by enhancing performance while maintaining computational efficiency through sparse expert activation. However, their large parameter sizes and memory requirements pose significant challenges for deployment in resource-constrained environments. Vector Quantization (VQ) presents a promising solution for ultra-low-bit compression in Large Language Models (LLMs) by utilizing a codebook to map weight vectors to the most similar discrete codewords. Nevertheless, directly applying VQ to MoEs often results in substantial performance degradation due to two critical challenges: (1) redundant representations among experts lead VQ to repeatedly quantize similar representations for each expert, and (2) the lack of effective bias correction mechanisms. To address these issues, a novel KBVQ-MoE approach is proposed, integrating KLT-guided SVD and bias-corrected vector quantization to enhance the performance and efficiency of MoE models in resource-constrained environments.