FedAdaVR: 有限客户端参与下自适应方差缩减的鲁棒联邦学习

Federated learning (FL) encounters substantial challenges due to heterogeneity, leading to gradient noise, client drift, and partial client participation errors. The latter is particularly pervasive but remains insufficiently addressed in current literature. FedAdaVR proposes a novel FL algorithm specifically designed to mitigate heterogeneity issues arising from sporadic client participation by integrating an adaptive optimizer with a variance reduction mechanism. The core innovation of FedAdaVR lies in its ability to dynamically adjust learning rates and variance reduction strengths, adapting to varying client participation rates across different communication rounds. When the number of participating clients is small, FedAdaVR effectively suppresses the increase in gradient variance, preventing oscillations and divergence during model training. By incorporating statistical information about local data distributions, FedAdaVR intelligently assigns appropriate weights to each client and optimizes the global model aggregation strategy. Specifically, FedAdaVR introduces an adaptive variance estimation method based on historical gradient information, enabling accurate estimation and compensation of gradient variance even in complex scenarios involving frequent client churn and non-IID data distributions. Experimental evaluations demonstrate that FedAdaVR significantly accelerates convergence and achieves higher model accuracy compared to existing state-of-the-art FL algorithms across various heterogeneous datasets and diverse client participation patterns. Its robustness and performance advantages are particularly pronounced under extremely low client participation rates. FedAdaVR offers an efficient and practical solution to the persistent problem of uncertain client participation in federated learning, thereby facilitating its deployment in real-world applications.