DecHW: 利用二阶信息的异构去中心化联邦学习

Decentralized Federated Learning (DFL) is a serverless collaborative machine learning paradigm where devices directly collaborate with neighboring devices to exchange model information for learning a generalized model. However, variations in individual experiences and different levels of device interactions lead to data and model initialization heterogeneities across devices. Such heterogeneities result in discrepancies in local model parameters, which hinder convergence and degrade performance when first-order gradient-based methods are employed. To address this issue, DecHW (Decentralized Heterogeneous Federated Learning with Second-Order Information) is proposed. DecHW leverages second-order information to more effectively handle heterogeneity. Specifically, it introduces an adaptive learning rate adjustment mechanism based on the Hessian matrix, enabling each device to dynamically adjust its model update step size according to the curvature information of its local data distribution. This approach better captures non-linear relationships between data from different devices, leading to faster convergence and higher model accuracy in heterogeneous environments. Furthermore, DecHW introduces a novel neighbor aggregation strategy that not only considers parameter differences among neighbor models but also incorporates a weight allocation mechanism based on second-order information, allowing neighbors with higher information gain to contribute more significantly to the global model update. Experimental results demonstrate that DecHW achieves significant performance improvements compared to existing decentralized federated learning methods across various heterogeneous datasets and network topologies, particularly in terms of convergence speed and final model accuracy. This method provides an effective solution for addressing data and model heterogeneity in decentralized federated learning, opening new avenues for future research.