Online federation learning

Abstract

Federated learning (FL) is a popular framework for training a model in a distributed setting, where the data is spread over a network of devices. In traditional FL, the devices communicate with a central server to share their data and update the model parameters. However, this approach can be privacy-invasive, as the devices may need to share sensitive data with the server. online federated learning, addressing algorithm design, theoretical analysis, and practical considerations. We propose novel online learning algorithms tailored for the federated setting, considering the unique challenges posed by distributed data streams and privacy constraints. Through rigorous theoretical analysis, we establish regret bounds that quantify the performance of the OFL algorithms in terms of their cumulative prediction errors compared to the best off-line model. The impact of different communication strategies, data heterogeneity, and privacy settings on the performance of OFL algorithms. We propose adaptive approaches to handle varying environments and demonstrate the algorithms' ability to adapt to dynamic distributed networks effectively. FL algorithm called FedOMD that is designed to be privacy-preserving. FedOMD works by having the devices perform local processing steps before uploading their data to the server. This allows the devices to keep their data private while still allowing the model to be trained effectively. FedOMD achieves sublinear regret bounds that match their centralized counterparts (up to constants) for both convex and strongly convex losses. We also use our regret guarantees to derive high probability excess risk bounds that characterize the generalization ability of FedOMD.