Personalized federated self supervised multitask learning

Abstract

Federated learning has been a popular research subject in allowing diverse people and businesses to collaborate on machine learning model training while maintaining privacy. Additionally, in today’s world there is an abundance of unlabelled data while labelled data is scarce for model training. Given the constraints of data minimization of federated learning, this paper approaches the problem of utilizing unlabelled data at clients to learn better representations through the novel technique of self supervised learning in a multitask fashion. This allows us to combine the pretext and downstream phases through multitask learning techniques like hard parameter sharing at each client. Extending further, given the statistical heterogeneity prevalent in real world data distributions, we adopt our novel methodology to Personalized Federated Learning through techniques of multitask learning, parameter decoupling & partially unsupervised training. Finally, this paper also assesses its extension using hypernetworks