-
[pdf]
[supp]
[arXiv]
[bibtex]@InProceedings{Soni_2024_CVPR, author = {Soni, Sunny and Saeed, Aaqib and Asano, Yuki M.}, title = {Federated Learning with a Single Shared Image}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, month = {June}, year = {2024}, pages = {7782-7790} }
Federated Learning with a Single Shared Image
Abstract
Federated Learning (FL) enables multiple machines to collaboratively train a machine learning model without sharing of private training data. Yet especially for heterogeneous models a key bottleneck remains the transfer of knowledge gained from each client model with the server. One popular method FedDF uses distillation to tackle this task with the use of a common shared dataset on which predictions are exchanged. However in many contexts such a dataset might be difficult to acquire due to privacy and the clients might not allow for storage of a large shared dataset. To this end in this paper we introduce a new method that improves this knowledge distillation method to only rely on a single shared image between clients and server. In particular we propose a novel adaptive dataset pruning algorithm that selects the most informative crops generated from only a single image. With this we show that federated learning with distillation under a limited shared dataset budget works better by using a single image compared to multiple individual ones. Finally we extend our approach to allow for training heterogeneous client architectures by incorporating a non-uniform distillation schedule and client-model mirroring on the server side.
Related Material