FLSeg: Enhancing Privacy and Robustness in Federated Learning under Heterogeneous Data via Model Segmentation

Zichun Su, Zhi Lu, Yutong Wu, Renfei Shen, Songfeng Lu; Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2025, pp. 3916-3925

Federated Learning (FL) enables collaborative training of a global model without data sharing, yet it faces critical challenges from privacy leakage and Byzantine attacks. Existing privacy-preserving robust FL frameworks suffer from three key limitations: high computational costs, restricted application of Robust Aggregation Rules (RAR), and inadequate handling of data heterogeneity. To address these limitations, we propose the FLSeg framework, which leverages Segment Exchange and Segment Aggregation to avoid excessive encryption computations while enabling unrestricted use of any RAR. Additionally, a regularization term in local training balances personalization with global model performance, effectively adapting to heterogeneous data. Our theoretical and experimental analyses demonstrate FLSeg's semi-honest security and computational efficiency: it achieves client and server time complexities of O(l) and O(nl), with empirical results showing significantly reduced computational times. Extensive experiments further confirm FLSeg's robustness across diverse heterogeneous and adversarial scenarios.