Cost-Efficient Feature Selection for Horizontal Federated Learning

Abstract

Horizontal federated learning (HFL) exhibits substantial similarities in feature space across distinct clients. However, not all features contribute significantly to the training of the global model. Moreover, the curse of dimensionality delays the training. Therefore, reducing irrelevant and redundant features from the feature space makes training faster and inexpensive. This work aims to identify the common feature subset from the clients in federated settings. We introduce a hybrid approach called Fed-MOFS, ^{1 1}

This manuscript is an extension of Banerjee et al. [1].

utilizing mutual information (MI) and clustering for local FS at each client. Unlike the Fed-FiS, which uses a scoring function for global feature ranking, Fed-MOFS employs multiobjective optimization to prioritize features based on their higher relevance and lower redundancy. This article compares the performance of Fed-MOFS ^{2 2}

We share our code, data, and supplementary copy through https://github.com/DevBhuyan/Horz-FL/blob/main/README.md.

with conventional and federated FS methods. Moreover, we tested the scalability, stability, and efficacy of both Fed-FiS and Fed-MOFS across diverse datasets. We also assessed how FS influenced model convergence and explored its impact in scenarios with data heterogeneity. Our results show that Fed-MOFS enhances global model performance with a 50% reduction in feature space and is at least twice as fast as the FSHFL method. The computational complexity for both approaches is O( $d^{2}$ ), which is lower than the state of the art.