Development of Synthetic Data Benchmarks for Evaluating Feature Selection Algorithms

Abstract

The primary objective of this paper is to present a set of synthetically generated datasets as a benchmark for evaluating feature selection algorithms (FSAs). The use of synthetic datasets is encouraged because of their utility in controlling data parameters, including the exact number of relevant, redundant, and irrelevant features. This paper proposes four numeric datasets with several sources of inspiration, namely based on geometric objects, trigonometric equations and multi-cut linear combinations. These synthetically generated datasets come with a fixed number of relevant, redundant and irrelevant features, which are then evaluated using feature selection algorithms currently popular within industry and academia. This highlights the function of these datasets as benchmarks for future researchers in the field of feature selection. Accordingly, the datasets will also be made available through GitHub for use as evaluation metrics, whilst the code is made available to be modified according to the application for the researcher. This may include research into the performance of FSAs, the development of new synthetic data, and beyond.