cp3-bench: a tool for benchmarking symbolic regression algorithms demonstrated with cosmology

Abstract

We introduce cp3-bench, a tool for comparing/benching symbolic regression algorithms, which we make publicly available at https://github.com/CP3-Origins/cp3-bench. In its current format, cp3-bench includes 12 different symbolic regression algorithms which can be automatically installed as part of cp3-bench. The philosophy behind cp3-bench is that is should be as user-friendly as possible, available in a ready-to-use format, and allow for easy additions of new algorithms and datasets. Our hope is that users of symbolic regression algorithms can use cp3-bench to easily install and compare/bench an array of symbolic regression algorithms to better decide which algorithms to use for their specific tasks at hand. To introduce and motivate the use of cp3-bench we present a small benchmark of 12 symbolic regression algorithms applied to 28 datasets representing six different cosmological and astroparticle physics setups. Overall, we find that most of the benched algorithms do rather poorly in the benchmark and suggest possible ways to proceed with developing algorithms that will be better at identifying ground truth expressions for cosmological and astroparticle physics datasets. Our demonstration benchmark specifically studies the significance of dimensionality of the feature space and precision of datasets. We find both to be highly important for symbolic regression tasks to be successful. On the other hand, we find no indication that inter-dependence of features in datasets is particularly important, meaning that it is not in general a hindrance for symbolic regression algorithms if datasets e.g. contain both z and H(z) as features. Lastly, we note that we find no indication that performance of algorithms on standardized datasets are good indicators of performance on particular cosmological and astrophysical datasets. This suggests that it is not necessarily prudent to choose symbolic regression algorithms based on their performance on standardized data. Instead, a more robust approach is to consider a variety of algorithms, chosen based on the particular task at hand that one wishes to apply symbolic regression to.