Real-bogus scores for active anomaly detection

Abstract

In the task of anomaly detection in modern time-domain photometric surveys, the primary goal is to identify astrophysically interesting, rare, and unusual objects among a large volume of data. Unfortunately, artifacts — such as plane or satellite tracks, bad columns on CCDs, and ghosts — often constitute significant contaminants in results from anomaly detection analysis. In such contexts, the Active Anomaly Discovery (AAD) algorithm allows tailoring the output of anomaly detection pipelines according to what the expert judges to be scientifically interesting. We demonstrate how the introduction real-bogus scores, obtained from a machine learning classifier, improves the results from AAD. Using labeled data from the SNAD ZTF knowledge database, we train four real-bogus classifiers: XGBoost, CatBoost, Random Forest, and Extremely Randomized Trees. All the models perform real-bogus classification with similar effectiveness, achieving ROC-AUC scores ranging from 0.93 to 0.95. Consequently, we select the Random Forest model as the main model due to its simplicity and interpretability. The Random Forest classifier is applied to 67 million light curves from ZTF DR17. The output real-bogus score is used as an additional feature for two anomaly detection algorithms: static Isolation Forest and AAD. The number of artifacts detected by both algorithms decreases significantly with the inclusion of the real-bogus score in cases where the feature space regions are densely populated with artifacts. However, it remains almost unchanged in scenarios where the overall number of artifacts in the outputs is already small. We conclude that incorporating the real-bogus classifier result as an additional feature in the active anomaly detection pipeline reduces the number of artifacts in the outputs, thereby increasing the incidence of astrophysically interesting objects presented to human experts.