Leveraging open science machine learning challenges for data constrained planetary mission instruments

Abstract

We set up two open-science machine learning (ML) challenges focusing on building models to automatically analyze mass spectrometry (MS) data for Mars exploration. ML challenges provide an excellent way to engage a diverse set of experts with benchmark training data, explore a wide range of ML and data science approaches, and identify promising models based on empirical results, as well as to get independent external analyses to compare to those of the internal team. These two challenges were proof-of-concept projects to analyze the feasibility of combining data collected from different instruments in a single ML application. We selected mass spectrometry data from 1) commercial instruments and 2) the Sample Analysis at Mars (SAM, an instrument suite that includes a mass spectrometer subsystem onboard the Curiosity rover) testbed. These challenges, organized with DrivenData, gathered more than 1,150 unique participants from all over the world, and obtained more than 600 solutions contributing powerful models to the analysis of rock and soil samples relevant to planetary science using various mass spectrometry datasets. These two challenges demonstrated the suitability and value of multiple ML approaches to classifying planetary analog datasets from both commercial and flight-like instruments. We present the processes from the problem identification, challenge setups, and challenge results that gathered creative and diverse solutions from worldwide participants, in some cases with no backgrounds in mass spectrometry. We also present the potential and limitations of these solutions for ML application in future planetary missions. Our longer-term goal is to deploy these powerful methods onboard the spacecraft to autonomously guide space operations and reduce ground-in-the-loop reliance.