How Good Is Your Scientific Data Generative Model?

Abstract

Nowadays, leveraging data augmentation methods on helping resolving scientific problems becomes prevailing. And many scientific problems benefit from data augmentation methods build with deep generative models. Yet due to the complexity of the scientific data, commonly used evaluation methods of generative models appear not so suitable for generated scientific data. In this paper, we explore how do we effectively evaluate data augmentation methods for scientific data generative models? To answer this question, we use one example of real world scientific problem to show how we evaluate the quality of the generated data from two domain specific deep generative models. We observe that most existing state-of-art evaluation metrics are incompetent. They either show completely contradicting results or provide inaccurate insight from real data.