Real-like synthetic sperm video generation from learned behaviors

Abstract

Computer-assisted sperm analysis is an open research problem, and a main challenge is how to test its performance. Deep learning techniques have boosted computer vision tasks to human-level accuracy, when sufficiently large labeled datasets were provided. However, when it comes to sperm (either human or not) there is lack of sufficient large datasets for training and testing deep learning systems. In this paper we propose a solution that provides access to countless fully annotated and realistic synthetic video sequences of sperm. Specifically, we introduce a parametric model of a spermatozoon, which is animated along a video sequence using a denoising diffusion probabilistic model. The resulting videos are then rendered with a photo-realistic appearance via a style transfer procedure using a CycleGAN. We validate our synthetic dataset by training a deep object detection model on it, achieving state-of-the-art performance once validated on real data. Additionally, an evaluation of the generated sequences revealed that the behavior of the synthetically generated spermatozoa closely resembles that of real ones.