Adapting a style based generative adversarial network to create images depicting cleft lip deformity.

Abstract

Training a machine learning system to evaluate any type of facial deformity is impeded by the scarcity of large datasets of high-quality, ethics board-approved patient images. We have built a deep learning-based cleft lip generator called CleftGAN designed to produce an almost unlimited number of high-fidelity facsimiles of cleft lip facial images with wide variation. A transfer learning protocol testing different versions of StyleGAN as the base model was undertaken. Data augmentation maneuvers permitted input of merely 514 frontal photographs of cleft-affected faces adapted to a base model of 70,000 normal faces. The Frechet Inception Distance was used to measure the similarity of the newly generated facial images to the cleft training dataset. Perceptual Path Length and the novel Divergence Index of Normality measures also assessed the performance of the novel image generator. CleftGAN generates vast numbers of unique faces depicting a wide range of cleft lip deformity with variation of ethnic background. Performance metrics demonstrated a high similarity of the generated images to our training dataset and a smooth, semantically valid interpolation of images through the transfer learning process. The distribution of normality for the training and generated images were highly comparable. CleftGAN is a novel instrument that generates an almost boundless number of realistic facial images depicting cleft lip. This tool promises to become a valuable resource for the development of machine learning models to objectively evaluate facial form and the outcomes of surgical reconstruction.