MVI-DCGAN Insights into Heterogenous EO and Passive RF Fusion

Abstract

As technology trends towards automation, deep neural network (DNN) based methods become more and more desirable from a technological, economical, and societal standpoint. However, owing to the way that these black box technologies operate, it can be difficult to troubleshoot potential errors, especially when dealing with data that the human mind cannot intuitively understand. For this reason, the use of explainable artificial intelligence (XAI) is integral to obtaining interpretability and understanding of these systems' techniques. The paper explores some of the known uses of XAI in Generative Adversarial Networks (GANs); i.e., in processing electro-optical (EO) and passive radiofrequency (Passive RF) data to achieve heterogenous sensor fusion. GANs are capable of generating realistic images, music text, and other forms of data, and the use of deep convolutional generative adversarial networks (DCGANs) to process such information provides “richer” corrective feedback from which the model can train from. Using the DCGAN approach, tone can provide visualizations from different types of neural networks and use them as a training source for the multiple visualizations input (MVI) DCGAN. The MVI-DCGAN uses these visualizations in order to track the vehicle target and further differentiate between other overlay visualization data and the generated overlay input visualizations. The paper demonstrates multiple sources of visualization input from different neural networks for the training of the MVI-DCGAN for a more robust training and directing the discriminator towards focusing on the P-RF aspects of the visualizations.