Prediction of Sensor Performance Required for Reliable Aircraft Target Discrimination

Abstract

For new military aircraft a specification of sensor characteristics and performance is required at an early stage in the design cycle, well before testing of a prototype. In the early days of military aviation the Johnson criteria [1] were used to determine the sensor resolution required for target recognition by a human. In the present day sensor data are processed by computer using various Automatic Target Recognition (ATR) algorithms. However there is no accepted method for predicting the sensor resolution and SNR required for reliable ATR and hence there is risk that any chosen sensor may fail to support the required ATR performance. This paper reports a study into the use of publicly-available CAD models for aircraft to address this requirement. The study considers the worst-case confusion between two views of 15 different aircraft types. For simplicity only rotations by an angle θ about the Z (vertical) axis are considered. Firstly the sensor resolution and noise level is fixed. Then for each aircraft type and view angle an ensemble of synthetic silhouettes are generated. Using these ensembles, a-posteriori distributions of 5 standard scale-invariant shape features (eccentricity, orientation, solidity, circularity and bounding box aspect ratio) are estimated for each view angle θ. The performance of ATR at the given resolution and noise level is predicted by estimating the Bayes Error Rate [2] when deciding between each aircraft type and the 14 non-matching types using these features. The worst-case confusion in terms of erroneous aircraft type and view angle is identified. The sensor resolution is then changed and the above process repeated to investigate the effect of varying sensor resolution on performance. As expected, high sensor resolution leads to low probability of misclassification, even in the worst-case. Reduction in resolution and increasing noise level causes the Bayes Error Rate to rise quickly. The Bayes Error Rate gives a fundamental limit to the reliability of classification, irrespective of the actual type of classification algorithm used. The predictions from the model are confirmed by testing against a standard classifier for specific discrimination examples. Further development of the approach presented is expected to yield a method for specifying sensor resolution requirements for specific ATR problems.