An approach for evaluating assisted target detection technology

The literature is replete with assisted target recognition (ATR) techniques, including methods for ATR evaluation. Yet, relatively few methods find their way to use in practice. Part of the problem is that the evaluation of an ATR may not go far enough in characterizing its optimal use in practice. For example, a thorough understanding of a method's operating conditions is crucial, e.g., performance across different sensor capabilities, scene context, target occlusions, etc. This paper describes a process for a rigorous evaluation of ATR performance, including a sensitivity analysis. Ultimately, an ATR algorithm is deemed valuable if it is actually utilized in practice by users. Thus, quantitative analysis alone is not necessarily sufficient. Qualitative user assessment derived from user testing, surveys, and questionnaires is often needed to provide a more complete interpretation of an evaluation for a particular method. We demonstrate our ATR evaluation process using methods that perform target detection of civilian vehicles.