Scenario testing methodology for the assessment of screening technologies

A range of advanced screening technologies has been developed to detect substances or objects that can pose a threat if carried on commercial aircraft. However, outside of technical testing in a laboratory setting, very little testing has been conducted to estimate their performance in an airport setting. This paper presents protocols developed by DSTO to assess the performance of security devices to detect contraband concealed on a person. These protocols are classified as scenario evaluations because they test the devices in a controlled environment that mimics crucial aspects of the airport environment (e.g., baggage and mock passengers) using trained operators. Such evaluations provide a range of data including reliability rates and timing estimates as well as detection and error rates which are very difficult to determine in either technical or full operational evaluations. The ability to detect and recognise dangerous objects at a safe distance is a very important task in a number of other defence, police and security applications and these protocols may be useful in other environments.