Designing a fusion of visible and infra-red camera streams for remote tower operations

The research project INVIDEON evaluated requirements, technical solutions and the benefit of fusing visible (VIS) and infra-red (IR) spectrum camera streams into a single panorama video stream. In this paper, the design process for developing a usable and accepted fusion is described. As both sensors have strengthens and weaknesses, INVIDEON proposes a fused panorama optimized out of both sensors to be presented to the ATC officer (ATCO). This paper gives an overview of the project and reports results of acceptance and usability of the INVIDEON solution. The process of supporting the definition of requirements by means of rapid prototyping and taking a user-centered approach is described. Main findings of requirements for fusing VIS and IR camera data for remote tower operations are highlighted and set into context with the air traffic controller's tasks. A specific fusion approach was developed within the project and evaluated by means of recorded IR and VIS data. For evaluation, a testbed was set up at a regional airport and data representing different visibility conditions were selected out of 70 days data recordings. Five air traffic controllers participated in the final evaluation. Subjective data on perceived usability, situational awareness and trust in automation was assessed. Furthermore, qualitative data on HMI design and optimization potential from debriefings and comments was collected and clustered.