Multimodal Information Presentation in Support of NextGen Operations

ABSTRACT Objective: This study examined the effectiveness of visual, auditory, tactile, and redundant auditory-visual information presentation in the context of a medium-fidelity ‘Next Generation Air Transportation System’ (NextGen) flight simulation. Background: Data overload, especially in the visual channel, and associated breakdowns in monitoring represent a major challenge in aviation. These problems are expected to worsen with NextGen, which will require pilots to manage increased amounts of data and adopt new responsibilities. The introduction of multimodal interfaces (interfaces that distribute information across multiple sensory channels) has been proposed as a means to offload the overburdened visual channel and thus address data overload. Method: Experienced commercial airline pilots completed 2 scenarios using a medium-fidelity flight simulator. For each scenario, NextGen tasks and events were presented either using technology that is currently available (visual and auditory displays) or technology proposed as part of NextGen design concepts (i.e., tactile and redundant displays). Performance was measured based on response time and accuracy. Results: Faster responses were observed for redundant displays, compared to either vision or audition alone. No significant benefit of redundancy was found for accuracy and workload did not mediate redundancy effects. For traffic events, there were faster response times with tactile displays, but higher response accuracy with auditory displays. Conclusion: The findings from this research add to the knowledge base in multimodal information processing and can inform the design of displays for NextGen operations.