Classification framework to identify similar visual scan paths using multiple similarity metrics

Abstract

Analyzing visual scan paths, the time-ordered sequence of eye fixations and saccades, can help us understand how operators visually search the environment before making a decision. To analyze and compare visual scan paths, prior studies have used metrics such as string edit similarity, which considers the order used to inspect areas of interest (AOIs), as well as metrics that consider the AOIs shared between visual scan paths. However, to identify similar visual scan paths, particularly in tasks and environments in which operators may apply variations of a common underlying visual scanning behavior, using solely one similarity metric might not be sufficient. In this study, we introduce a classification framework using a combination of the string edit algorithm and the Jaccard coefficient similarity. We applied our framework to the visual scan paths of nine tower controllers in a high-fidelity simulator when a “clear-to-take-off” clearance was issued. The classification framework was able to provide richer and more meaningful classifications of the visual scan paths compared to the results when using either the string edit algorithm or Jaccard coefficient similarity.