Weight watchers: NASA-TLX weights revisited

Abstract

Abstract National Aeronautics and Space Administration Task Load Index (NASA-TLX) is a popular method to evaluate mental workload. NASA-TLX assesses mental workload across six load dimensions. When the dimensions are not assumed to be approximately equally important, they are weighted by conducting a pairwise comparison for every dimension pair, followed by the normalisation of weights reflecting the importance of the dimensions. This original NASA-TLX weighting method creates some challenges that are difficult to identify when the weights are being assigned. First, the original NASA-TLX weighting does not allow directly expressing two or more dimensions as equally important. Second, if pairwise comparisons are conducted consistently, there exists only one possible importance order for the dimensions. Third, with consistently conducted pairwise comparisons, a weight of 0.33 is artificially forced on the most important dimension. Swing and Analytic Hierarchy Process weighting methods for eliciting the weights of the dimensions are proposed as a solution to these challenges. The advantages of applying these methods in NASA-TLX are introduced theoretically and demonstrated empirically using data from virtual air combat simulations. The objective of this paper is to help scholars and practitioners to use NASA-TLX in mental workload assessments such that the discussed weighting issues are avoided.