How Could an Intranet be Like a Friend to Me?: Why Standardized UX Scales Don't Always Fit

Abstract

"I hope that this survey is a joke because it made me laugh so much". This quote is just one example of many negative respondents' reactions gathered during a large-scale user experience (UX) study. Unfortunately, the survey was no joke, rather a well-constructed and validated standardized UX scale. This paper critically reflects on the use and relevance of standardized UX scales for the evaluation of UX in business contexts. We report on a real-world use case where the meCUE questionnaire has been used to assess employees' experience (N=263) with their organization's intranet. Strong users' reactions to the survey's items and statistical analyses both suggest that the scale is unsuitable for the evaluation of business-oriented systems. Drawing on the description of this inadequacy, we discuss the quality of academic UX tools, calling into question the relevance for practice of academic methods.