(Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services

Assuring user engagement has become a key issue for Internet Service Providers and Over-the-Top Providers. How long are users consuming a service? When are they likely to abandon it due to quality problems? Rather than just estimating perceived audiovisual quality, future quality prediction models will also factor in possible user behavior. This contribution presents a novel test method to assess short-term user behavior in web video services, in a controlled living-room-like environment. We show that typical behavioral responses (such as seeking, reloading, or selecting another video) can be elicited, with the real purpose of the test hidden from the viewers. We can also see that when users are not focused on judging quality, their perception of errors changes significantly. This paper highlights the strong impact of laboratory test situations on users' behavior and discusses the challenges revolving around finding valid test methods.