Impact of Content on Subjective Quality of Experience Assessment for 3D Video

Ongoing improvements in the field of visual entertainment may incline users to display 3D video content on various terminal devices provided satisfying Quality of Experience. This study aims to determine whether the cognitive features of appealing and yet uncommon 3D content may obfuscate subjective QoE measurements performed under different bitrates. To test the hypothesis two 3D video databases are compared in terms of a perceived QoE under an innovative scenario. The reference database is GroTruQoE-3D (VQEG) including short artificial clips. The authorial 3D video database DJ3D contains longer clips from feature films with a proved substantial level of cognitive features. The 3D video content features are operationalised by three cognitive attributes (attractiveness, interestingness, 3D effect experience). Gradation of video quality is introduced by streaming at four bitrate levels. The collected subjects' scores are statistically analysed with a stochastic dominance test adjusted to a 5-point Likert scale. The obtained results show that quality assessment scores depend on the intensity of the cognitive attributes of the content. Sequences commonly used in subjective QoE experiments are more vulnerable to the intensity of subjective content attributes (visual attractiveness, interestingness, and 3D effect experience) than sequences from commercial feature films and documentaries. Moreover, it is shown that the test material commonly used in research is assessed higher for lower bitrates. In view of key results QoE researchers should consider to use test material originating from commercially available content to minimize content impact on QoE assessment scores collected during subjective experiment. The research contributes to the QoE best practices by paying attention to 3D cognitive attributes that may obfuscate subjective scores. The innovative scenario for comparing video databases with the stochastic dominance test adjusted to the ordinal scale is proposed. The approach may be useful in a broader context when an emerging service operator wants to ascertain whether subjective QoE tests are not substantially biased by the service novelty.