Visuals versus textual scales: Optimizing reliability and user experience in observational assessment of parent-child interaction

Abstract

Objective

Assessment of parent-child interaction by practitioners is of great importance but hindered by a lack of instruments that withstand the constraints daily practice places on usage. Visuals may offer an alternative format. Visualizations were tested on reliability, accuracy, and feasibility in observational assessment of parent-child interaction, as alternatives for textual rating scales.

Methods

In Study 1, 95 students rated parent-child interactions with a video or text anchor scale, and in Study 2, 217 professionals rated the same interactions with a decision tree including visual components or a text anchor scale.

Results

Students using the video anchor scale were less reliable and accurate, slower, and had a less positive user experience than students using the text anchor scale. Professionals using the decision tree did not differ in reliability and were comparable in user experience with professionals using the text anchor scale. Rater accuracy showed similar dependency on quality of parental behavior for both scales: ratings were less accurate when the quality of the parent-child interaction was low, and more accurate when the quality was high. However, professionals were less accurate and slower in using the decision tree than the text anchor scale.

Conclusion

With a first iteration of a decision tree performing the same to or only slightly worse, efforts to further develop decision trees might be worthwhile.

Innovation

These nonintuitive findings underscore the value of experimental testing in assessment design in daily practice.