Can We Trust Our Ears? How Accurate and Reliable Are Speech-Language Pathologists' Estimates of Children's Speech Intelligibility?

Abstract

Purpose: This study aimed to investigate the accuracy and reliability of subjective intelligibility estimates of young children's speech made by speech-language pathologists (SLPs) compared to naive listeners, and to examine how the severity of the child's speech impairment influences SLPs' intelligibility estimates.

Method: Eighteen certified SLPs and 18 naive listeners provided intelligibility ratings of single-word speech samples produced by six preschoolers with speech disorders. All listeners rated intelligibility using two different methods: orthographic transcription and subjective estimation of the percentage of words understood. Absolute differences between estimated and transcription intelligibility scores were used to examine accuracy of intelligibility estimates in both listener groups, and intraclass correlations were used to evaluate interrater reliability.

Results: Subjective intelligibility estimates differed from orthographic transcription-based intelligibility scores by 12.4% in the SLP listener group and 18.9% in the naive listener group. Interrater reliability of estimated intelligibility was substantially lower than transcription intelligibility in both listener groups.

Conclusion: Results of this preliminary study suggest that subjective intelligibility estimates by SLPs are not adequately accurate or reliable for measurement of children's speech intelligibility.