Reliability of Variables of Consonant Production and Velopharyngeal Competence in 10-year-olds with Cleft Palate Using Multiple Raters.

Objective: To assess the reliability of speech data and speech-related quality indicators in the Swedish quality registry for cleft lip and palate (CLP) at 10 years of age.

Design: Retrospective study.

Setting: University hospitals.

Participants: One hundred twenty-one 10-year-olds with unilateral or bilateral CLP.

Main outcome measures: Six independent raters reassessed audio recordings for comparison with registry data. For calculation of agreement, the single measures intraclass correlation coefficient (ICC) was used for percentage of consonants correct (PCC) and non-oral speech errors, quadratic weighted kappa for velopharyngeal competence (VPC), and percentage agreement and kappa for quality indicators. The results of the three to four raters with the highest intra-rater and inter-rater reliability were used for comparison with registry data.

Results: There was excellent agreement between registry data and reassessments for PCC (ICC, 0.93) and percentage of non-oral errors (ICC, 0.80). For VPC, one rater and registry data had good agreement (k, 0.704); the remaining cases had fair agreement (k, 0.476-0.554). The percentage agreement between registry data and reassessments for quality indicators ranged from fair to excellent. When calculated with kappa, agreement was good to excellent (mean of all k values, 0.67-0.70).

Conclusions: The CLP registry variables PCC and percentage of non-oral errors and the quality indicators without non-oral speech errors and competent/marginally incompetent velopharyngeal function are reliable for use in clinical audits and research of 10-year-olds. The three-tier ratings of VPC have weaker reliability but can still be useful in more detailed analyses if interpreted with caution.