Development and validation of OSATS tool for formative assessment of dental students' endodontic access cavity opening skills

Aim

Endodontic Access Cavity (EAC) opening is a critical step in Root Canal Treatment. yet, a robust formative assessment tool for this procedure is lacking in the literature. Therefore, the overarching goal of our study was to develop and validate an OSATS tool specifically for the formative assessment of EAC opening skill. The specific objectives of our study were (1) to evaluate the ability of the performance ratings on the newly developed OSATS tool in differentiating between different levels of dental students (Final year students [FY] and House officers [HO]) in the EAC opening procedure on extracted natural teeth and (2)-to determine the inter-rater reliability of the newly developed OSATS tool in assessing the EAC opening procedure.

Methodology

Conducted over 6 months in 2024, the tool was developed collaboratively by an endodontic expert (EE) and two medical educationists, following an extensive literature review. Performance rubrics and guidelines were devised, and the tool underwent two review cycles by 11 EEs, achieving satisfactory Content Validity Index scores (Scale-CVI: 0.9, Universal agreement: 0.7). Two trained assessors applied the tool to evaluate 30 students (15 FY, 15 HO) across three tooth types (anterior, premolar, molar), generating 180 OSATS forms. Data analysis in IBM SPSS v24 assessed inter-rater reliability (intraclass correlation), internal consistency (Cronbach's alpha) and group performance differences (Mann–Whitney U, Kruskal-Wallis H-tests).

Results

Inter-rater reliability (0.83) and internal consistency (Cronbach's alpha = 0.83) were strong. HO significantly outperformed FY (p < .05) on pre-access, initial access and final access sub-scales, with superior global ratings on premolars (p = .003). HO exhibited significant performance differences across sub-scales for anterior, premolar and molar teeth (p < .05), excelling on premolars and anteriors compared to molars.

Conclusions

The newly developed OSATS tool demonstrated strong internal consistency and inter-rater reliability. It effectively differentiated performance ratings across various student levels and tooth types, establishing itself as a reliable and effective instrument for formative assessment of access cavity opening skill in endodontics.