Mobile-based Assessment of Entrustable Professional Activities in Urology Training: Implementation and Outcomes

Abstract

Background and objective

Urology residency programs often lack consistent feedback mechanisms and tracking of an individual’s progress. Competency-based medical education, using entrustable professional activities (EPAs), offers a solution but faces challenges in implementation. The aim of this study was to implement and assess a smartphone application for real-time EPA assessments in urology residency programs.

Methods

A mobile application was introduced to 13 urology residents and ten supervisors at a Swiss training institution. Assessment characteristics were analyzed using descriptive statistics. Agreement between trainees and supervisors on task complexity and supervision levels was evaluated using Cohen’s and Fleiss’ κ metrics.

Key findings and limitations

Over a period of 7 mo, 246 EPA assessments were recorded, of which 214 assessments were completed, representing a completion rate of 86%. Procedural EPAs accounted for 92% of the assessments, and nonprocedural EPAs for 8%. Cohen’s κ indicated moderate agreement for task complexity (κ = 0.56) and supervision levels (κ = 0.55). Higher agreement was observed when trainees were rated competent in supervising others (κ = 0.71). Limitations include the focus on procedural tasks and the small sample size.

Conclusions and clinical implications

A mobile application can effectively facilitate real-time EPA assessments in urology training, promoting frequent feedback conversations and efficient tracking of resident progress. However, greater emphasis on nonprocedural EPAs is needed.

Patient summary

We tested a mobile phone app that provides real-time feedback to urology residents and found that it enhanced their training experience. While the app effectively tracks progress in carrying out procedures, more focus is needed on developing nonprocedural skills such as patient counseling.