Computer-Based Assessment of Non-Cognitive Attributes of Occupational Therapy Students: A Pilot Evaluation

Abstract

Purpose: Holistic admissions tools have been considered for use to diversify student admissions. The ability to screen non-cognitive attributes of applicants is an important element of holistic admissions. The objective of this study was to establish instrument reliability, inter-rater reliability, validity, item discrimination, item difficulty, and bias of the Computerized Assessment of Non-Cognitive Attributes of Health Care Professionals (CANA-HP). Methods: A novel methodology was developed to screen non-cognitive attributes of health profession applicants. Using a cross-sectional design, a convenience sample of students invited for interviews into a Mid-western occupational therapy program were recruited for participation. The 37 participants who consented, completed a demographic survey followed by the 12 question CANA-HP. Results: Open-ended questions had adequate internal reliability, discrimination, and difficulty. Multiple choice questions had low reliability and discrimination. No correlation was found between the CANA-HP and standardized cognitive assessments, except non-science GPA which was significantly correlated to the total open-ended (essay) scores and total overall score. Conclusions and Recommendations: The CANA-HP was not biased toward individuals from varied demographic backgrounds. Predictive validity of this tool and non-cognitive measurements of success are still needed. Occupational therapists in educational settings could consider adding open-ended ethical questions to the application process when interviews are not feasible. Classes in the non-sciences and other non-cognitive markers of success may also help identify students with ‘grit’ and critical reasoning which are important to practicing clinicians