Scoring models in competency-based educational assessment

Abstract

Background

Assessments can be broadly classified into two categories based on how they are scored: compensatory or conjunctive. Compensatory models allow for strong performance in one content area to compensate for poor performance in another content area as long as the overall score meets the performance standard. Conjunctive scoring models require examinees to meet performance standards for each specified content area or some portion of content areas.

Methods

This study used data from a large competency-based university to analyze the impact of retroactively switching from a compensatory model to one of four possible conjunctive models.

Results

Results indicated that scoring model has a strong impact on the percent of students who are classified as being competent. The percent of students “mis-classified” as competent varied by college and the number of competencies measured by the test.

Discussion

Ultimately, setting performance standards is a policy decision. Policy considerations for model selection are discussed.

Conclusion

This preliminary research provided some evidence that students being classified as competent under a compensatory model may not display competence in all the areas being measured by the assessments.