The journey towards the best quality of MCQs test, a success story in a medical school

Abstract

Multiple choice question (MCQ) test is the most popular assessment tool that used in medical schools, faculty training on construction of multiple choice questions and peer review are the main factors that contributing to have a good quality MCQ test. This study aimed to evaluate the effect of these two factors on the quality of MCQ test. An extensive training program was conducted for the staff members who were participated in teaching and assessment of undergraduate medical students of clinical biochemistry and psychiatric courses. Post-test item analysis was used to compare pre-training with post-training item analysis parameters. Chi-square test was applied, level of significance was 5%. Regarding difficulty index, it showed great improvement in number of questions which lie in acceptable range of difficulty from 16%, 25% before the training program to 61%,42% after the training program for psychiatry and clinical biochemistry courses respectively and this is   significantly different (P0.05). In psychiatry course the number of questions with negatively discrimination power had been decreased from 36% before training program to 10% after it (P value 0.05). No significant improvement noticed in biochemistry course regarding this aspect. Both courses were showed improvement in the function of distracters, the percentage of test items where all distracters were functioning was increased from 11% to 41% in psychiatry course (P value=0.0002), and from 12% to 46%, in biochemistry course (P value=0.0001). The two courses which included in the study showed increased reliability after the training program and peer review processes, but this improvement is not statistically significant. A dramatic improvement was observed in the quality of MCQs after peer review and training program. We recommend adopting training program on construction of MCQs to be as teaching license for all medical school staff members in Saudi Arabia.