Çevrimiçi çoktan seçmeli değerlendirme aracının güvenilirliğinin genellenebilirlik kuramı ile değerlendirilmesi

Abstract

Introduction: Due to the COVID-19 pandemic in our country, it has been decided by the Higher Education Council (CoHE) to apply online measurement and evaluation in higher education, valid only for the pandemic period. Aim: In our study, it is aimed to evaluate the basic analyzes of the multiple-choice assessment tool, which is applied online for 1st year and 3rd commitee students at Süleyman Demirel University, with classical test and generalizability theory. Method: Our study is designed in quantitative research design. The population of the study was determined as first year students actively studying at Süleyman Demirel University Faculty of Medicine (n: 271). In the study, the 3rd commitee applied online to the 1st year students was analyzed with the multiple choice assessment tool SPSS and EduG. Results: When the exam was evaluated over 100 points, the average was 78.5 ± 11.05 (min: 27.4; max: 98.0), variance 122.229, kurtosis -1.196 and skewness 1.683. The average item difficulty was 0.785, the mean discrimination index was 0.262, and the reliability coefficient (KR-20) was 0.902. For the 95-item exam, the G value was calculated as 0.91 and the Phi value as 0.90. Discussion: Our faculty has also ensured the monitoring of measurement-evaluation practices in distance education in line with the recommendations of CoHE and the Association for Evaluation and Accreditation of Medical Education Programs. Although the evaluation of a single assessment tool was considered as a limitation for our study, our study provided valuable information about the review of the basic analysis and assessment of the multiple-choice measurement tools. Based on this experience, we believe that the basic analysis of online measurement and evaluation applications can be preferred for the analysis of measurement tools after the pandemic.