Lena Rössler, Manfred Herrmann, Annette Wiegand, Philipp Kanzow
{"title":"总结性考试中多项选择题的使用情况:德国牙科本科生培训项目问卷调查。","authors":"Lena Rössler, Manfred Herrmann, Annette Wiegand, Philipp Kanzow","doi":"10.2196/58126","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>Multiple-choice examinations are frequently employed among German dental schools. However, details regarding the used item types and applied scoring methods are lacking.</p><p><strong>Objective: </strong>We aimed to gain an insight into the current usage of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programmes.</p><p><strong>Methods: </strong>A paper-based 10-item questionnaire regarding the employed assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the Deans of Studies and to the Heads of Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using Fisher's exact test (<i>P</i><.05).</p><p><strong>Results: </strong>The response rate amounted to 90.0% (27/30 dental schools). All respondent dental schools employed multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70.4% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (88.9%) which accounted for the largest number of items in about half of the dental schools. Further item types (eg, conventional multiple-select items, Multiple-True-False, Pick-N) were only used by fewer dental schools (≤66.7%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit, requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (73.7%, 14/19 vs. 50.0%, 4/8). However, this difference was statistically not significant (<i>P</i>=.375). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (55.6%, 15/27). Not a single school employed alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (55.6%, 15/27).</p><p><strong>Conclusions: </strong>Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.</p><p><strong>Clinicaltrial: </strong></p>","PeriodicalId":36236,"journal":{"name":"JMIR Medical Education","volume":" ","pages":""},"PeriodicalIF":3.2000,"publicationDate":"2024-05-07","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Usage of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programmes.\",\"authors\":\"Lena Rössler, Manfred Herrmann, Annette Wiegand, Philipp Kanzow\",\"doi\":\"10.2196/58126\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>Multiple-choice examinations are frequently employed among German dental schools. However, details regarding the used item types and applied scoring methods are lacking.</p><p><strong>Objective: </strong>We aimed to gain an insight into the current usage of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programmes.</p><p><strong>Methods: </strong>A paper-based 10-item questionnaire regarding the employed assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the Deans of Studies and to the Heads of Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using Fisher's exact test (<i>P</i><.05).</p><p><strong>Results: </strong>The response rate amounted to 90.0% (27/30 dental schools). All respondent dental schools employed multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70.4% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (88.9%) which accounted for the largest number of items in about half of the dental schools. Further item types (eg, conventional multiple-select items, Multiple-True-False, Pick-N) were only used by fewer dental schools (≤66.7%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit, requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (73.7%, 14/19 vs. 50.0%, 4/8). However, this difference was statistically not significant (<i>P</i>=.375). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (55.6%, 15/27). Not a single school employed alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (55.6%, 15/27).</p><p><strong>Conclusions: </strong>Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.</p><p><strong>Clinicaltrial: </strong></p>\",\"PeriodicalId\":36236,\"journal\":{\"name\":\"JMIR Medical Education\",\"volume\":\" \",\"pages\":\"\"},\"PeriodicalIF\":3.2000,\"publicationDate\":\"2024-05-07\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"JMIR Medical Education\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.2196/58126\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION, SCIENTIFIC DISCIPLINES\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"JMIR Medical Education","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2196/58126","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION, SCIENTIFIC DISCIPLINES","Score":null,"Total":0}
Usage of Multiple-Choice Items in Summative Examinations: Questionnaire Survey Among German Undergraduate Dental Training Programmes.
Background: Multiple-choice examinations are frequently employed among German dental schools. However, details regarding the used item types and applied scoring methods are lacking.
Objective: We aimed to gain an insight into the current usage of multiple-choice items (ie, questions) in summative examinations in German undergraduate dental training programmes.
Methods: A paper-based 10-item questionnaire regarding the employed assessment methods, multiple-choice item types, and applied scoring methods was designed. The pilot-tested questionnaire was mailed to the Deans of Studies and to the Heads of Department of Operative/Restorative Dentistry at all 30 dental schools in Germany in February 2023. Statistical analysis was performed using Fisher's exact test (P<.05).
Results: The response rate amounted to 90.0% (27/30 dental schools). All respondent dental schools employed multiple-choice examinations for summative assessments. Examinations were delivered electronically by 70.4% (19/27) of the dental schools. Almost all dental schools used single-choice Type A items (88.9%) which accounted for the largest number of items in about half of the dental schools. Further item types (eg, conventional multiple-select items, Multiple-True-False, Pick-N) were only used by fewer dental schools (≤66.7%, up to 18 out of 27 dental schools). For the multiple-select item types, the applied scoring methods varied considerably (ie, awarding [intermediate] partial credit, requirements for partial credit). Dental schools with the possibility of electronic examinations used multiple-select items slightly more often (73.7%, 14/19 vs. 50.0%, 4/8). However, this difference was statistically not significant (P=.375). Dental schools used items either individually or as key feature problems consisting of a clinical case scenario followed by a number of items focusing on critical treatment steps (55.6%, 15/27). Not a single school employed alternative testing methods (eg, answer-until-correct). A formal item review process was established at about half of the dental schools (55.6%, 15/27).
Conclusions: Summative assessment methods among German dental schools vary widely. Especially, a large variability regarding the use and scoring of multiple-select multiple-choice items was found.