Hervaldo Sampaio Carvalho, Maria do Carmo Barros Melo, Fabiana Maria Kakehasi, Priscila Menezes Ferri Liu, Fernanda Gontijo Minafra Silveira Santos, Ricardo Luiz de Melo Martins, Marilene Vale de Castro Monteiro, Marcelo Esteves Chaves Campos, Roberto Zonato Esteves, Matheus Belloni Torsani, Iolanda de Fátima Lopes Calvo Tiberio, Akihito Inca Atahualpa Urdiales, Andrea Mora De Marco Novellino, Rosiane Viana Zuza Diniz, Helena Ferreira Moura, Maria Alice de Vilhena Toledo, Célia Regina Machado Saldanha, Andrea Pedrosa Ribeiro Alves Oliveira, Patricia Zen Tempski, Itamar de Souza Santos, Milton de Arruda Martins
{"title":"欧安组织和欧安组织在评估巴西医学课程准则所要求的能力方面的能力比较：一项多中心研究。","authors":"Hervaldo Sampaio Carvalho, Maria do Carmo Barros Melo, Fabiana Maria Kakehasi, Priscila Menezes Ferri Liu, Fernanda Gontijo Minafra Silveira Santos, Ricardo Luiz de Melo Martins, Marilene Vale de Castro Monteiro, Marcelo Esteves Chaves Campos, Roberto Zonato Esteves, Matheus Belloni Torsani, Iolanda de Fátima Lopes Calvo Tiberio, Akihito Inca Atahualpa Urdiales, Andrea Mora De Marco Novellino, Rosiane Viana Zuza Diniz, Helena Ferreira Moura, Maria Alice de Vilhena Toledo, Célia Regina Machado Saldanha, Andrea Pedrosa Ribeiro Alves Oliveira, Patricia Zen Tempski, Itamar de Souza Santos, Milton de Arruda Martins","doi":"10.1186/s12909-024-06631-4","DOIUrl":null,"url":null,"abstract":"<p><strong>Background: </strong>The COVID-19 pandemic has led to the popularization of information and communication technology in medical education. This study aimed to compare the objective structured clinical examination (OSCE) and the virtual objective structured clinical examination (vOSCE), based on expert opinion, as tools for assessing the competencies needed under medical curriculum guidelines in Brazil.</p><p><strong>Methods: </strong>In this multicenter study, the suitability levels of the OSCE and vOSCE for assessing the competencies needed under the Brazilian National Curriculum Guidelines (DCNs) were compared. The DCNs encompass five groups of domains, namely, health education; general; health management; public health needs; and individual health needs. Sixteen teaching experts served as judges. They responded to a questionnaire with the provided Likert scores indicating the ability of the OSCE and vOSCE to assess each type of competency. Virtual meetings were held via the Delphi method and a focus group-based approach to enable the experts to reach a consensus.</p><p><strong>Results: </strong>A total of 200 items included in the guidelines (100 for the OSCE and 100 for the vOSCE) were evaluated across a total of 3,200 responses. Analysis via the Likert scale revealed a high proportion of positive evaluations for the use of the vOSCE, with kappa values ranging from 0.4 to 0.72. The values of the absolute agreement (positive view) between examiners and equivalence with the kappa statistic for the adoption of the vOSCE ranged from 0.38 to 0.72. Via the use of virtual meetings (Delphi and focus groups), consensus was reached regarding the capacity of the OSCE and vOSCE to evaluate 97% and 92% of these items, respectively.</p><p><strong>Conclusion: </strong>The vOSCE can be employed to assess relevant competencies. However, it faces limitations regarding 8% (8/100) of the items, and some items (3%) cannot be assessed via either method. The difference in results between the two assessment methodologies (OSCE and vOSCE) is 5%. It is necessary to invest in the creation of instruments that can be used to apply the vOSCE and to conduct cost‒benefit analysis of its broader application in health education institutions. We conclude that the vOSCE is an effective tool for assessing most competencies of medical professionals and students required under Brazilian guidelines.</p>","PeriodicalId":51234,"journal":{"name":"BMC Medical Education","volume":"25 1","pages":"54"},"PeriodicalIF":2.7000,"publicationDate":"2025-01-13","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11727823/pdf/","citationCount":"0","resultStr":"{\"title\":\"Comparison of the adequacies of the OSCE and vOSCE to assess the competencies required under Brazilian medical curriculum guidelines: a multicenter study.\",\"authors\":\"Hervaldo Sampaio Carvalho, Maria do Carmo Barros Melo, Fabiana Maria Kakehasi, Priscila Menezes Ferri Liu, Fernanda Gontijo Minafra Silveira Santos, Ricardo Luiz de Melo Martins, Marilene Vale de Castro Monteiro, Marcelo Esteves Chaves Campos, Roberto Zonato Esteves, Matheus Belloni Torsani, Iolanda de Fátima Lopes Calvo Tiberio, Akihito Inca Atahualpa Urdiales, Andrea Mora De Marco Novellino, Rosiane Viana Zuza Diniz, Helena Ferreira Moura, Maria Alice de Vilhena Toledo, Célia Regina Machado Saldanha, Andrea Pedrosa Ribeiro Alves Oliveira, Patricia Zen Tempski, Itamar de Souza Santos, Milton de Arruda Martins\",\"doi\":\"10.1186/s12909-024-06631-4\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"<p><strong>Background: </strong>The COVID-19 pandemic has led to the popularization of information and communication technology in medical education. This study aimed to compare the objective structured clinical examination (OSCE) and the virtual objective structured clinical examination (vOSCE), based on expert opinion, as tools for assessing the competencies needed under medical curriculum guidelines in Brazil.</p><p><strong>Methods: </strong>In this multicenter study, the suitability levels of the OSCE and vOSCE for assessing the competencies needed under the Brazilian National Curriculum Guidelines (DCNs) were compared. The DCNs encompass five groups of domains, namely, health education; general; health management; public health needs; and individual health needs. Sixteen teaching experts served as judges. They responded to a questionnaire with the provided Likert scores indicating the ability of the OSCE and vOSCE to assess each type of competency. Virtual meetings were held via the Delphi method and a focus group-based approach to enable the experts to reach a consensus.</p><p><strong>Results: </strong>A total of 200 items included in the guidelines (100 for the OSCE and 100 for the vOSCE) were evaluated across a total of 3,200 responses. Analysis via the Likert scale revealed a high proportion of positive evaluations for the use of the vOSCE, with kappa values ranging from 0.4 to 0.72. The values of the absolute agreement (positive view) between examiners and equivalence with the kappa statistic for the adoption of the vOSCE ranged from 0.38 to 0.72. Via the use of virtual meetings (Delphi and focus groups), consensus was reached regarding the capacity of the OSCE and vOSCE to evaluate 97% and 92% of these items, respectively.</p><p><strong>Conclusion: </strong>The vOSCE can be employed to assess relevant competencies. However, it faces limitations regarding 8% (8/100) of the items, and some items (3%) cannot be assessed via either method. The difference in results between the two assessment methodologies (OSCE and vOSCE) is 5%. It is necessary to invest in the creation of instruments that can be used to apply the vOSCE and to conduct cost‒benefit analysis of its broader application in health education institutions. We conclude that the vOSCE is an effective tool for assessing most competencies of medical professionals and students required under Brazilian guidelines.</p>\",\"PeriodicalId\":51234,\"journal\":{\"name\":\"BMC Medical Education\",\"volume\":\"25 1\",\"pages\":\"54\"},\"PeriodicalIF\":2.7000,\"publicationDate\":\"2025-01-13\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11727823/pdf/\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"BMC Medical Education\",\"FirstCategoryId\":\"3\",\"ListUrlMain\":\"https://doi.org/10.1186/s12909-024-06631-4\",\"RegionNum\":2,\"RegionCategory\":\"医学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"EDUCATION & EDUCATIONAL RESEARCH\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"BMC Medical Education","FirstCategoryId":"3","ListUrlMain":"https://doi.org/10.1186/s12909-024-06631-4","RegionNum":2,"RegionCategory":"医学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"EDUCATION & EDUCATIONAL RESEARCH","Score":null,"Total":0}
Comparison of the adequacies of the OSCE and vOSCE to assess the competencies required under Brazilian medical curriculum guidelines: a multicenter study.
Background: The COVID-19 pandemic has led to the popularization of information and communication technology in medical education. This study aimed to compare the objective structured clinical examination (OSCE) and the virtual objective structured clinical examination (vOSCE), based on expert opinion, as tools for assessing the competencies needed under medical curriculum guidelines in Brazil.
Methods: In this multicenter study, the suitability levels of the OSCE and vOSCE for assessing the competencies needed under the Brazilian National Curriculum Guidelines (DCNs) were compared. The DCNs encompass five groups of domains, namely, health education; general; health management; public health needs; and individual health needs. Sixteen teaching experts served as judges. They responded to a questionnaire with the provided Likert scores indicating the ability of the OSCE and vOSCE to assess each type of competency. Virtual meetings were held via the Delphi method and a focus group-based approach to enable the experts to reach a consensus.
Results: A total of 200 items included in the guidelines (100 for the OSCE and 100 for the vOSCE) were evaluated across a total of 3,200 responses. Analysis via the Likert scale revealed a high proportion of positive evaluations for the use of the vOSCE, with kappa values ranging from 0.4 to 0.72. The values of the absolute agreement (positive view) between examiners and equivalence with the kappa statistic for the adoption of the vOSCE ranged from 0.38 to 0.72. Via the use of virtual meetings (Delphi and focus groups), consensus was reached regarding the capacity of the OSCE and vOSCE to evaluate 97% and 92% of these items, respectively.
Conclusion: The vOSCE can be employed to assess relevant competencies. However, it faces limitations regarding 8% (8/100) of the items, and some items (3%) cannot be assessed via either method. The difference in results between the two assessment methodologies (OSCE and vOSCE) is 5%. It is necessary to invest in the creation of instruments that can be used to apply the vOSCE and to conduct cost‒benefit analysis of its broader application in health education institutions. We conclude that the vOSCE is an effective tool for assessing most competencies of medical professionals and students required under Brazilian guidelines.
期刊介绍：
BMC Medical Education is an open access journal publishing original peer-reviewed research articles in relation to the training of healthcare professionals, including undergraduate, postgraduate, and continuing education. The journal has a special focus on curriculum development, evaluations of performance, assessment of training needs and evidence-based medicine.
京公网安备 11010802042870号
京ICP备2023020795号-1
分享
求助内容：
应助结果提醒方式：
扫码关注我们
