Despite the grand demand to receive diagnostic information about students’ difficulties in reading, there are very few tests specifically designed for diagnostic purposes. Therefore, many researches in cognitive diagnostic approach (CDA) use large-scale test results to provide fine and reliable diagnostic feedback on the strengths and weaknesses of students other than the total scores or percentiles ranks, which allow appropriate intervention. This study shows an example of the application of diagnostic modeling using data from 4,762 Canadian students who completed booklet 13 of the PIRLS test in 2011. The results highlight the potential for detailed diagnostic feedback of students’ strengths and weaknesses on the underlying skills identified in the test.
{"title":"Cognitive Diagnostic Analyses of the Progress in International Reading Literacy Study (PIRLS) 2011 Results","authors":"Dan Thanh Duong Thi, Nathalie Loye","doi":"10.7202/1084131ar","DOIUrl":"https://doi.org/10.7202/1084131ar","url":null,"abstract":"\u0000 Despite the grand demand to receive diagnostic information about students’ difficulties in reading, there are very few tests specifically designed for diagnostic purposes. Therefore, many researches in cognitive diagnostic approach (CDA) use large-scale test results to provide fine and reliable diagnostic feedback on the strengths and weaknesses of students other than the total scores or percentiles ranks, which allow appropriate intervention. This study shows an example of the application of diagnostic modeling using data from 4,762 Canadian students who completed booklet 13 of the PIRLS test in 2011. The results highlight the potential for detailed diagnostic feedback of students’ strengths and weaknesses on the underlying skills identified in the test.\u0000","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42271480","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
É. Frenette, S. Fontaine, Marie-Hélène Hébert, Mikhaël Éthier
This paper presents the seven-step approach to maximizing the evidence of validity that led to the development of the Questionnaire sur la tricherie aux examens à l’université (QTEU) [Questionnaire on Cheating in University Exams (QCUE)]. Composed of 28 items divided into 7 factors (propensity to cheat in exams, peer influence, cheating methods, institutional context, perception of control, performance goal, and commitment to one’s studies), the QCUE design was based on a comprehensive conceptual analysis of the scientific literature on cheating in exams, and on the work of Frenette, Hébert, Thibodeau, and Ndinga (2018) on how to develop a questionnaire maximizing the accumulation of validity evidence. With good psychometric properties, the QCUE meets a need for a French-language questionnaire on the propensity to cheat in exams and allows to measure the scope of cheating among university students.
{"title":"A Study on the Propensity to cheat in University Exams: Development and Validation Process of the Questionnaire on Cheating in University Exams (QCUE)","authors":"É. Frenette, S. Fontaine, Marie-Hélène Hébert, Mikhaël Éthier","doi":"10.7202/1084129ar","DOIUrl":"https://doi.org/10.7202/1084129ar","url":null,"abstract":"\u0000 This paper presents the seven-step approach to maximizing the evidence of validity that led to the development of the Questionnaire sur la tricherie aux examens à l’université (QTEU) [Questionnaire on Cheating in University Exams (QCUE)]. Composed of 28 items divided into 7 factors (propensity to cheat in exams, peer influence, cheating methods, institutional context, perception of control, performance goal, and commitment to one’s studies), the QCUE design was based on a comprehensive conceptual analysis of the scientific literature on cheating in exams, and on the work of Frenette, Hébert, Thibodeau, and Ndinga (2018) on how to develop a questionnaire maximizing the accumulation of validity evidence. With good psychometric properties, the QCUE meets a need for a French-language questionnaire on the propensity to cheat in exams and allows to measure the scope of cheating among university students.\u0000","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42997394","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This research focuses on evaluating the effectiveness of a training course in a hospital setting, using indicators of effect size and heterogeneity index. The evaluation focused on a training course in intermediate care for nurses. The course lasted 23 days and included clinical teaching at the patient’s bedside. The competencies were measured at the beginning and end of the training course, using an observation grid based on five domains: Clinical Expert, Communicator, Collaborator, Leader and Learner-Trainer. Cohen’s and Glass’s estimators demonstrated significant effects of training on the five domains while the heterogeneity index showed a reduction in behavioural disparities within the nursing group at the end of the training course. The discussion addresses issues relating to the boundaries used to interpret effect sizes.
{"title":"Assessing the Transfer of Nursing Competencies: Analysis of the effects of intermediate care training in a hospital setting","authors":"Yves Chochard, Jenny Gentizon, Serge Gallant","doi":"10.7202/1084130ar","DOIUrl":"https://doi.org/10.7202/1084130ar","url":null,"abstract":"\u0000 This research focuses on evaluating the effectiveness of a training course in a hospital setting, using indicators of effect size and heterogeneity index. The evaluation focused on a training course in intermediate care for nurses. The course lasted 23 days and included clinical teaching at the patient’s bedside. The competencies were measured at the beginning and end of the training course, using an observation grid based on five domains: Clinical Expert, Communicator, Collaborator, Leader and Learner-Trainer. Cohen’s and Glass’s estimators demonstrated significant effects of training on the five domains while the heterogeneity index showed a reduction in behavioural disparities within the nursing group at the end of the training course. The discussion addresses issues relating to the boundaries used to interpret effect sizes.\u0000","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42949751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Based on self-determination theory, this research presents the development of a scale to assess motivation for engineering studies in a Francophone context. Three phases of data collection were conducted (N = 462, 545 and 864) for a total of 1871 engineering students (59.2% female). Results from both exploratory and confirmatory factor analyses support a seven-factor structure for the scale: 1) intrinsic motivation, extrinsic motivations characterised by 2) identified regulation – altruism, 3) introjected regulation – ego, 4) introjected regulation – conscience, 5) external regulation – security, 6) external regulation – social prestige and, finally, 7) amotivation. The dimensions of altruism and security are specific to engineering studies. The scale meets generally accepted criteria for reliability and verifies different types of validity evidence.
{"title":"Developing a Motivation Scale for Engineering Studies in a Francophone Context","authors":"Marie Chédru","doi":"10.7202/1084127ar","DOIUrl":"https://doi.org/10.7202/1084127ar","url":null,"abstract":"\u0000 Based on self-determination theory, this research presents the development of a scale to assess motivation for engineering studies in a Francophone context. Three phases of data collection were conducted (N = 462, 545 and 864) for a total of 1871 engineering students (59.2% female). Results from both exploratory and confirmatory factor analyses support a seven-factor structure for the scale: 1) intrinsic motivation, extrinsic motivations characterised by 2) identified regulation – altruism, 3) introjected regulation – ego, 4) introjected regulation – conscience, 5) external regulation – security, 6) external regulation – social prestige and, finally, 7) amotivation. The dimensions of altruism and security are specific to engineering studies. The scale meets generally accepted criteria for reliability and verifies different types of validity evidence.\u0000","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42669158","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Curricular alignment refers to the links of coherence, found in any teaching-learning process, between curriculum objectives, learning tasks and assessment approaches. This model makes it possible to understand the coherence of any assessment approach. By mobilizing data from a collaborative study carried out on eight secondary school teachers of mathematics and French, we will show, however, that its meaning is rather limited when it comes to understanding coherence in graded summative assessment practices and that, consequently, the model needs to be expanded conceptually. To this end, we will draw on an example of a summative test modelled in this way. Our findings demonstrate the relevance of analyzing summative assessment practices with the help of the expanded model, while considering the role that context plays in certain of its aspects.
{"title":"Conceptually expanding the curricular alignment model to understand the coherence of the graded summative assessment practices of teachers: issues and perspectives","authors":"Raphaël Pasquini","doi":"10.7202/1084128ar","DOIUrl":"https://doi.org/10.7202/1084128ar","url":null,"abstract":"\u0000 Curricular alignment refers to the links of coherence, found in any teaching-learning process, between curriculum objectives, learning tasks and assessment approaches. This model makes it possible to understand the coherence of any assessment approach. By mobilizing data from a collaborative study carried out on eight secondary school teachers of mathematics and French, we will show, however, that its meaning is rather limited when it comes to understanding coherence in graded summative assessment practices and that, consequently, the model needs to be expanded conceptually. To this end, we will draw on an example of a summative test modelled in this way. Our findings demonstrate the relevance of analyzing summative assessment practices with the help of the expanded model, while considering the role that context plays in certain of its aspects.\u0000","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"41551423","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
{"title":"Translation Issue","authors":"","doi":"10.7202/1084126ar","DOIUrl":"https://doi.org/10.7202/1084126ar","url":null,"abstract":"","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":" ","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43700024","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Baptiste Motte, G. Aiguier, Pauline Reumaux, Gérard Forzy, Anthony Piermattéo, G. Ficheux, Dominique Vanpee, J. Cobbaut
Preparing students to deal with and tolerate uncertainty or ambiguity is a major issue in medical education. There are many English-language scales to assess tolerance of uncertainty and ambiguity but no French-language scale has ever demonstrated validity evidence for its scores. We selected the Tolerance of Ambiguity in Medical Students And Doctors ( TAMSAD ) scale. Through a structured process, the original questionnaire was translated, culturally adapted, and assessed after being administered to a sampling of medical students. Test-retest reliability was verified by presenting the questionnaire to the students again after two months. The assessment of internal consistency revealed satisfactory value. Test-retest reliability is assessed by intraclass correlation that presents good reproducibility of scores obtained by students in first completion and second completion. These results indicate that the French version of the TAMSAD scale can be used to assess French medical students’ tolerance to ambiguity.
{"title":"Cultural adaptation and assessment of validity evidence for scores obtained using a French version of the Tolerance of Ambiguity in Medical Students and Doctors scale","authors":"Baptiste Motte, G. Aiguier, Pauline Reumaux, Gérard Forzy, Anthony Piermattéo, G. Ficheux, Dominique Vanpee, J. Cobbaut","doi":"10.7202/1095681ar","DOIUrl":"https://doi.org/10.7202/1095681ar","url":null,"abstract":"Preparing students to deal with and tolerate uncertainty or ambiguity is a major issue in medical education. There are many English-language scales to assess tolerance of uncertainty and ambiguity but no French-language scale has ever demonstrated validity evidence for its scores. We selected the Tolerance of Ambiguity in Medical Students And Doctors ( TAMSAD ) scale. Through a structured process, the original questionnaire was translated, culturally adapted, and assessed after being administered to a sampling of medical students. Test-retest reliability was verified by presenting the questionnaire to the students again after two months. The assessment of internal consistency revealed satisfactory value. Test-retest reliability is assessed by intraclass correlation that presents good reproducibility of scores obtained by students in first completion and second completion. These results indicate that the French version of the TAMSAD scale can be used to assess French medical students’ tolerance to ambiguity.","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71256422","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Nathan Béchard, Simon Langlois, G. Poliquin, Stéphane Cyr
The integration of academic disciplines such as science and mathematics (M&S) can help foster students’ interest and sense of self-efficacy (S-E). However, few questionnaires appear to have been developed to measure these perceptual variables in this type of context. Nor do any appear to exist that are suitable for elementary students from low-income and multicultural schools. This article reports the results of the validation process of such a questionnaire comprising six scales conducted among 1,553 Grade 5 or 6 students from schools in Canada’s Montréal area. In its final version, the Science and Mathematics Integration Questionnaire (SMIQ) measures six perceptual variables: S-E in science, S-E in mathematics, interest in science, interest in mathematics, interest in integrated M&S activities and perception of the links between science and mathematics. To our knowledge, the last two variables have never been measured and the proposed scales to measure them are new.
{"title":"Development of the SMIQ questionnaire measuring interest, sense of self-efficacy and perception of the links existing between mathematics and science in an integrated context","authors":"Nathan Béchard, Simon Langlois, G. Poliquin, Stéphane Cyr","doi":"10.7202/1100057ar","DOIUrl":"https://doi.org/10.7202/1100057ar","url":null,"abstract":"The integration of academic disciplines such as science and mathematics (M&S) can help foster students’ interest and sense of self-efficacy (S-E). However, few questionnaires appear to have been developed to measure these perceptual variables in this type of context. Nor do any appear to exist that are suitable for elementary students from low-income and multicultural schools. This article reports the results of the validation process of such a questionnaire comprising six scales conducted among 1,553 Grade 5 or 6 students from schools in Canada’s Montréal area. In its final version, the Science and Mathematics Integration Questionnaire (SMIQ) measures six perceptual variables: S-E in science, S-E in mathematics, interest in science, interest in mathematics, interest in integrated M&S activities and perception of the links between science and mathematics. To our knowledge, the last two variables have never been measured and the proposed scales to measure them are new.","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71269983","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Many graduate students encounter challenges in their academic formation for which they consult university support services. The first aim of this study is to conduct a literature review of the main difficulties encountered. A classification of the latter, by content analysis, reveals nine categories. From this preliminary classification, a scale screening these difficulties, the EDECS, was constructed for graduate students support services. The exploratory factor analysis of the EDECS with 500 students grouped the nine initial categories of difficulties into four subscales, thus modifying the preliminary classification. The use of the test-retest method among 188 participants revealed good temporal stability. Various proofs of validity (content, internal structure and correlation between variables) were obtained. The conclusion offers suggestions for using the EDECS by student support services and discusses clinical considerations. Many graduate students encounter challenges in their academic formation for which they consult university support services. The first aim of this study is to conduct a literature review of the main difficulties encountered. A classification of the latter, by content analysis, reveals nine categories. From this preliminary classification, a scale screening these difficulties, the EDECS, was constructed for graduate students support services. The exploratory factor analysis of the EDECS with 500 students grouped the nine initial categories of difficulties into four subscales, thus modifying the preliminary classification. The use of the test-retest method among 188 participants revealed good temporal stability. Various proofs of validity (content, internal structure and correlation between variables) were obtained. The conclusion offers suggestions for using the EDECS by student support services and discusses clinical considerations.
{"title":"Élaboration et processus de validation de l’Échelle d’évaluation des difficultés aux études de cycles supérieurs (EDECS) : étude préliminaire","authors":"Lian Boulet, France Landry, G. Goupil","doi":"10.7202/1090464ar","DOIUrl":"https://doi.org/10.7202/1090464ar","url":null,"abstract":"Many graduate students encounter challenges in their academic formation for which they consult university support services. The first aim of this study is to conduct a literature review of the main difficulties encountered. A classification of the latter, by content analysis, reveals nine categories. From this preliminary classification, a scale screening these difficulties, the EDECS, was constructed for graduate students support services. The exploratory factor analysis of the EDECS with 500 students grouped the nine initial categories of difficulties into four subscales, thus modifying the preliminary classification. The use of the test-retest method among 188 participants revealed good temporal stability. Various proofs of validity (content, internal structure and correlation between variables) were obtained. The conclusion offers suggestions for using the EDECS by student support services and discusses clinical considerations. Many graduate students encounter challenges in their academic formation for which they consult university support services. The first aim of this study is to conduct a literature review of the main difficulties encountered. A classification of the latter, by content analysis, reveals nine categories. From this preliminary classification, a scale screening these difficulties, the EDECS, was constructed for graduate students support services. The exploratory factor analysis of the EDECS with 500 students grouped the nine initial categories of difficulties into four subscales, thus modifying the preliminary classification. The use of the test-retest method among 188 participants revealed good temporal stability. Various proofs of validity (content, internal structure and correlation between variables) were obtained. The conclusion offers suggestions for using the EDECS by student support services and discusses clinical considerations.","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71250751","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Estimating language complexity is an important aspect of educational measurement and assessment that can be used, for instance, to control for unwanted variance due to language
估计语言复杂性是教育测量和评估的一个重要方面，例如，可以用来控制由于语言而产生的不必要的差异
{"title":"ILSA: an automated language complexity analysis tool for French","authors":"Guillaume Loignon","doi":"10.7202/1095682ar","DOIUrl":"https://doi.org/10.7202/1095682ar","url":null,"abstract":"Estimating language complexity is an important aspect of educational measurement and assessment that can be used, for instance, to control for unwanted variance due to language","PeriodicalId":90014,"journal":{"name":"Mesure et evaluation en education","volume":"1 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2021-01-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"71256437","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：