The dimensionality is one of the most investigated concepts in the psychological assessment, and there are many ways to determine the dimensionality of a measured construct. The Automated Item Selection Procedure (AISP) and the DETECT are non-parametric methods aiming to determine the factorial structure of a data set. In the current study, dimensionality results provided by the two methods were compared based on the original factorial structure defined by the scale developers. For the comparison of the two methods, the data was obtained by implementing a scale measuring academic dishonesty levels of bachelor students. The scale was conducted on junior students studying at a public and a private university. The dataset was analyzed by using the AISP and DETECT analyses. The “mokken” and “sirt” packages on the R program were utilized for the AISP and DETECT analyses, respectively. The similarities and differences between the findings provided by the methods were analyzed depending on the original factor structure of the scale verified by the scale developers.
{"title":"The comparison of the dimensionality results provided by the automated item selection procedure and DETECT analysis","authors":"Ezgi Mor Dirlik, S. Kartal","doi":"10.21449/ijate.1059200","DOIUrl":"https://doi.org/10.21449/ijate.1059200","url":null,"abstract":"The dimensionality is one of the most investigated concepts in the psychological assessment, and there are many ways to determine the dimensionality of a measured construct. The Automated Item Selection Procedure (AISP) and the DETECT are non-parametric methods aiming to determine the factorial structure of a data set. In the current study, dimensionality results provided by the two methods were compared based on the original factorial structure defined by the scale developers. For the comparison of the two methods, the data was obtained by implementing a scale measuring academic dishonesty levels of bachelor students. The scale was conducted on junior students studying at a public and a private university. The dataset was analyzed by using the AISP and DETECT analyses. The “mokken” and “sirt” packages on the R program were utilized for the AISP and DETECT analyses, respectively. The similarities and differences between the findings provided by the methods were analyzed depending on the original factor structure of the scale verified by the scale developers.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42158192","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
There is an increasing understanding that assessment is an integral part of teaching and learning and that teachers are largely not adequately prepared for their assessment responsibilities. Consequently, there is a need for research on what teachers need to improve their assessment practices. To determine what Jamaican secondary school teachers need, this mixed methods study was conducted to describe the assessment tools and strategies used by secondary school teachers of various subjects and in different types of schools as the basis for future interventions. Data was collected from a survey of 1088 secondary school teachers of varying subjects and school types and further explored through interviews and observations of 32 teachers of English. Analysis of the data using descriptive statistics and ANOVA in the quantitative phase revealed that secondary school teachers primarily used traditional assessment tools and strategies, particularly tests, despite school type. Pattern coding and pattern matching in the qualitative phase confirmed these results. The findings also revealed statistically significant differences in the frequency of use of traditional and alternative assessment tools and strategies based on the subject the teachers taught. Qualitative explorations revealed that school policies that require a quota of grades and state or express positive attitudes towards tests influenced teachers despite school type to use traditional methods. The findings imply that school administrators need to implement supportive school-level policies and display positive attitudes toward alternative assessments to maximize the use of assessment to improve learning.
{"title":"Assessment tools and strategies used by Jamaican secondary school teachers","authors":"Clavia WİLLİAMS-MCBEAN","doi":"10.21449/ijate.980870","DOIUrl":"https://doi.org/10.21449/ijate.980870","url":null,"abstract":"There is an increasing understanding that assessment is an integral part of teaching and learning and that teachers are largely not adequately prepared for their assessment responsibilities. Consequently, there is a need for research on what teachers need to improve their assessment practices. To determine what Jamaican secondary school teachers need, this mixed methods study was conducted to describe the assessment tools and strategies used by secondary school teachers of various subjects and in different types of schools as the basis for future interventions. Data was collected from a survey of 1088 secondary school teachers of varying subjects and school types and further explored through interviews and observations of 32 teachers of English. Analysis of the data using descriptive statistics and ANOVA in the quantitative phase revealed that secondary school teachers primarily used traditional assessment tools and strategies, particularly tests, despite school type. Pattern coding and pattern matching in the qualitative phase confirmed these results. The findings also revealed statistically significant differences in the frequency of use of traditional and alternative assessment tools and strategies based on the subject the teachers taught. Qualitative explorations revealed that school policies that require a quota of grades and state or express positive attitudes towards tests influenced teachers despite school type to use traditional methods. The findings imply that school administrators need to implement supportive school-level policies and display positive attitudes toward alternative assessments to maximize the use of assessment to improve learning.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"47801864","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Grading is a complicated decision-making process that needs teachers to make value judgements on the learning, accomplishment, and development of their students. When assigning grades, teachers tend to examine a variety of criteria, including students' efforts, work habits, and accomplishments. Grades are a reflection of the value judgements made about students based on the evaluation of their academic performance. Consequently, describing how to reach a value judgment utilizing general measures will contribute to a better understanding of the difficulties encountered throughout the grading process. The purpose of this research is to adapt the Teacher Perceptions of Grading Practices Scale into Turkish and to examine the measurement invariance. This scale, which examines teachers' perceptions of grading methods, has six components: importance, usefulness, student effort, student ability, teacher's grading patterns, and perceived self-efficacy of the grading process. Before adapting the scale, permission was first acquired from the researcher who developed it. To ensure linguistic comparability, bilingual translators were recruited in the second phase. The semantic, experiential, conceptual, and idiomatic equivalence between the two variants of the scale were evaluated. The original and adapted scales were administered to a group of English teachers twice at a predetermined interval, and the consistency between the two applications was analyzed due to the fact that the language employed in the original test was a widely spoken group. Confirmatory Factor Analysis (CFA) was used to examine the factor structure of the original scale. Cronbach's alpha and Omega coefficients were calculated for the reliability of the data obtained from the scale. Finally, the measurement invariance of the scale according to gender was examined by using Multiple Group Confirmatory Factor Analysis (MGCFA), and it was determined that the measurement model fulfilled the criteria of complete gender-group invariance.
评分是一个复杂的决策过程，需要教师对学生的学习、成就和发展做出价值判断。在分配分数时，老师倾向于考察各种标准，包括学生的努力、工作习惯和成就。成绩反映了对学生的价值判断，这是基于对学生学业表现的评估。因此，描述如何达到一个价值判断利用一般措施将有助于更好地理解整个分级过程中遇到的困难。本研究的目的是将教师评分实践量表改编为土耳其语，并检查测量的不变性。该量表检查教师对评分方法的看法，有六个组成部分:重要性，有用性，学生努力，学生能力，教师的评分模式，以及对评分过程的感知自我效能。在调整该量表之前，首先要获得开发该量表的研究人员的许可。为了确保语言的可比性，在第二阶段征聘了双语翻译。评估了两种量表变体之间的语义、经验、概念和习惯对等性。在预定的时间间隔内，对一组英语教师进行了两次原始和调整后的量表，并分析了两个应用程序之间的一致性，因为原始测试中使用的语言是一个广泛使用的群体。采用验证性因子分析(CFA)对原量表的因子结构进行检验。通过计算Cronbach's alpha和Omega系数来确定量表数据的可靠性。最后，采用多组验证性因子分析(Multiple Group Confirmatory Factor Analysis, MGCFA)检验量表按性别的测量不变性，确定测量模型满足完全性别-群体不变性标准。
{"title":"Adaptation of Teachers' Perceptions of Grading Practices Scale to Turkish and Examination of Measurement Invariance","authors":"Yeşim ÖZER ÖZKAN, Meltem Acar Güvendir, Emre Guvendi̇r","doi":"10.21449/ijate.1115317","DOIUrl":"https://doi.org/10.21449/ijate.1115317","url":null,"abstract":"Grading is a complicated decision-making process that needs teachers to make value judgements on the learning, accomplishment, and development of their students. When assigning grades, teachers tend to examine a variety of criteria, including students' efforts, work habits, and accomplishments. Grades are a reflection of the value judgements made about students based on the evaluation of their academic performance. Consequently, describing how to reach a value judgment utilizing general measures will contribute to a better understanding of the difficulties encountered throughout the grading process. The purpose of this research is to adapt the Teacher Perceptions of Grading Practices Scale into Turkish and to examine the measurement invariance. This scale, which examines teachers' perceptions of grading methods, has six components: importance, usefulness, student effort, student ability, teacher's grading patterns, and perceived self-efficacy of the grading process. Before adapting the scale, permission was first acquired from the researcher who developed it. To ensure linguistic comparability, bilingual translators were recruited in the second phase. The semantic, experiential, conceptual, and idiomatic equivalence between the two variants of the scale were evaluated. The original and adapted scales were administered to a group of English teachers twice at a predetermined interval, and the consistency between the two applications was analyzed due to the fact that the language employed in the original test was a widely spoken group. Confirmatory Factor Analysis (CFA) was used to examine the factor structure of the original scale. Cronbach's alpha and Omega coefficients were calculated for the reliability of the data obtained from the scale. Finally, the measurement invariance of the scale according to gender was examined by using Multiple Group Confirmatory Factor Analysis (MGCFA), and it was determined that the measurement model fulfilled the criteria of complete gender-group invariance.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45292086","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Students may be at a disadvantage when learning if they cannot follow lessons face to face due to such reasons as epidemics, disasters, transportation, or family. The main purpose of this study is to perform alternative measurement and evaluation practices in hybrid learning environments in a way that will make students in online physics lessons active participants in the process. The research uses the developmental, emancipatory, and critical action research models within the scope of the qualitative research method. The research was carried out over three weeks under the guidance of the researcher with 32 10th-graders at the school where the researcher taught physics for 12 years. Semi-structured interview forms, rubric forms, and documents were used as data collection tools. The interviews and documents were evaluated using content analysis, while the rubrics were evaluated using descriptive analysis. The students’ active and decisive roles during the assessment and evaluation activities within the context-based learning activities regarding physics subjects as well as at the end of learning encouraged the students attending the lesson online and those attending in person to learn under the same conditions. In this context, activities in which students are a part of the learning and measurement-evaluation processes should be encouraged in online and hybrid-learning environments. Developing context-based activities with regard to experiments, analogy, and theoretical applications and developing qualified practices in which students will be active throughout the process under the guidance of action researchers will be beneficial for ensuring this.
{"title":"Measurement-evaluation applications of context-based activities in hybrid learning environments","authors":"A. Kumaş","doi":"10.21449/ijate.1111886","DOIUrl":"https://doi.org/10.21449/ijate.1111886","url":null,"abstract":"Students may be at a disadvantage when learning if they cannot follow lessons face to face due to such reasons as epidemics, disasters, transportation, or family. The main purpose of this study is to perform alternative measurement and evaluation practices in hybrid learning environments in a way that will make students in online physics lessons active participants in the process. The research uses the developmental, emancipatory, and critical action research models within the scope of the qualitative research method. The research was carried out over three weeks under the guidance of the researcher with 32 10th-graders at the school where the researcher taught physics for 12 years. Semi-structured interview forms, rubric forms, and documents were used as data collection tools. The interviews and documents were evaluated using content analysis, while the rubrics were evaluated using descriptive analysis. The students’ active and decisive roles during the assessment and evaluation activities within the context-based learning activities regarding physics subjects as well as at the end of learning encouraged the students attending the lesson online and those attending in person to learn under the same conditions. In this context, activities in which students are a part of the learning and measurement-evaluation processes should be encouraged in online and hybrid-learning environments. Developing context-based activities with regard to experiments, analogy, and theoretical applications and developing qualified practices in which students will be active throughout the process under the guidance of action researchers will be beneficial for ensuring this.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46149196","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In education, examining students' learning in detail, determining their strengths and weaknesses and giving effective feedback have gained importance over time. The aim of this study is to determine the distribution of students’ answers to the reading comprehension achievement test items which were written at different cognitive levels and to investigate the affective variables that are effective in classifying students based on their incorrect, blank, and unrelated answers identified via rubric. For this purpose, a reading comprehension achievement test, a student information form, the perceived academic self-efficacy scale and the learned helplessness tendency scale were used to collect data. The student information form included perseverance, achievement motivation, exposure to bullying and test anxiety subscales. A rubric was used to determine the students’ response categories. According to the findings of the study, the rate of blank and incorrect answers increases as the cognitive level of the items become more complex. While the most correct response rates are decreasing, partially-correct answers are increasing relatively. While students' learned helplessness tendencies were effective in classifying their blank and unrelated answers at the most basic reading comprehension level, as the cognitive process became more complex, the affective characteristics classifying the student responses increased in number. It was concluded that these variables are important in improving the students’ answers and in leading them to the partially correct and the most correct answer. It can be suggested to create trainings and classroom environments that will equip and improve students’ features about these variables.
{"title":"Are different responses related to the different affective features? CHAID analysis study","authors":"N. T. Özyeter","doi":"10.21449/ijate.1132687","DOIUrl":"https://doi.org/10.21449/ijate.1132687","url":null,"abstract":"In education, examining students' learning in detail, determining their strengths and weaknesses and giving effective feedback have gained importance over time. The aim of this study is to determine the distribution of students’ answers to the reading comprehension achievement test items which were written at different cognitive levels and to investigate the affective variables that are effective in classifying students based on their incorrect, blank, and unrelated answers identified via rubric. For this purpose, a reading comprehension achievement test, a student information form, the perceived academic self-efficacy scale and the learned helplessness tendency scale were used to collect data. The student information form included perseverance, achievement motivation, exposure to bullying and test anxiety subscales. A rubric was used to determine the students’ response categories. According to the findings of the study, the rate of blank and incorrect answers increases as the cognitive level of the items become more complex. While the most correct response rates are decreasing, partially-correct answers are increasing relatively. While students' learned helplessness tendencies were effective in classifying their blank and unrelated answers at the most basic reading comprehension level, as the cognitive process became more complex, the affective characteristics classifying the student responses increased in number. It was concluded that these variables are important in improving the students’ answers and in leading them to the partially correct and the most correct answer. It can be suggested to create trainings and classroom environments that will equip and improve students’ features about these variables.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46432913","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
The study deals with the variation of Turkish students' reading comprehension performance according to perceived teacher support and reading activities in the classroom. This study, which is grounded on the data drawn from the PISA 2018 database, investigates the relationship between certain variables. In the analyses performed on the PISA IDE server, the PISA 2018 reading literacy general averages of Türkiye were associated with the identified variables, and the differences in the averages were examined. As a result, perceived teacher support, teacher's adaptation of the course, and stimulation of reading engagement have a positive relationship with reading comprehension; however, it was found out that the frequency of receiving feedback had a negative relationship with reading performance. In addition, the general reading average of the students who reported that they had not performed activities such as summarizing, comparing the content of the text with their own experiences, comparing the text they have read with other texts written on similar topics, and writing about the text that has been read was much higher than those who reported that they had performed these activities. These results have strengthened the conclusion that teachers give feedback to poor readers more frequently. On the other hand, it is possible that good readers may find the learning activities in the course inadequate. In summary, reading comprehension performance is positively or negatively affected by teacher support, adaptive instruction, feedback, and engagement in reading activities in the classroom.
{"title":"The role of teacher support and in class teaching practices on reading performance: Evidence from PISA 2018 outcomes for Türkiye","authors":"B. Bozkurt","doi":"10.21449/ijate.1132980","DOIUrl":"https://doi.org/10.21449/ijate.1132980","url":null,"abstract":"The study deals with the variation of Turkish students' reading comprehension performance according to perceived teacher support and reading activities in the classroom. This study, which is grounded on the data drawn from the PISA 2018 database, investigates the relationship between certain variables. In the analyses performed on the PISA IDE server, the PISA 2018 reading literacy general averages of Türkiye were associated with the identified variables, and the differences in the averages were examined. As a result, perceived teacher support, teacher's adaptation of the course, and stimulation of reading engagement have a positive relationship with reading comprehension; however, it was found out that the frequency of receiving feedback had a negative relationship with reading performance. In addition, the general reading average of the students who reported that they had not performed activities such as summarizing, comparing the content of the text with their own experiences, comparing the text they have read with other texts written on similar topics, and writing about the text that has been read was much higher than those who reported that they had performed these activities. These results have strengthened the conclusion that teachers give feedback to poor readers more frequently. On the other hand, it is possible that good readers may find the learning activities in the course inadequate. In summary, reading comprehension performance is positively or negatively affected by teacher support, adaptive instruction, feedback, and engagement in reading activities in the classroom.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-11-28","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44161016","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study aims to develop a ‘Teacher Expectation Scale’ (TES) to accurately and reliably measure teachers' expectations from their students. The development process of TES has an exploratory mixed method research design. The maximum variety sampling method was used when collecting qualitative data for the study, and the simple random sampling method was used for quantitative data. In the study groups of the research, there are 27 teachers for semi-structured interviews, 423 teachers for Exploratory Factor Analysis (EFA) and 750 teachers for Confirmatory Factor Analysis (CFA). For the content and face validity of the scale, six experts' opinions were obtained. A structure consisting of 36 items and 2 factors was revealed, which explains 73.54% of the total variance as a result of EFA. It has been seen that the items contained in TES show high levels of affiliation to the relevant factors and that all items are discriminative. The explored structure with EFA was evaluated using CFA. The following results were obtained when examining the compliance indices of the obtained model: χ²/df=4.53<5; CFI=0.99; TLI=0.99; RMSEA=0.07; SRMR=0.05. From the calculated reliability coefficients, McDonald's Omega (0.98) and stratified alpha coefficient (0.96) for the scale overall and Cronbach alpha coefficient (.98) for the dimensions were obtained. Reliability and validity results, obtained from TES showed that it is a valid and reliable measurement tool with two factors and 36 items. The subject of teacher expectation can be examined in terms of many variables using TES developed in the current research.
{"title":"A study of reliability, validity and development of the teacher expectation scale","authors":"Hasan İğde, L. Yakar","doi":"10.21449/ijate.976585","DOIUrl":"https://doi.org/10.21449/ijate.976585","url":null,"abstract":"This study aims to develop a ‘Teacher Expectation Scale’ (TES) to accurately and reliably measure teachers' expectations from their students. The development process of TES has an exploratory mixed method research design. The maximum variety sampling method was used when collecting qualitative data for the study, and the simple random sampling method was used for quantitative data. In the study groups of the research, there are 27 teachers for semi-structured interviews, 423 teachers for Exploratory Factor Analysis (EFA) and 750 teachers for Confirmatory Factor Analysis (CFA). For the content and face validity of the scale, six experts' opinions were obtained. A structure consisting of 36 items and 2 factors was revealed, which explains 73.54% of the total variance as a result of EFA. It has been seen that the items contained in TES show high levels of affiliation to the relevant factors and that all items are discriminative. The explored structure with EFA was evaluated using CFA. The following results were obtained when examining the compliance indices of the obtained model: χ²/df=4.53<5; CFI=0.99; TLI=0.99; RMSEA=0.07; SRMR=0.05. From the calculated reliability coefficients, McDonald's Omega (0.98) and stratified alpha coefficient (0.96) for the scale overall and Cronbach alpha coefficient (.98) for the dimensions were obtained. Reliability and validity results, obtained from TES showed that it is a valid and reliable measurement tool with two factors and 36 items. The subject of teacher expectation can be examined in terms of many variables using TES developed in the current research.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-09-11","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49606069","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the .20, .30 and .35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the .20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the .35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.
{"title":"A novel approach for calculating the item discrimination for Likert type of scales","authors":"Ümit Çelen, E. C. Aybek","doi":"10.21449/ijate.1173356","DOIUrl":"https://doi.org/10.21449/ijate.1173356","url":null,"abstract":"Item analysis is performed by developers as an integral part of the scale development process. Thus, items are excluded from the scale depending on the item analysis prior to the factor analysis. Existing item discrimination indices are calculated based on correlation, yet items with different response patterns are likely to have a similar item discrimination index. This study proposed a new item discrimination index that can be used in Likert type of scales and examined its effect on factor analysis results. For this purpose, simulative datasets were generated, and items were excluded from the analysis according to the .20, .30 and .35 item discrimination index criteria, and exploratory factor analysis was performed for a single factor. Accordingly, it was found that more variance could be explained by a single factor with fewer items compared to other discrimination indices when the .20 criterion of the slope coefficient was used as suggested in this study. Similar findings were obtained using the .35 criterion with other discrimination indices. In this context, it is recommended to use the slope coefficient as an additional discrimination index calculation method in the scale development process.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-09-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"44878077","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Erdinç Duru, Ozen Yildirim, Asuman Duatepe Paksu, S. Duru
Bu çalışmanın amacı, üniversite öğrencilerinin eleştirel düşünme becerilerini ölçen kanıt temelli, geçerlik ve güvenilirliği yüksek bir ölçme aracı geliştirmektir. Pamukkale Eleştirel Düşünme Becerileri Ölçeği, çoktan seçmeli ve açık uçlu olmak üzere iki ayrı form olarak geliştirilmiştir. Çoktan seçmeli formla ilgili geçerlik güvenirlik çalışmaları klasik test kuramı ve madde-tepki kuramı olmak üzere iki farklı kuramsal çerçeve üzerine temellendirilmiştir. Klasik test kuramına göre ölçeğin çoktan seçmeli formunun yapı geçerliğini test etmek için açımlayıcı ve doğrulayıcı faktör analizi yapılmıştır. Analiz sonuçları ölçeğin tek boyutlu yapısını desteklemiştir. Ölçeğin güvenirlik çalışmaları kapsamında yapılan analizler, ölçeğin iç tutarlılık katsayısının .92 olduğunu, madde-toplam korelasyon değerlerinin .37 ile .84 arasında değiştiğini, eleştirel düşünme düzeyi düşük ve yüksek grupları anlamlı olarak ayırt edebildiğini göstermiştir. Test-tekrar test analizi sonuçları, ölçeğin ölçtüğü davranış alanına ilişkin zaman içinde kararlılık gösterdiğini destekler niteliktedir. Madde Tepki Kuramı temelli analiz sonuçları da ölçeğin, madde model uyumu varsayımlarını karşıladığını göstermiştir. Ölçeğin açık uçlu formunun değerlendirilmesinde ise araştırmacı/lar tarafından geliştirilen “Dereceli Puanlama Anahtarı” kullanılmıştır. Açık uçlu formun geçerlik ve güvenirliğiyle ilgili de bir dizi çalışma yapılmış, analiz sonuçları ölçeğin geçerliği ve güvenirliğine yönelik psikometrik destek sağlamıştır. Sonuç olarak, iki form halinde geliştirilen Pamukkale Eleştirel Düşünme Becerileri Ölçeği üniversite öğrencilerinin eleştirel düşünme becerilerini ölçmede kullanılabilecek geçerli ve güvenilir bir ölçme aracıdır. Bulgular alanyazın ışığında tartışılmış ve bazı öneriler verilmiştir.
{"title":"Pamukkale eleştirel düşünme becerisi ölçeği: geçerlik ve güvenirlik çalışması","authors":"Erdinç Duru, Ozen Yildirim, Asuman Duatepe Paksu, S. Duru","doi":"10.21449/ijate.1169977","DOIUrl":"https://doi.org/10.21449/ijate.1169977","url":null,"abstract":"Bu çalışmanın amacı, üniversite öğrencilerinin eleştirel düşünme becerilerini ölçen kanıt temelli, geçerlik ve güvenilirliği yüksek bir ölçme aracı geliştirmektir. Pamukkale Eleştirel Düşünme Becerileri Ölçeği, çoktan seçmeli ve açık uçlu olmak üzere iki ayrı form olarak geliştirilmiştir. Çoktan seçmeli formla ilgili geçerlik güvenirlik çalışmaları klasik test kuramı ve madde-tepki kuramı olmak üzere iki farklı kuramsal çerçeve üzerine temellendirilmiştir. Klasik test kuramına göre ölçeğin çoktan seçmeli formunun yapı geçerliğini test etmek için açımlayıcı ve doğrulayıcı faktör analizi yapılmıştır. Analiz sonuçları ölçeğin tek boyutlu yapısını desteklemiştir. Ölçeğin güvenirlik çalışmaları kapsamında yapılan analizler, ölçeğin iç tutarlılık katsayısının .92 olduğunu, madde-toplam korelasyon değerlerinin .37 ile .84 arasında değiştiğini, eleştirel düşünme düzeyi düşük ve yüksek grupları anlamlı olarak ayırt edebildiğini göstermiştir. Test-tekrar test analizi sonuçları, ölçeğin ölçtüğü davranış alanına ilişkin zaman içinde kararlılık gösterdiğini destekler niteliktedir. Madde Tepki Kuramı temelli analiz sonuçları da ölçeğin, madde model uyumu varsayımlarını karşıladığını göstermiştir. Ölçeğin açık uçlu formunun değerlendirilmesinde ise araştırmacı/lar tarafından geliştirilen “Dereceli Puanlama Anahtarı” kullanılmıştır. Açık uçlu formun geçerlik ve güvenirliğiyle ilgili de bir dizi çalışma yapılmış, analiz sonuçları ölçeğin geçerliği ve güvenirliğine yönelik psikometrik destek sağlamıştır. Sonuç olarak, iki form halinde geliştirilen Pamukkale Eleştirel Düşünme Becerileri Ölçeği üniversite öğrencilerinin eleştirel düşünme becerilerini ölçmede kullanılabilecek geçerli ve güvenilir bir ölçme aracıdır. Bulgular alanyazın ışığında tartışılmış ve bazı öneriler verilmiştir.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-09-06","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43808346","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This study aimed to adapt the COVID-19 Stress Scales (CSS) into Turkish and provide evidence for construct validity. For this purpose, firstly, Confirmatory factor analysis (CFA) was applied for the 5-factor model obtained during the development of CSS and the theoretically expected 6-factor model with total of 546 respondents. The findings revealed that the 6-factor model of CSS had a better fit in the Turkish sample. Factor loadings varied between .62 - .95 and correlations between subscales were between .44 - .76. Cronbach's Alpha and McDonald’s ω coefficients for each subscale indicated good-to-excellent internal consistency. To evaluate the criterion-related validity, the Turkish version of The Fear of COVID-19 Scale (FCV-19S) was administered to the participants and the correlation coefficients between this scale and the six subscale of CSS were calculated. We also conducted the Rasch analysis with related items to provide psychometric evidence for their unidimensional structure of each of the six subscales. Lastly, Differential item functioning (DIF) analysis was performed across subgroups by gender, having COVID-19, and being a student. Overall, the results of both CFA and Rasch analyses provided evidence to support the substantive aspect of validity and the appropriateness of the CSS as a measure of COVID-19 stress level in a Turkish sample.
{"title":"Adaptation and psychometric evaluation of the COVID-19 stress scales in Turkish sample","authors":"M. Şahin, S. Şen, Deniz Güler","doi":"10.21449/ijate.1067542","DOIUrl":"https://doi.org/10.21449/ijate.1067542","url":null,"abstract":"This study aimed to adapt the COVID-19 Stress Scales (CSS) into Turkish and provide evidence for construct validity. For this purpose, firstly, Confirmatory factor analysis (CFA) was applied for the 5-factor model obtained during the development of CSS and the theoretically expected 6-factor model with total of 546 respondents. The findings revealed that the 6-factor model of CSS had a better fit in the Turkish sample. Factor loadings varied between .62 - .95 and correlations between subscales were between .44 - .76. Cronbach's Alpha and McDonald’s ω coefficients for each subscale indicated good-to-excellent internal consistency. To evaluate the criterion-related validity, the Turkish version of The Fear of COVID-19 Scale (FCV-19S) was administered to the participants and the correlation coefficients between this scale and the six subscale of CSS were calculated. We also conducted the Rasch analysis with related items to provide psychometric evidence for their unidimensional structure of each of the six subscales. Lastly, Differential item functioning (DIF) analysis was performed across subgroups by gender, having COVID-19, and being a student. Overall, the results of both CFA and Rasch analyses provided evidence to support the substantive aspect of validity and the appropriateness of the CSS as a measure of COVID-19 stress level in a Turkish sample.","PeriodicalId":42417,"journal":{"name":"International Journal of Assessment Tools in Education","volume":" ","pages":""},"PeriodicalIF":0.9,"publicationDate":"2022-09-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48823657","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：