Studies in Educational Evaluation最新文献

The aim of this study was to examine how achievement varied within and between schools at different grade levels, and long-term trends in variation within and across multiple countries. We used science achievement data from five cycles of Trends in International Mathematics and Science Study (TIMSS) from 2003 to 2019 involving 10 countries from Asia, Europe, and the United States. Employing exploratory data mining methods of variance decomposition, correlation analysis, and Gaussian mixture modeling of data distributions, we found the following: First, between-school variances generally remained consistent across two decades, suggesting that inequality between schools has not increased over time. Second, between-school variances were relatively small for elementary grade level but increased at secondary grade level, though marginally even for countries with early tracking. Third, higher-achieving schools tended to have more equal student achievement levels than lower-achieving schools, lending within-country support for the “virtuous” efficiency-equality trade-off. We further found that reduced equality within lower-achieving schools was associated with bimodality in achievement distribution. Overall, there is little evidence of inequality across schools changing over time. However, there may be evidence of increased inequalities associated with student subpopulations, particularly within lower-achieving schools, with implications on classroom instruction and school cohesion.

Background

Classic assessment methods present negative emotional alterations for students, such as stress, anxiety, fear and nervousness; these could be solved by applying the Escape Room, which is a promising tool where students experience positive emotions that are critical to learning.

Aim

To explore the experiences and perceptions of physiotherapy students regarding the use of an Escape Room game-based model for their assessment in contrast to conventional approaches.

Methods

A phenomenological-hermeneutical study was conducted. The assessment took place in two different modalities and on different days: 1) a traditional assessment method and 2) an assessment method incorporating a game-based model. All students participated in the two assessment processes. Fifty-six physiotherapy students took part in this study.

Results

The detailed analysis of the results allowed us to classify them into two main themes: 1) Strengths of the Escape Room as part of an assessment approach and, 2) Weaknesses of the Escape Room as part of an assessment approach. In turn, from these main themes emerged the sub-themes and their different units of meaning.

Conclusions

The results suggest that new teaching and assessment methodologies incorporating innovative models such as the Escape Room are effective in evaluating the skills and performance of physiotherapy students, which can be used to complement the traditional assessment methods.

Based on data from a large-scale assessment of school quality in China, this study examined the differences in the impact of learning and teaching strategies on the students’ academic achievements between elementary and middle schools. Two-level hierarchical linear models and coarsened exact matching revealed three major findings: (a) Cognitive, meta-cognitive, and inquiry-based learning strategies were significant predictors of students’ academic achievements, and the effect sizes of these strategies were larger in middle schools; (b) Individualized instruction, participatory instruction, and guided inquiry teaching strategies were significant predictors of students’ achievements, and the effect sizes of these strategies were larger in elementary schools; (c) Overall teaching strategies significantly improved learning strategies, and the effects were greater in elementary schools. These findings call for more research on learning/teaching strategies to improve students' academic performance.

Much attention has been paid to the relationship between parent-child relationships and adolescent academic performance, especially the mediating role of subjective educational expectations of parents and children. However, most of the research is a single study on parents' or children's educational expectations, and there is a lack of overall research on the independent components and combined effects of the subjective educational expectations system. Based on a large sample of 8867 middle school students, this study showed that parental expectation and self-expectation not only had independent indirect effects on parent-child relationship and academic performance, but also had a series of mediating effects together. The parental expectation was the strongest indirect influencing factor. These findings contribute to a comprehensive understanding of the deep mediating mechanism of educational expectations between parent-child relationship and academic performance in adolescents, and provide important new ideas for educational design practices in home-school co-education.

Differential item functioning (DIF) analysis is essential to ensuring the equity of measurement for different subgroups at the item level and is an integral part of validity. However, existing DIF research often overlooks within-group heterogeneity, commonly assuming that test takers from different subgroups comprise a homogeneous population. This study investigated DIF across gender, academic background, and their interaction in listening comprehension assessment using Rasch measurement. It found that ignoring within-group heterogeneity would lead to the under-detection of DIF, likely due to the cancellation of DIF at broader group levels. In addition, the study is the first to investigate DIF in a linked test, a scenario more prevalent in practical testing. The findings of the study highlight the importance of accounting for within-group heterogeneity in test fairness investigations in language assessment research and point to the potential effect of test linking and equating on DIF analysis and interpretation.

While the body of research on student feedback literacy in the field of education is growing, the association of such literacy with feedback engagement in L1 writing education remains relatively unexplored. Moreover, the potential relationship of a growth mindset, which serves as a motivational factor, with student feedback literacy has yet to be thoroughly investigated. Considering these gaps, the present study aimed to explore the relationships between a growth mindset, student writing feedback literacy, and feedback engagement. A sample of 236 sophomore English major students from a Chinese university was recruited. Employing path analysis, the study revealed that having a growth mindset was positively associated with all five features of student writing feedback literacy: appreciating feedback, acknowledging different feedback sources, making judgments, managing affect, and taking action. Moreover, the study revealed a nuanced relationship between students’ writing feedback literacy and their feedback engagement, with different features of writing feedback literacy having differential associations with distinct dimensions of feedback engagement. The study contributes to the literature in three ways. First, it expands our knowledge of the relationship between student feedback literacy and feedback engagement by delving deeper into the one-to-one association between different features of writing feedback literacy and various dimensions of feedback engagement. Second, it advances our understanding of how a growth mindset is related to shaping students’ writing feedback literacy. Finally, it provides evidence supporting the construct validity of the student writing feedback literacy scale in L1 writing learning.

Second language (L2) writing teachers’ personal beliefs can shape their feedback-giving practices. As a burgeoning area in L2 education, the mindsets framework offers us a novel theoretical perspective to analyze the impact of personal belief on feedback-giving practices. With 288 university English writing teachers from 28 provinces in China, this study examined their growth and fixed mindsets, as well as the five types of feedback-giving practices: scoring feedback, written corrective feedback, process-oriented feedback, expressive feedback, and peer and self-feedback. Paired sample t-test reported that teachers held significantly higher levels of growth than fixed mindsets. Repeated measures analysis of variance results indicated that expressive feedback was the most frequently used feedback approach, whereas written corrective feedback the least used approach. Structural equation modeling results found that growth mindsets were positively associated with all five types of feedback approaches, while fixed mindsets were only positively related to written corrective and process-oriented feedback. We interpreted the underlying mechanisms behind the varied associations between growth versus fixed mindsets with the five types of feedback-giving practices, and proposed suggestions for L2 writing teacher education.

Educational research has focused increasingly on comparing student civic outcomes across various educational systems. However, the study of non-cognitive, attitudinal measures has assumed a subsidiary role to the study of cognitive measures, despite its potential to shape policy and practice. This article investigates the validity and cross-country measurement equivalence or invariance (ME/I) of political trust measures among adolescents. Drawing on student data from ICCS 2016 covering 15 European educational systems (N = 52,788), the standard test for exact ME/I (i.e., Multi-Group Confirmatory Factor Analysis) is juxtaposed with a novel method for approximate ME/I (i.e., Alignment Optimization). The results underscore a two-dimensional structure of political trust, separating trust in order institutions from trust in representative institutions. While MGCFA results suggest configural invariance, rendering correlational analyses and country mean comparisons meaningless, Alignment Optimization yields valid latent means with an acceptable degree of non-invariance (i.e., 16%). Implications for comparative research on political trust among adolescents are discussed.

This study investigated pre-service primary teachers' reading strategies using the Metacognitive Awareness of Reading Strategy Inventory (MARSI) in the Indonesian version. The participants were 4377 pre-service primary teachers in Indonesia. The data were collected via an online form, and the invitations were sent out randomly. The participants were divided into two studies: 2912 participants were in study 1, which used a 5-point Likert scale, and 1465 participants were in study 2, using a 4-point Likert scale. The Rasch measurement approach was used in investigating the validity and reliability, rating improvement analysis, checking item difficulty functioning based on gender, and classifying pre-service primary teacher reading strategies. The adapted questionnaire satisfied all psychometric requirements based on Rasch modeling. The questionnaire with a 4-point scale showed better function than the 5-point one based on the observed measure average and curve probability category. The teachers' abilities ranged from − 6 logit to 6.45 logit.

In many countries, educational policy increasingly expects teachers to use results from nationwide achievement tests for data-based decision-making. Theoretical models suggest that teachers’ data use is not only related to their attitudes and the organizational culture of data use in their schools but also to the use of data by their principals. However, the relationship between teachers’ and principals’ data use has rarely been tested, and existing studies only provide limited evidence. This study examined these relationships using data from two large-scale samples (N_{mathematics teachers} =796, N_{German teachers} = 693). Results of structural equation models show that data use by teachers is significantly associated with data use by principals. Moreover, data use culture fully mediates this relationship, but teacher attitudes do not. We replicate our results for two samples, providing evidence for the generalizability. Thus, supporting principals to develop a positive data use culture seems promising for future professional development.