Studies in Educational Evaluation最新文献

Student Evaluation of Teaching (SET) instruments can be administered online or with paper and pencil. These procedures involve different interactions between respondents and instruments which could cause errors when administrators and researchers want to compare or relate online results with paper-based results. This study aimed to analyze if the Romanian version of the Student Evaluations of Educational Quality instrument presents similar psychometric characteristics regardless if it is completed online or by paper and pencil. Data was collected from two groups of students (N = 809). One group completed the SEEQ on paper-pencil support (312 responders, 38.6%). The other group completed the scale for the same teachers and courses, but online and one year later. Psychometrical checks and measurement invariance analysis suggested that the Romanian version of SEEQ has passed the internal validity and reliability test and presented configural, metric, and scalar invariance across the results obtained from both administration procedures.

Using longitudinal data across Grades 3 to 5 on students and teachers from 123 elementary schools, we developed a multivariate multilevel model incorporating changes among both students and teachers to examine the relationship among schools between change in school racial-ethnic gap in academic achievement (English language arts and mathematics) and change in school renewal effort (by school principals). Across Grades 3 to 5, we found statistically significant improvement in school renewal effort, whereas the school racial-ethnic gap in academic achievement was statistically stable. In terms of their relationship, we found that, across Grades 3 to 5, improvement in school renewal effort were weakly correlated with enlargement in school racial-ethnic gap in academic achievement concerning English language arts (correlation = .10). Meanwhile, improvement in school renewal effort were marginally correlated with reduction in school racial-ethnic gap in academic achievement concerning mathematics (correlation = −.18).

Student self-reports of knowledge are widely used by researchers and educators, though their accuracy has been questioned due to potential biases. The overclaiming technique (OCT), based on the familiarity ratings of existing (reals) and non-existing (foils) items, has been used to identify accuracy and exaggeration in respondents’ self-reports. We developed an original English OCT measure specifically for students who learn English as a foreign language (EFL). Our sample consists of Czech lower secondary students (N = 1391). We show that students’ foil claiming relates to their gender, EFL exposure at school, at home, in their free time, and through peers, but not to their school type. School type, however, is a relevant factor in the claiming of reals. Warning about foils relates to lower foil claiming. Further research could examine students’ understanding of OCT items through cognitive interviews and extend the use of the OCT to other foreign languages.

This paper addresses the theoretical and practical question of how to specify criteria used in educational evaluations. People and groups involved in and affected by an educational initiative typically bring different values to bear on the question of what a quality or successful initiative means. This poses a challenge of balancing and prioritizing between differing values when specifying criteria. To address this challenge, we present a framework and process that involves systematic consideration of multiple sources of criteria (i.e., who and where) and domains (e.g., design, outcomes) followed by explicitly defining criteria within an evaluation. We illustrate our use of this framework in three evaluations: a high school mathematics teacher program, K-12 principal professional development initiatives, and a graduate-level online healthcare administration program. Together, the framework and illustrations provide guidance and highlight future directions for strengthening explicit criteria specification in educational evaluation.

While recent decades have witnessed the proliferation of studies on peer feedback in L2 writing, little is known about how L2 learners engage affectively, behaviorally, and cognitively with this practice. To narrow this gap, the present study employing a mixed-methods approach examined L2 learners’ engagement with peer feedback in the Chinese secondary school context. Data were collected from a variety of sources over 12 weeks, including questionnaires, semi-structured interviews, writing journals, and learners’ first and revised writing samples. The results showed that the teacher’s scaffolding with systematic instruction comprising pre-feedback sessions, multiple feedback practices, and post-feedback reinforcement helped the participants engage with peer feedback proactively in affect, behavior, and cognition. Furthermore, the participants’ perception of their improvement in the three dimensions was also evident over the semester. Overall, this study demonstrates the role of teachers in student engagement and advances our understanding of L2 learner engagement with peer feedback. Additionally, it offers important pedagogical implications for fostering and promoting L2 learners’ engagement.

The aim of this study was to examine how achievement varied within and between schools at different grade levels, and long-term trends in variation within and across multiple countries. We used science achievement data from five cycles of Trends in International Mathematics and Science Study (TIMSS) from 2003 to 2019 involving 10 countries from Asia, Europe, and the United States. Employing exploratory data mining methods of variance decomposition, correlation analysis, and Gaussian mixture modeling of data distributions, we found the following: First, between-school variances generally remained consistent across two decades, suggesting that inequality between schools has not increased over time. Second, between-school variances were relatively small for elementary grade level but increased at secondary grade level, though marginally even for countries with early tracking. Third, higher-achieving schools tended to have more equal student achievement levels than lower-achieving schools, lending within-country support for the “virtuous” efficiency-equality trade-off. We further found that reduced equality within lower-achieving schools was associated with bimodality in achievement distribution. Overall, there is little evidence of inequality across schools changing over time. However, there may be evidence of increased inequalities associated with student subpopulations, particularly within lower-achieving schools, with implications on classroom instruction and school cohesion.

Background

Classic assessment methods present negative emotional alterations for students, such as stress, anxiety, fear and nervousness; these could be solved by applying the Escape Room, which is a promising tool where students experience positive emotions that are critical to learning.

Aim

To explore the experiences and perceptions of physiotherapy students regarding the use of an Escape Room game-based model for their assessment in contrast to conventional approaches.

Methods

A phenomenological-hermeneutical study was conducted. The assessment took place in two different modalities and on different days: 1) a traditional assessment method and 2) an assessment method incorporating a game-based model. All students participated in the two assessment processes. Fifty-six physiotherapy students took part in this study.

Results

The detailed analysis of the results allowed us to classify them into two main themes: 1) Strengths of the Escape Room as part of an assessment approach and, 2) Weaknesses of the Escape Room as part of an assessment approach. In turn, from these main themes emerged the sub-themes and their different units of meaning.

Conclusions

The results suggest that new teaching and assessment methodologies incorporating innovative models such as the Escape Room are effective in evaluating the skills and performance of physiotherapy students, which can be used to complement the traditional assessment methods.

Based on data from a large-scale assessment of school quality in China, this study examined the differences in the impact of learning and teaching strategies on the students’ academic achievements between elementary and middle schools. Two-level hierarchical linear models and coarsened exact matching revealed three major findings: (a) Cognitive, meta-cognitive, and inquiry-based learning strategies were significant predictors of students’ academic achievements, and the effect sizes of these strategies were larger in middle schools; (b) Individualized instruction, participatory instruction, and guided inquiry teaching strategies were significant predictors of students’ achievements, and the effect sizes of these strategies were larger in elementary schools; (c) Overall teaching strategies significantly improved learning strategies, and the effects were greater in elementary schools. These findings call for more research on learning/teaching strategies to improve students' academic performance.

Much attention has been paid to the relationship between parent-child relationships and adolescent academic performance, especially the mediating role of subjective educational expectations of parents and children. However, most of the research is a single study on parents' or children's educational expectations, and there is a lack of overall research on the independent components and combined effects of the subjective educational expectations system. Based on a large sample of 8867 middle school students, this study showed that parental expectation and self-expectation not only had independent indirect effects on parent-child relationship and academic performance, but also had a series of mediating effects together. The parental expectation was the strongest indirect influencing factor. These findings contribute to a comprehensive understanding of the deep mediating mechanism of educational expectations between parent-child relationship and academic performance in adolescents, and provide important new ideas for educational design practices in home-school co-education.

Differential item functioning (DIF) analysis is essential to ensuring the equity of measurement for different subgroups at the item level and is an integral part of validity. However, existing DIF research often overlooks within-group heterogeneity, commonly assuming that test takers from different subgroups comprise a homogeneous population. This study investigated DIF across gender, academic background, and their interaction in listening comprehension assessment using Rasch measurement. It found that ignoring within-group heterogeneity would lead to the under-detection of DIF, likely due to the cancellation of DIF at broader group levels. In addition, the study is the first to investigate DIF in a linked test, a scenario more prevalent in practical testing. The findings of the study highlight the importance of accounting for within-group heterogeneity in test fairness investigations in language assessment research and point to the potential effect of test linking and equating on DIF analysis and interpretation.