Studies in Educational Evaluation最新文献

With a sample of 2274 teachers from 181 lower-secondary schools in China, this study examined the principal instructional leadership effects on the rural-urban instructional quality gap, with a particular inquiry into the mechanism of teacher professional learning. The results show that there were significant gaps in three aspects of instructional quality (i.e., supportive climate, effective management, and cognitive activation) between rural and urban schools. Principal instructional leadership had significant total mediating effects on the rural-urban instructional disparities; however, its effects on the disparities between rural and urban teachers’ cognitive activation were partially suppressed. Particularly, the mechanism of teacher professional learning was identified through which principal instructional leadership explained the rural-urban instructional disparities. Research implications are also discussed.

This paper describes the development and initial validation of the Cognition and Emotion/Motivation Regulation (CEMOR) questionnaire, a task-specific questionnaire for upper elementary school students that measures self-regulated learning (SRL). Using a multistep procedure, 22 items were developed, divided over five theory-informed dimensions (Planning, Monitoring, Cognition Control, Emotion/Motivation Control, and Reflecting). The CEMOR was applied in a math context. Children from grades 3–6 (N = 547, 54.7 % females) completed the CEMOR. Confirmatory factor analyses indicated that the five proposed scales have adequate to good model fit, with factor loadings ranging from .54 to .83, and acceptable to good composite reliability (ρ range = .75–.85). To find further validity support, the SRL scales were correlated with students’ performance on a math task, experienced emotions, and level of motivation during the task. Most correlations were statistically significant and in the expected direction. Hence, the CEMOR questionnaire shows promise as a new SRL instrument for elementary education.

Learning losses during the COVID-19 pandemic have been well documented. In this paper, we attempt to learn more about the determinants of the magnitude of the adverse effects of the emergency shift to remote learning in Polish upper secondary schools. Because of concerns that differences in access to and quality of local internet would disadvantage lower socio-economic status children, the analysis focuses on local internet speed and the structure of remote classes. We adopt a value-added framework to account for differences among students, families, and prior school quality and find that local internet speed and the structure of remote classes are strongly related to the achievement of vocational school students. Those in communities with internet speed in the lowest decile fare particularly poorly in mathematics, highlighting the adverse effects of deficiencies in infrastructure and access to higher-quality internet services.

Numerous studies have highlighted the significant role of Student Evaluations of Teaching (SETs) as a key metric for assessing teaching quality in Higher Education (HE). Building upon these insights, our study introduces an innovative four-tiered model, derived from diverse research, to examine the reliability of SETs. This model addresses biases associated with SETs, delving into both statistical anomalies and cognitive biases, with particular emphasis on often-overlooked hidden context and timing factors. We reveal that these biases can distort SET scores, leading to potentially inaccurate representations of both individual and comparative academic performances. The implications of our research are significant for those influencing HE policy-making and performance evaluation. We echo previous calls for a more expansive approach to teaching effectiveness, essential for genuine insight into teaching quality. By adopting this perspective, HE can design better-informed strategies, ensuring policies and practices reflect the diverse nature of teaching excellence.

Student Evaluation of Teaching (SET) instruments can be administered online or with paper and pencil. These procedures involve different interactions between respondents and instruments which could cause errors when administrators and researchers want to compare or relate online results with paper-based results. This study aimed to analyze if the Romanian version of the Student Evaluations of Educational Quality instrument presents similar psychometric characteristics regardless if it is completed online or by paper and pencil. Data was collected from two groups of students (N = 809). One group completed the SEEQ on paper-pencil support (312 responders, 38.6%). The other group completed the scale for the same teachers and courses, but online and one year later. Psychometrical checks and measurement invariance analysis suggested that the Romanian version of SEEQ has passed the internal validity and reliability test and presented configural, metric, and scalar invariance across the results obtained from both administration procedures.

Using longitudinal data across Grades 3 to 5 on students and teachers from 123 elementary schools, we developed a multivariate multilevel model incorporating changes among both students and teachers to examine the relationship among schools between change in school racial-ethnic gap in academic achievement (English language arts and mathematics) and change in school renewal effort (by school principals). Across Grades 3 to 5, we found statistically significant improvement in school renewal effort, whereas the school racial-ethnic gap in academic achievement was statistically stable. In terms of their relationship, we found that, across Grades 3 to 5, improvement in school renewal effort were weakly correlated with enlargement in school racial-ethnic gap in academic achievement concerning English language arts (correlation = .10). Meanwhile, improvement in school renewal effort were marginally correlated with reduction in school racial-ethnic gap in academic achievement concerning mathematics (correlation = −.18).

Student self-reports of knowledge are widely used by researchers and educators, though their accuracy has been questioned due to potential biases. The overclaiming technique (OCT), based on the familiarity ratings of existing (reals) and non-existing (foils) items, has been used to identify accuracy and exaggeration in respondents’ self-reports. We developed an original English OCT measure specifically for students who learn English as a foreign language (EFL). Our sample consists of Czech lower secondary students (N = 1391). We show that students’ foil claiming relates to their gender, EFL exposure at school, at home, in their free time, and through peers, but not to their school type. School type, however, is a relevant factor in the claiming of reals. Warning about foils relates to lower foil claiming. Further research could examine students’ understanding of OCT items through cognitive interviews and extend the use of the OCT to other foreign languages.

This paper addresses the theoretical and practical question of how to specify criteria used in educational evaluations. People and groups involved in and affected by an educational initiative typically bring different values to bear on the question of what a quality or successful initiative means. This poses a challenge of balancing and prioritizing between differing values when specifying criteria. To address this challenge, we present a framework and process that involves systematic consideration of multiple sources of criteria (i.e., who and where) and domains (e.g., design, outcomes) followed by explicitly defining criteria within an evaluation. We illustrate our use of this framework in three evaluations: a high school mathematics teacher program, K-12 principal professional development initiatives, and a graduate-level online healthcare administration program. Together, the framework and illustrations provide guidance and highlight future directions for strengthening explicit criteria specification in educational evaluation.

While recent decades have witnessed the proliferation of studies on peer feedback in L2 writing, little is known about how L2 learners engage affectively, behaviorally, and cognitively with this practice. To narrow this gap, the present study employing a mixed-methods approach examined L2 learners’ engagement with peer feedback in the Chinese secondary school context. Data were collected from a variety of sources over 12 weeks, including questionnaires, semi-structured interviews, writing journals, and learners’ first and revised writing samples. The results showed that the teacher’s scaffolding with systematic instruction comprising pre-feedback sessions, multiple feedback practices, and post-feedback reinforcement helped the participants engage with peer feedback proactively in affect, behavior, and cognition. Furthermore, the participants’ perception of their improvement in the three dimensions was also evident over the semester. Overall, this study demonstrates the role of teachers in student engagement and advances our understanding of L2 learner engagement with peer feedback. Additionally, it offers important pedagogical implications for fostering and promoting L2 learners’ engagement.

The aim of this study was to examine how achievement varied within and between schools at different grade levels, and long-term trends in variation within and across multiple countries. We used science achievement data from five cycles of Trends in International Mathematics and Science Study (TIMSS) from 2003 to 2019 involving 10 countries from Asia, Europe, and the United States. Employing exploratory data mining methods of variance decomposition, correlation analysis, and Gaussian mixture modeling of data distributions, we found the following: First, between-school variances generally remained consistent across two decades, suggesting that inequality between schools has not increased over time. Second, between-school variances were relatively small for elementary grade level but increased at secondary grade level, though marginally even for countries with early tracking. Third, higher-achieving schools tended to have more equal student achievement levels than lower-achieving schools, lending within-country support for the “virtuous” efficiency-equality trade-off. We further found that reduced equality within lower-achieving schools was associated with bimodality in achievement distribution. Overall, there is little evidence of inequality across schools changing over time. However, there may be evidence of increased inequalities associated with student subpopulations, particularly within lower-achieving schools, with implications on classroom instruction and school cohesion.