Educational Assessment Evaluation and Accountability最新文献

The current study presents a systematic review of teachers’ data literacy, arising from a synthesis of 83 empirical studies published between 1990 to 2021. Our review identified 95 distinct indicators across five dimensions: (a) knowledge about data, (b) skills in using data, (c) dispositions towards data use, (d) data application for various purposes, and (e) data-related behaviors. Our findings indicate that teachers' data literacy goes beyond addressing the needs of supporting student learning and includes elements such as teacher reflection, collaboration, communication, and participation in professional development. Considering these findings, future policies should acknowledge the significance of teacher dispositions and behaviors in relation to data, recognizing that they are as important as knowledge and skills acquisition. Additionally, prioritizing the provision of system-level support to foster teacher collaboration within in-school professional development programs may prove useful in enhancing teachers’ data literacy.

Assessing student writing assignments and providing effective feedback are a complex pedagogical skill teacher candidates need to master. Scarce literature has closely examined the type of feedback that pre-service high-school teachers spontaneously deliver when assessing student writings, which is the main goal of our study. In a sample of 255 high school teacher candidates, we examined the type of feedback that they provided when assessing two writing assignments that differed in quality. One thousand eight hundred thirty-five comments were analyzed and coded into 11 sub-categories. Results showed that candidates’ feedback not only focused on task performance but also on the writing process. Although candidates provided critical and past-oriented evaluations frequently, they also crafted feedback in a neutral tone and included future-oriented suggestions. Further, feedback varied as a function of candidates’ gender, academic discipline, and students’ quality of writing. Teacher training programs may use this information to design resources to address nuances of feedback provision.

This paper presents findings from a year-long network ethnography into the strategies, networks, and outcomes of More Than a Score (MTAS)—a campaign against standardised testing in UK primary schools. Focusing specifically on the parent-based groups of the organisation, we use theorisations of symbolic capital to challenge traditional understandings of how capital can be leveraged for group advancement. We argue that MTAS frames itself as a grassroots organisation, using this image to promote its agenda amongst possible allies. Parent groups serve a critical role in accentuating the ‘grassroots’ image, as they bring a level of credibility to this claim. At the same time, the individuals who run these groups also bring technical skills, professional experiences, and connections that provide logistical and expert capital to the range of MTAS’s strategies and agendas. In doing so, their political and social capital as ‘parents’ provide a sort of legitimising capital to MTAS.

Scores from observational measures of teaching have recently been put to many uses within school systems, including communicating a standard of practice and providing teacher feedback, identifying teachers for professional development, monitoring system equity, and making employment decisions. In each of these uses, observation scores are interpreted as representing some aspect of the enacted instruction or teachers’ capacity to enact instruction, as seen through the observation systems lens for understanding teaching quality. The quality of these interpretations, or the extent to which observation scores are composed of a signal that accurately reflects the interpretation, has important implications for the overall validity of uses of observation systems. Starting from an explicit conceptualization of instruction, this paper combines generalizability theory and hierarchical linear modelling approaches to decompose observation scores to explore the extent to which scores from observation systems are composed of signal, error, and bias across four different uses (i.e., teacher feedback, professional development, monitoring system equity, and employment decisions) of scores. We show that the quality of observation scores may depend more on what scores are interpreted as representing (i.e., the proposed use) than on the specific observation rubric being used. Further, we show that rater errors and biases are a major threat to any attempt to interpret observation scores as capturing the observation system’s understanding of teaching quality. We discuss implications for using scores from observation systems.

Many schools aim to implement Assessment for Learning (AfL) to stimulate students to take more ownership of their learning and develop self-regulatory skills. This survey-based study is among the few in the field showing how students experience the extent of implementation of AfL, here in English language and mathematics classes in 12 Dutch secondary schools. Analysis showed no differences between the subjects. Students (N = 685) experienced activities to clarify the learning intentions and success criteria and eliciting evidence on their learning progress regularly (between 50–74% of the lessons). Students hardly ever experienced activities aimed at peer- and self-assessment (less than 25% of the lessons). However, cluster analysis revealed three distinct clusters related to the extent of AfL strategy use students experienced. Overall, we can conclude that AfL is not yet fully integrated into teaching practices. Teachers’ skills, knowledge and attitudes required to increase student engagement in AfL practices and strengthen students’ self-regulated learning need more attention in future teacher professional development trajectories.

The present study investigates whether alternatively and traditionally certified teachers differ in their teaching quality. We conducted doubly latent multilevel analysis using a large-scale dataset with student ratings on the quality of instruction provided by 1550 traditionally and 135 alternatively certified secondary school mathematics teachers in Germany. Findings show no evidence of differences in teaching quality between alternatively and traditionally certified teachers. Independent of the type of certification, novice teachers scored lower on classroom management than experienced teachers. Findings suggest that beginning teachers—whether alternatively or traditionally certified—need to be better prepared for classroom management before starting to teach.

Abstract

Accountability policies through standardized testing are widespread in diverse educational systems. Based on an ethnographic research study, we sought to understand how a learning assessment policy, used for over three decades in primary and lower secondary education, is lived and interpreted in daily school life in Chile. This article analyzes how learning assessment policies are re-contextualized (translated and interpreted) in three elementary and high schools located in poor neighborhoods with different performance categorization according to students’ test results. We conducted interviews and observations with teachers and school leadership team members. Results show that school actors experience the effects of testing both as a menace and as a pressure despite their different school categorization performance. They consider the tests a decontextualized and unjust measurement and therefore respond with similar strategies that seek to avoid these negative consequences. This study contributes to understanding the effects of tight accountability systems that attempt to place different levels of pressure through categorization.

Based on the theoretical approaches of social capital and institutional trust, this paper seeks to identify contextual factors and conditions behind teacher behaviours which aim to alter the results of standardised tests in the Italian low-stakes accountability system. Numerous studies report significant factors associated with student cheating, but research into the factors of teacher-led opportunistic actions is scarce. Logistic regression models with fixed-effects at classroom level, with interaction terms, were carried out to identify factors increasing the likelihood of teacher misbehaviour. Models included approximately 79,100 primary, lower and upper secondary classrooms. Indicators of teacher cheating were estimated through algorithms based on suspicious answer strings from standardised tests. The results suggest that teacher cheating may be understood as a form of support for the most vulnerable students, since it is, to a greater extent, found helping low-income students, grade-retained students, as well as students in socially homogenous school settings. The findings also reveal that teacher cheating is consistently related to collectively share non-civic-minded behaviours and practices undertaken by teachers, which do not match legal requirements, such as within-school social segregation and exclusion of students from tests. Heterogeneous effects show that, even in classrooms with external controllers, the lower the civic capital in a school, the more misbehaviour are found. Relevant implications for research, social theory and policy are discussed.

This study uses evidence from a longitudinal survey (PISA Plus, Germany) to examine the potential of bias in international large-scale assessments (ILSAs). In PISA Plus, participation was mandatory at the first measurement point, but voluntary at the second measurement point. The study provides evidence for relevant selection bias regarding student competencies and background variables when participation is voluntary. Sample dropout at the second measurement point was related to characteristics such as family background, achievement in mathematics, reading and science, and other student and school demographic variables at both the student and school levels, with lower performing students and those with less favorable background characteristics having higher dropout frequencies, from which higher dropout probabilities of such students can be inferred. We further contrast the possibilities for addressing non-response through weight adjustments in longitudinal surveys with those in cross-sectional surveys. Considering our results, we evaluate and confirm the validity and appropriateness of strict participation rate requirements in ILSAs. Likely magnitudes of bias in cross-sectional studies in varying scenarios are illustrated. Accordingly, if combined participation rates drop below 70%, a difference of at least one-fifth of a standard deviation in an achievement score between non-respondents and participants leads to relevant bias. When participation drops below 50%, even a very small difference (one-tenth of a standard deviation) will cause non-negligible bias. Finally, we conclude that the stringent participation rate requirements established in most ILSAs are fully valid, reasonable, and important since they ensure a relatively low risk of biased results.

Test-based accountability offers schools limited guidance on informing sustained reforms, whereas on-site school evaluations provide broader and more nuanced evidence. In order to transform inspection feedback into long-term reforms, school leaders’ attitudes towards these evaluations are critical. Relying on organizational change theory, this study examines principals’ beliefs regarding school inspection feedback effectiveness and appropriateness as well as their readiness to implement changes. A large US district that has experimented with such school inspection serves as the case study. The results demonstrate the principals’ strong positive attitudes towards inspection, and these attitudes are associated with sustained change. But, because the accountability framework did not center on inspection, principals were ambivalent about implementing specific changes. This study highlights the tensions involved in incorporating on-site, expert evaluations in test-based accountability systems.