Studies in Educational Evaluation最新文献

Although learner characteristics and item properties are known to affect students’ reading comprehension performance, little research has focused on the interactions between these two types of features. This study investigated the effects of learner factors (i.e., gender, the main language spoken at home, multilingualism, and instruction on reading strategies) and item factors (i.e., item format and cognitive process), as well as their interactions, on Canadian students’ performance (n = 16,688) in the reading test of PISA 2018. The results indicated that all learner and item factors significantly predicted reading comprehension. Open-response items were more difficult for male students compared to their female peers. Receiving instruction on reading strategies was most helpful in answering the understanding type of items. Students who mainly spoke the test language at home were specifically better at answering items focused on locating information, while multilingualism appeared to compensate for not speaking the test language at home.

This paper explores the contributions of co-teaching to learning-oriented assessment in eight co-taught undergraduate courses, over two academic years, in the fields of Education and Translation and Interpreting; specifically, whether collaborative teaching enhances assessment design and practices while developing instructors’ and students’ assessment literacy. Evidence from reflective teacher diaries and student focus group data was triangulated, with results suggesting a positive effect of co-teaching on formative assessment, especially regarding the quantity and quality of feedback received by students. Concerning summative assessment, transparency and communication were found to be key factors in reducing student concerns over the grading process. Finally, co-teaching appears to enable a two-way street encouraging students to provide feedback to their instructors and, in the process of doing so, develop their own feedback literacy. Overall, co-teaching, when well-planned and coordinated, can facilitate learning-oriented assessment, as well as provide a fruitful space for instructors’ professional development in assessment practices.

Increasing investment in educational ICT resources is a strategy adopted worldwide to improve student performance in the digital era. Nevertheless, whether the investment is effective and how to realize its positive effect remains unclear. To address these limitations, the dataset of PISA 2018 in five countries and regions with superior academic performance was used, and multilevel structural equation modelling was applied. The results indicated that, first, educational ICT resources are negatively associated with student academic performance. Second, educational ICT resources positively correlate with cognitive-motivational, curriculum-based engagement, and non-curricular engagement. School and home educational ICT resources exhibit strong associations with curriculum-based and cognitive-motivational engagement, respectively. Third, student engagement mediates the relationship between educational ICT resources and academic performance, and this mediating role is different at the school level and the student level. Last, the above relationships show heterogeneities across research countries and regions. Practical implications for governments, schools, and teachers were discussed.

This study examined boundary-crossing collaboration in a research-practice partnership (RPP) in initial teacher education in Germany, exploring improvements for the theory-practice link. This study aimed to evaluate this particular cooperation format, especially to identify success factors for RPPs and differences between actor groups. Characteristics examined include motivational factors, collaboration aspects, co-constructive knowledge acquisition, competence enhancement and satisfaction. The study drawed on a quantitative survey with 78 participants from four groups: school practitioners, researchers, teacher students, partners from extracurricular organizations. A one-way ANOVA was conducted, and a path model on effective relationships was calculated. Results revealed that RPPs are collaboration “on equal footing”; participation enables professional development in teaching, assessing, and innovation; sharing practical experiences is the dominant factor in knowledge co-construction and main driver of success; and motivation to participate is mainly regulated intrinsically. This study contributes to the international debate on RPPs and advocates their establishment in teacher education.

Most of previous studies examined the consistency among two or three student raters giving scores to different peers (i.e. ratees), whereas few of them researched the consistency among more student raters giving scores to different peers (i.e. ratees). The present study aimed to see whether there is a consistency among student raters giving scores to peers in an online assessment activity. Participants of the study were 266 graduate students, with 137 males and 129 females, from 27 classes in a department at some university. Each student was required to give a presentation graded by the peers anonymously. From to the perspective of class, the result of the Kendall coefficient of concordance showed that a proportion of more than half-classes presented a significant correlation with a moderate or strong consistency among student raters giving scores to peers, and thus the number of raters or peers should be more for a higher reliability. The result of Chi-square test from the perspective of individual revealed that students held a consistent standard when giving scores to peers' presentations. Finally the implications for theories and practices are proposed.

Being able to self-regulate can positively impact learners’ academic achievement. An inherent catalyst of Self-Regulated Learning (SRL) is internal feedback, the new knowledge which is generated when comparing current knowledge against reference information. Learners may not always generate internal feedback, hampering further SRL. Supporting SRL can be done with a Learning Analytics Dashboard (LAD), in which reference frames allow for comparisons and facilitate internal feedback generation. This study explores internal feedback generation using a LAD and the effect of reference frame availability. A multiple method design examined the interplay of reference frames, comparison processes, internal feedback generation and preparatory activities engagement. Differences between three conditions were explored using Bain ANOVA’s. Results showed that reference frames almost exclude other external comparators and are used in parallel with an internal comparator. A peer reference frame leads to most verbalizations of internal feedback, and potentially to most verbalizations of preparatory activities.

Many studies show cognitive gains from peer feedback. However, no previous study has explored reciprocal and nonreciprocal peer tutoring and mentoring, with associated implications for peer feedback. University students (n = 446) completed an online questionnaire at the end of their first year. Participants numbered 166 (37 %), while comparison non-participants numbered 280. The questionnaire investigated social and academic integration, using elements of three published scales of known reliability and validity. Analysis focused on participants vs. non-participants, with supplementary comparison of effect sizes regarding impact of the different interventions. Results showed nonreciprocal peer tutoring better enhanced students’ academic integration. However, reciprocal peer mentoring better enhanced social integration. Nonreciprocal peer mentoring better enhanced student persistence. Types of feedback between methods were explored. Overall, the type of intervention(s) recommended might be tailored to the presenting needs of each student. Informing students of the likely outcomes of different types of peer assisted learning should assist them to choose optimally effective forms for their own purposes.

This study reports on the role of students’ reflections in their teacher written feedback. Informed by a qualitative case study approach, this study collected data from 18 students across a semester of an English-as-a-foreign language writing course at a university in China. In the course, teacher written feedback and students’ reflections were both informed by writing as a meaning-making process. The data sources included 72 pieces of student essays, 144 pieces of reflection written by the students, and post-semester interviews with the students. The study showed that engaging students in writing reflections was helpful for developing their knowledge of writing, although the process somewhat followed a zigzag trajectory. The exercise helped them transcend their previous knowledge repertoires, thereby assisting them in critically understanding their revision process and refining their knowledge of writing in response to the teacher’s written feedback. In particular, the students felt that the meaning-making-based reflections provided clear and accessible layers for them to understand the close relationship between language and content. As such, the students felt that they were able to harness their reflections as a written venue through which they could critically digest the teacher’s written feedback and develop their knowledge of writing as a meaning-making process while dialoguing with their instructor.

Sustaining professional learning communities (PLCs) focused on data use is a challenge for schools. Sustainability is achieved by making the PLC’s core components evident in schools’ organizational routines. Leadership can support sustainability according to literature, but it is unclear what leadership for sustainability looks like in practice. We therefore investigated what sustainable data use PLCs and leadership look like in three secondary schools, and explored how leadership influences sustainable data use PLCs. A mixed methods design (observations, social network questionnaire, school documents) was used to gain in-depth insight. The findings show that sustainability is difficult to achieve at both the ostensive and performative level. The schools differed in the form of sustainability they achieved. A broad set of leadership practices (e.g., organize meetings, participate actively in PLC, know content and procedure) were carried out in interplay by both formal and informal leaders.

Despite the generally positive learning effects of peer-assessment (PA), there are some undesirable signs and troublesome behavior needed to be addressed (e.g., biased judgments, superficial-level engagement). Anchoring on the concept of individual accountability and assessment validity, while attending to other important elements (including both subjective and objective criteria, individuals’ differentiation ability across the points on the assessment scale, and completion rate), a measure was proposed for PA quality control. This measure consists of four indexes/variables and enables the quantification and appraisal of the peer-assessor’s performance. A validation study involving two classes of undergraduate students (N = 103) engaged in the assessment of peers’ oral presentations was conducted to evaluate the effects of the proposed measure. The statistically significant results confirmed the efficacy of the proposed measure to promote better online PA performance. Additionally, to assess its applicability, sensitivity analysis targeting different aspects of our proposed measure (i.e., different plausible weighting for each of the associated indexes/variables, different correlation values between peers’ and the teacher’s ratings, and different bound values for the scale) was conducted. All significant results of the 281 test cases further substantiated the applicability and efficacy of our proposed measure under various assessment schemes for PA quality control.