Assessing Writing最新文献

This study addresses a critical need in large-scale L2 writing assessment by emphasizing the significance of tailoring assessments to specific teaching and learning contexts. Focusing on the CET-4 writing test in China, the research unfolded in two phases. In Phase I, an empirically-developed analytic rating scale designed for the CET-4 writing test was rigorously validated. Twenty-one raters used this scale to rate 30 essays, and Many-Facets Rasch Model (MFRM) analysis was performed on the rating data. The outcomes demonstrate the scale’s robustness in effectively differentiating examinees’ writing performance, ensuring consistency among raters, and mitigating rater variation at both individual and group level. Phase II extends the research scope by applying the validated scale to score 142 CET-4 writing scripts. Utilizing Hierarchical and K-Means cluster analyses, this phase unveils three distinct score profiles. These findings are significant for both the CET-4 writing test and other L2 large-scale writing assessment. Theoretically, this study introduces a perspective that aims to enhance our understanding of learners’ performance in large-scale L2 writing assessment. Methodologically, this study presents a framework that integrates the validation of the rating scale with the identification of distinct score clusters, thus aiming to provide a more detailed solution for tailoring assessments to specific learning contexts.

Given increased prevalence of automated writing evaluation (AWE) systems in classroom settings, more research is needed to explore the potential for bias in automated scores with respect to English language learners (ELLs). Thus, this research study investigated and compared the predictive validity of automated and human scoring methods for elementary-age English ELLs on a writing test designed for ELLs and a state writing test designed for the general population. This study focused on the MI Write AWE system and sampled 2829 students comprising ELLs and non-ELLs in Grades 3–5. Results of multilevel regression analyses and simple slopes estimation indicated that, for ELLs, the automated MI Write score had similar predictive validity to the human score for both writing tests. However, automated and human scores for ELLs were less closely related to the state writing test score than scores for non-ELL students. Findings suggest that MI Write’s automated scoring was not uniquely biased relative to human scoring but does reproduce the same biases evident with human scoring. Implications and directions for future research are discussed.

Picture description (PD) tasks are widely used in second language (L2) teaching, research, and assessment to elicit and/or evaluate L2 learners’ writing performance. However, there is little research on the effects of the characteristics of such tasks on L2 writing performance. This study aimed to examine the effects of task difficulty and learner variables on the linguistic characteristics of responses to the 2020 version of the Duolingo English Test (DET) PD writing tasks. The written responses of 1439 test takers from four first language (L1) backgrounds and at different levels of L2 proficiency to 335 DET PD tasks at two levels of difficulty were analyzed and compared in terms of measures of fluency, accuracy, and complexity using various computer programs. The findings indicated that task difficulty did not affect writing performance significantly; high-proficiency learners tended to perform better than did their low-proficiency counterparts; responses receiving higher scores tended to be longer and more accurate; and learner L1 was significantly associated with writing grades and linguistic features. The findings and their implications for the DET PD task and the DET validity argument are discussed.

Research into the contribution of multimodality to language learning is gaining momentum. While most studies pave the way for new understandings of language teaching and learning, there is an increasing demand for comprehensive assessment practices, particularly within higher education contexts. A few studies have emphasized the importance of reflecting on and establishing criteria for the assessment of multimodal literacy. This is necessary to understand students’ contributions in detail and to provide them with effective support in developing their multimodal skills. This study discusses the assessment of multimodal writing in English for Specific Purposes (ESP) contexts. It presents the design of an analytical tool for assessing multimodal texts and provided an example of its application. This tool covers assessment categories such as language use, content expression, interpersonal meaning, multimodality, and creativity and originality. As an example, we focus on the multimodal writing of a video game narrative, a genre that requires the integration of multiple modes of communication to convey meaning more effectively. Finally, this study offers pedagogical insights into the assessment of multimodal literacy in ESP.

This paper examines secondary education students’ multilingual writing based on two input modalities to understand the influence of input on argumentation and sourcing strategies. Participants produced texts in Basque, Spanish and English based on a video or a text, and texts were analysed to explore their production of argumentation elements and sourcing strategies. Differences were found between input modalities in the use of data and rebuttals, and in copying. Across-language differences were also found in the use of data and counterarguments, and in the use of original ideas and paraphrasing. Additionally, complex argumentation elements elicited more original ideas than simple ones. Findings suggest that different writing sub-processes might be activated when composing from different sources and that argumentation and sourcing might be transferable across languages. These results may have important implications for educators in multilingual programs who aim to support their students in acquiring academic writing skills in multiple languages.

This study examined differences in the overall noun phrase (NP) complexity and the use of specific types of NP modifiers among Korean learners of Chinese as a second language (CSL) at different proficiency levels and first language (L1) Chinese speakers. Our data consisted of a corpus of 134 narratives produced by 103 Korean CSL learners (33 beginner, 37 intermediate, and 33 advanced) and 31 L1 Chinese speakers on the same topic. Each narrative was manually analyzed using seven measures of overall NP complexity and 26 measures based on specific types of NP modifiers. Two measures of overall NP complexity (i.e., complex NP ratio and total length of all complex premodifiers) significantly discriminated all learner proficiency levels. The frequency of several types of NP modifiers consistently decreased or increased from lower to higher proficiency levels, while that of several others showed cross-proficiency fluctuations. Advanced learners demonstrated comparable levels of overall NP complexity as well as similar frequency of usage of specific types of NP modifiers as L1 Chinese speakers, with the exception that the latter used significantly more multiple modifiers. Our findings have useful implications for L2 Chinese writing syntactic complexity research and L2 Chinese writing pedagogy.

Student feedback literacy is essential to writing in a foreign language context. Feedback is a process through which learners become more familiar with their work quality and integrate constructive criticism to produce stronger written work. Implicit in this process is the assumption that learners should not passively receive information but actively seek, generate, process, and use feedback to apply new knowledge in current or subsequent writing tasks. Learners’ metacognitive awareness and skills may influence their participation in feedback-related activities that inform the monitoring and control of their writing process and determine their writing performance. However, little attention has been given to a systematic investigation of metacognition in feedback literacy. The present study, drawing upon a self-regulatory perspective, bridges this gap by fulfilling two purposes: (a) to measure a scale on metacognition-based student feedback literacy; and (b) to delineate the predictive effects of different components of the scale on academic writing performance. The results provided evidence for metacognitive awareness and skills in student feedback literacy and showed the predictive effects of the scale, e.g., feedback-related strategies in participation (FRSP), motivation (M), feedback-related monitoring strategies (FRMS), and strategy knowledge (SK) on EFL learners’ performance in academic writing. Relevant implications were also discussed.