Assessing Writing最新文献

Informed by task-based approaches to language teaching, recent L2 writing research has sought to determine the effect of task complexity features on the complexity, accuracy, and fluency of written L2 production (Johnson, 2017). However, two areas of task-informed research have received scant attention: a) the effect of task complexity features on L2 writers’ use of cohesive devices and b) the effect of task repetition as a form of implicit planning. Furthermore, interaction effects of task complexity and task repetition on different types of cohesive devices in L2 writing have not been explored. To bridge these gaps, this study examines the effects of resource-directing task complexity features (Robinson, 2005), task repetition (Lambert et al., 2017), and their interaction on L2 writers’ use of cohesive devices. Ninety-six participants composed two argumentative essays in counterbalanced order: a) a simple task and b) a complex task and then completed a task difficulty questionnaire. After an interval of one week, the participants repeated each task. Essays were then analyzed using the Tool for Automatic Analysis of Cohesion—or TAACO (Crossley et al., 2018)—for indices found to be predictors of human ratings of essay organization (Abdi Tabari & Johnson, 2023). A factorial repeated-measures MANOVA revealed limited effects of task repetition on the participants’ use of cohesive devices. Rather, task complexity features had a more robust effect on their use of textual and local cohesive devices.

The present study investigated the effects of task complexity and language aptitude on upper-intermediate EFL learners’ argumentative writing performance in terms of syntactic complexity, lexical complexity, accuracy and fluency. The findings of this study demonstrated that increasing task complexity manipulated by the number of elements and the degree of reasoning along the resource-directing dimension leads to enhancement of syntactic complexity and lexical diversity, and there are low correlations between language aptitude (mainly number learning and spelling clues) and writing performance. What’s more, task complexity and language aptitude (and its components) are predictors for writing performance in terms of intraclausal level of syntactic complexity, lexical diversity and fluency. These findings lend partial support to the Cognition hypothesis and Aptitude Complexes Hypothesis in L2 writing. Theoretical, methodological, and pedagogical implications of the study for task design and implementation as well as for task-based assessment in language education programs are discussed.

Reading and writing are core activities in higher education, by means of which students learn to participate in specialized discourses. Although there is consensus on the conceptualization of reading comprehension, its measurement, and development, the same is not true for written expression. Writing complexity has been found to improve with schooling, but there are ample differences between literacy practices at school and at the university that require extra attention in diagnosing students’ compositions. The present study set out to test a natural language processing tool to build domain profiles of writing complexity in first-year university students at a private university. The processing of texts resulted in 49 indices which, after exploratory factor analysis and theoretical discussion, gave rise to 4 dimensions of complexity explaining 52.3% of variance: lexical richness, syntactic complexity, informative text structure and specialized language use. Significant differences were found between more and less skilled writers in the aggregated scores, lexical richness, and syntactic complexity. Interestingly, novice and expert writers did not differ significantly in more over-arching aspects of writing. We discuss how this technology can help identify students’ needs in more superficial aspects of writing complexity that have been shown to improve by means of different strategies.

With the objective of improving writing assessment of language instruction, we examine the lexical and syntactic features in two corpora of high and low scoring French texts of the Test du Certificat de Compétence en Langue Seconde (Second Language Certification Test; TCCLS) at the University of Ottawa (uOttawa). We first situate the test in its local context, demonstrating how our research objectives are born from specific needs to improve student outcomes. We then describe our creation of two corpora of high and low performing test takers, followed by lexical bundle (LB) analyses (Phase 1) and further linguistic complexity analyses with a French-language tool (Phase 2). Results indicate that high level writers used more LBs and borrowed more text from the prompt than low level writers. In addition, specific elements of linguistic complexity were identified, suggesting high level writers produced texts that were lexically richer and more syntactically advanced. We discuss the importance of these findings in improving our writing instruction, as well as the challenges of adapting tools and approaches traditionally associated with English to French.

The torrent of research into peer feedback in academic writing in recent years has largely overlooked student revision process – how individual students engage with this feedback to revise their texts and why certain changes are made in their texts. In other words, the cognitive dimension of student engagement with peer feedback in revision is little known. Drawing on multiple student drafts, peer feedback on these drafts, reflective journals and interviews with students, this study examines how two L2 students engage with peer feedback to conduct revisions. We found that the two participants differed considerably in their revision processes and identified two patterns of engagement: deep engagement, characterized by self-regulated revising practices, and surface engagement, concerned with other-regulated revision operations. We were not only interested in students’ revision operations in their drafts, but also their reflective practices in their reflective journals that their teacher had assigned. The study suggests that effective student engagement with peer feedback largely depends on how they make use of cognitive and metacognitive strategies in the revision process. The study recommends that teachers seek to provide instructional scaffolding to facilitate student cognitive engagement with peer feedback on L2 writing.

Although many studies have elaborated the nature and importance of feedback literacy, a longitudinal approach to delving into the development of EFL students’ feedback literacy is scarce. Informed by an evidence-based framework of feedback literacy improvement in the context of academic writing (Yu & Liu, 2021), the study explored the development of feedback literacy of three first-year students who studied English as a Foreign Language (EFL) in understanding, evaluating, reflecting on, and using peer feedback and their emotional responses during a semester-long online writing program. By analyzing students’ voiced peer feedback, written drafts, immediate reflections after peer feedback, monthly reflection journals, end-of-term stimulated recalls and interviews, we found positive changes in their cognitive, behavioral, and emotional capacities in response to peer feedback, and increasing improvement in students’ assessment criteria knowledge through self-reflections after peer feedback, incorporation of peer feedback and reflections into revision. However, students did not develop at the same pace due to personal factors. Variations in students’ reflections and proactive actions were found in relation to the development of feedback literacy evidenced in their revision improvement. Implications for pedagogical facilitation of peer feedback literacy and use of peer feedback in revision in academic writing are discussed.

Research on corrective feedback (CF) has developed from its original focus on identifying which type of CF is most effective for developing L2 language learners’ grammatical accuracy to focusing on how learners use CF. Underpinning this is the assumption that learners know what to do with CF when they receive it. The concept of “feedback literacy” challenges this assumption. Carless and Boud (2018), define feedback literacy as “the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies” (p. 1316). Our intention in this paper is to reflect on the manner in which theoretical and empirical work on feedback literacy can contribute to advancing L2 written corrective feedback (WCF) research agendas. Central in our proposal is the partially under-researched aspect of experience in terms of the L2 writers’ educational background experience, particularly experience with L1 and L2 writing. We further argue that how learners were taught L1 writing and how the L1 educational culture/ society values writing can impact on how learners approach L2 writing tasks and accompanying feedback. Implications of this inclusive view of the learner for future research and pedagogy is discussed.

Advocating for measuring a broadened construct of writing ability, scenario-based assessment (SBA) has recently been used to assess writing ability. While SBA for writing has been explored in ESL contexts (e.g., Banerjee, 2019; Purpura, 2021), it has been rarely used or examined in the context of languages other than English, especially those that are typologically distant from English, such as Korean. This study explores the feasibility of designing a scenario-based Korean writing assessment (K-SBA). A pilot study was conducted with 51 participants from a Korean as a foreign language program at a US university. Through a goal-oriented scenario of a study abroad program in South Korea, examinees were presented with a collaborative problem-solving task where they were expected to learn about two potential class trip destinations and write a summary of the pros and cons of each destination based on what they learned. The results indicated that the K-SBA was psychometrically sound, providing adequate evidence of reliability. It elicited construct-relevant performances reflecting features unique to the Korean language, such as sociolinguistic competence through the use of honorifics or rhetorical control through a range of cohesive devices. Additionally, variations in examinee performances were observed across the different course levels.

The literature on peer feedback has yet to investigate the effects of translanguaging on learners’ affective engagement with feedback. Therefore, this case study 1) explored the affordances of translanguaging between two English learners in Iran, and 2) investigated any developments in these learners’ affective engagement with peer feedback mediated by translanguaging. The analysis of eight samples of student writing with peer feedback, stimulated recalls, and reflection journals showed that translanguaging mediated the learners’ affective engagement with peer feedback. Using their first language, Persian, helped improve their affective engagement with peer feedback on content-related issues. As shown by a sociocultural theory-motivated analysis, the learners’ affective engagement with peer feedback was nonlinear, dynamic, and evolving over the course of six weeks. Indeed, the participants deemed Persian as an effective tool in providing meta-feedback on content-related, and at times, grammar issues, thus leading to more positive affective engagement with peer feedback. We think that teachers should consider allowing their students to use their first language during the process of peer feedback to enhance their affective engagement with peer feedback.

The orthographic and morphological system of Mandarin Chinese requires more time and developmental stages for learners to acquire. This source of difficulty might present unique challenges and opportunities for writing assessment for Chinese as a Second Language (CSL). This study employed a corpus-based approach to examine the accuracy features of 10,750 essays written by test-takers from 17 first language (L1) backgrounds on the HSK test. Based on both orthographic types and economic-geopolitical factors, we classified test-taker L1s into 3 groups. We first factor-analyzed a comprehensive array of error types to identify the underlying dimensions of Chinese writing accuracy. Then, dimension scores were included in regression models to predict HSK writing scores for different L1 groups. The results revealed five dimensions related to syntactic, morphological, and lexical errors. Among them, dimensions on character and word-level errors were stronger predictors of HSK scores, although the discrimination power was stronger for test-takers from L1s that are orthographically dissimilar and economic-geopolitically distant from Mandarin Chinese. These findings suggest that Chinese morphology (i.e., the acquisition of characters and how characters form words) constitutes a unique source of difficulty for L2 learners. We argue that morphological elements should be an important subconstruct in Chinese writing assessments. (200 words)