Assessing Writing最新文献

Verbatim source use (VSU) in integrated argumentative writing tasks may enhance linguistic complexity of writing performance. This assistance might present an unequal advantage for test-takers across levels of writing proficiency, engendering validity and fairness concerns. While previous research has mostly examined the relationships between source use characteristics and proficiency levels, the relationship between VSU and linguistic complexity remains underexplored. To further unpack these relationships, this study examined both the direct impact of VSU on linguistic complexity of writing performances and its indirect impact through interaction with writing proficiency. Using natural language processing tools and techniques, we examined 34 linguistic complexity features and three VSU features of 3250 argumentative writing performances on a university-level English Placement Test (EPT). We performed exploratory factor analysis to identify linguistic complexity dimensions and applied mixed-effect models to examine how VSU features and proficiency level impacted these dimensions. Post-hoc analyses suggested weak direct impacts of different VSU features on linguistic complexity, which might reflect different essay writing strategies. However, no meaningful indirect impact was found. The findings help unravel the impact of VSU on argumentative writing and provide empirical evidence for validity arguments for integrated writing assessments.

The current study investigated the relationships among large-grained and fine-grained aspects of absolute syntactic complexity (SC) and expert-assessed writing quality of 446 argumentative writing samples of college-level Chinese EFL learners. Computational indices tapping into large-grained and fine-grained aspects of absolute SC were computed by TAASSC and L2SCA. Drawing upon rigorous SEM analyses, this paper demonstrated the utility of computational indices that tap into absolute SC. Overall, the measurements of absolute SC accounted for 42 % of the variance in human-judged overall writing scores. The results revealed that (1) noun phrase (NP) complexity was the underlying cause that determined trained raters’ judgement on argumentative writing quality; (2) among traditional large-grained indices, MLC, CN/C, and CN/T, were dependable metrics in representing SC and predicting writing quality; (3) among fine-grained indices, prepositional phrases and relative clauses as noun modifiers were prominent in representing NP complexity; (4) relative clause and adjectival modifiers had unique and complementary effects to large-grained NP complexity in affording explanations for human judgement; (5) the use of prepositions in NP was the most prominent contributor to the increase of large-grained NP complexity among the noun phrase modifiers in this specific corpus. Situated in previous research, the results provide an opportunity to evaluate L2 writing within the theoretical framework of absolute syntactic complexity.

The study examined the frequency and accuracy of four discourse competence (DC) parameters (i.e., establishment of a smooth old-to-new information flow, stance display, engagement with readers, and mastery of knowledge of discourse structure) in Hong Kong English-as-a-second-language (ESL) secondary students’ disciplinary research reports. The results showed that: (1) elements of establishment of a smooth old-to-new information flow constituted, on average, less than 9 % of the reports’ length, with mean accuracy rates ranging from 64.31 % to 80.33 %; (2) elements of stance display constituted, on average, no more than 3 % of the reports’ length, with mean accuracy rates ranging from 58.33 % to 98.17 %; (3) elements of engagement with readers constituted, on average, no more than 2 % of the reports’ length, with mean accuracy rates ranging from 33.33% to 88.64 %; (4) the mean accuracy rates of mastery of knowledge of discourse structure ranged from 0 % to 100 %. The findings reveal the secondary students’ DC in disciplinary research writing and pinpoint weak areas that instructors can work on. The proposed DC framework forms a foundation for future research on DC in discipline-specific research writing.

A systematic review prepared by the researchers or users is transparent, complete, and accurate information that gives deep insights into why the review was done, what they did, and what they found. Various tools can be utilized to conduct a systematic review of the research papers, which is essential for researchers or users who wish to write and publish papers and those who review and evaluate them. By conducting a systematic review, users can gain valuable insights into the landscape of existing literature and identify key trends and influential works in their field from references they used in the article. Many researchers enhance the strength of their systematic review articles by manually preparing and including the analysis of the preferred reporting items for systematic reviews and meta-analyses (PRISMA)-2020 checklist and flow diagram. Motivated by the aforementioned discussion, this paper uses a web-based application to provide a graphical visualization of the analysis and flow diagram for the PRISMA. This application is valuable for stakeholders or users involved in the research process, including researchers, students, editors, reviewers, academic institutions, funding agencies, and publication houses. The proposed application enables stakeholders or users to assess the quality of the articles considered for systematic review more effectively along with the student writing assessment. Offering a visual display of the results from individual studies and syntheses enhances their ability to analyze and interpret the findings of the research articles by analyzing reference of research papers concerning year, entry type, authors, keywords, and journals.

The study explored EFL students' development of syntactic complexity by employing the Latent Class Growth Analysis (LCGA) approach. A total of 214 tertiary EFL students from Southwest China were invited to write four argumentative essays over an academic semester. The unconditional models of LCGA were utilized to explore the optimal latent classes of students' development trajectories of syntactic complexity. The conditional models of LCGA were employed to investigate the predictive effect of English proficiency on the optimal latent classes. Results of the unconditional models revealed different latent classes of development trajectories for six indices of syntactic complexity rather than the remaining ones, which offers tentative evidence for the heterogeneity of L2 development trajectories. Results of the conditional models showed that English proficiency did not predict the membership in these latent classes. These results are discussed and implications for L2 instruction are attempted.

Directed self-placement (DSP) allows for student agency in writing placement. DSP has been implemented in many composition programs, although it has not been used as widely for L2 writers in higher education. This study investigates the relationship between student placement decisions and students’ prior educational backgrounds, particularly in relationship to whether they had attended an English-medium high school or an intensive English program (IEP). Actual placement results via an exam were compared to 804 students’ self-placement decisions and correlated with their prior educational backgrounds. Findings indicated that most students’ DSP decisions matched actual exam placement results. However, there was a large number of DSP decisions that were higher or lower than exam placement results. Additionally, the longer students studied at an English-medium instruction high school, the more likely they were to place themselves higher than their exam placement. We conclude that DSP can be used in L2 writing programs, but with careful attention to learners’ educational backgrounds, proficiency, and sense of identity.

The 2024 Tools & Technology forum underscores the significant role of emerging writing technologies in shaping writing assessment practices post-COVID-19, emphasizing the necessity of ensuring that these innovations uphold core principles of validity, fairness, and equity. AI-driven tools offer promising improvements but also require careful consideration to ensure that they reflect writing constructs, align with educational goals, and promote equitable assessment practices. Validity is explored through dimensions such as construct, content, and consequential validity, raising questions about how assessment tools may capture the complexity of writing and their broader impacts on educational stakeholders. Fairness in writing assessment is examined with regard to cultural responsiveness and accessibility, and how assessment tools may be designed to accommodate various student needs. Equity extends these considerations by addressing systemic inequities and promoting assessment practices that support diverse learning styles and reduce barriers for marginalized students. The reviews of three assessment tools—PERSUADE 2.0, EvaluMate, and a web application for systematic review writing—illustrate how innovations can support valid, fair, and equitable writing assessments across educational contexts. The forum emphasizes the importance of ongoing dialogue and adaptation to create inclusive and just educational experiences.

This research methods article introduces the open source PERSUADE 2.0 corpus. The PERSUADE 2.0 corpus comprises over 25,000 argumentative essays produced by 6th-12th grade students in the United States for 15 prompts on two writing tasks: independent and source-based writing. The PERSUADE 2.0 corpus also provides detailed individual and demographic information for each writer. The goal of the PERSUADE 2.0 corpus is to advance research into relationships between discourse elements, their effectiveness, writing quality, writing tasks and prompts, and demographic and individual differences.

Despite the increasing scholarly attention towards students’ writing feedback literacy in recent years, empirical explorations of effective approaches to enhancing this capacity remain scarce. While peer feedback often plays an important role in English as a Foreign Language (EFL) writing development, few studies seem to have addressed the potential impacts of peer feedback activities on students’ overall writing feedback literacy. To fill this gap, a mixed-methods study was designed to investigate the effect of peer feedback activities on students’ writing feedback literacy development across such dimensions as appreciating feedback, making judgements, acknowledging different sources of feedback, managing affect, and taking actions with feedback. Two intact classes, one as the experimental group and the other control group, participated in the study. The experimental group engaged in peer feedback activities during the semester (12 weeks), whereas the control group received conventional teacher feedback only. The pre- and post-intervention results based on a writing feedback literacy scale were compared between the two groups, in addition to the analysis of interviews with the teacher and focal students from the experimental group, as well as students’ written assignments and revisions after receiving peer feedback. Results showed that peer feedback activities could significantly improve students’ appreciation of feedback and their ability to make judgements. Nevertheless, no significant changes in other dimensions were identified. These findings extend the current understanding of EFL students’ writing feedback literacy and hold valuable pedagogical implications.

Though much research has dealt with feedback practices in L2 writing classes, scarce studies have tried to investigate learner and teacher feedback perspectives from a wide angle. Drawing on an 8-dimension framework of feedback in writing classes, this study investigated the potential matches and mismatches between Saudi university students' English writing feedback preferences and their teachers' reported practices. Quantitative and qualitative data was collected using a student questionnaire and a teacher one. The two surveys assessed students' preferences for and teachers' use of 26 writing feedback modes, strategies and activities. A total of 575 undergraduate English majors at 11 Saudi universities completed the student questionnaire, and 82 writing instructors completed the teacher questionnaire. The data analysis revealed that the differences between the students' English writing feedback preferences and their teachers' practices vary from one feedback dimension to another. The study generally indicates that the mismatches between the students' writing feedback preferences and the teachers' reported practices far exceed the matches. The qualitative data obtained from the answers to a set of open-ended questions in both questionnaires provided information about the students' and teachers' feedback-related beliefs and reasons. The paper ends with discussing the results and their implications.