Assessing Writing最新文献

In this study, we conceptualize two approaches, model-based and text-based, grounded on mental models and discourse comprehension theories, to computerized summary analysis. We juxtapose the model-based approach with the text-based approach to explore shared knowledge dimensions and associated measures from both approaches and use them to examine changes in students' summaries over time. We used 108 cases in which we computed model-based and text-based measures for two versions of students' summaries (i.e., initial and final revisions), resulting in a total of 216 observations. We used correlations, Principal Components Analysis (PCA), and Linear Mixed-Effects models. This exploratory investigation suggested a shortlist of text-based measures, and the findings of the PCA demonstrated that both model-based and text-based measures explained the three-dimensional model (i.e., surface, structure, and semantic). Overall, model-based measures were better for tracking changes in the surface dimension, while text-based measures were descriptive of the structure dimension. Both approaches worked well for the semantic dimension. The tested text-based measures can serve as a cross-reference to evaluate students' summaries along with the model-based measures. The current study shows the potential of using multidimensional measures to provide formative feedback on students' knowledge structure and writing styles along the three dimensions.

Technology facilitates teacher corrective feedback on students' writing, but there is a need to examine how written, audio and screencast modes affect teacher's evaluative language of electronic (e-) feedback from linguistic approaches. By using the engagement resources of the appraisal framework within Systemic Functional Linguistics, this study examined the effect of written, audio and screencast modes on the instructor's evaluative language in his e-feedback on writing and the text revisions of 15 pairs of Saudi EFL learners. The linguistic analysis of the e-feedback revealed that the instructor's engagement resources differed across the three e-feedback modes. Specifically, the screencast and audio e-feedback modes were dominated by expanding resources (resources expanding the space for dialogue) as opposed to the prevalence of contracting resources (resources limiting/shutting down the space for dialogue) in the written feedback mode. Moreover, the audio and screencast feedback modes contained more statements and suggestions whereas the written feedback mode was dominated by commands/orders and suggested corrections. The content analysis revealed that the screencast e-feedback mode addressed a higher number of global issues in writing; however, the audio and written e-feedback modes addressed a higher number of local issues in writing. Despite the higher overall rate of successful text revisions resulting from the screencast and audio e-feedback modes, no significant differences were found except in relation to students' global text revisions. The study offers useful pedagogical implications for instructors in effectively responding to students' writing.

Despite the burgeoning research on exploring learner engagement with feedback, how second language (L2) learners’ engagement with feedback in different processing conditions influences their subsequent writing development is under-explored. This study examines the effects of individual and collaborative processing (languaging) of teacher feedback on Chinese lower-secondary school EFL learners’ writing development. Eighty-one students aged 13–14 with A1-A2 levels of English proficiency (according to the Common European Framework of Reference) from two classes and two experienced English teachers participated in the study. Students were provided with comprehensive teacher feedback and were asked to process feedback provided on three writing tasks through either individual written or collaborative oral languaging over six weeks. Pre-, post-, and delayed post-tests were administered. Students’ writing development was analysed using complexity, accuracy, and fluency measures, as well as content and organisation writing scores. Findings showed that the two conditions did not influence students’ writing complexity and fluency differently, while only the collaborative oral languaging condition contributed to students’ sustainable accuracy gains. Results based on the analytic writing scores suggested that students in the two conditions significantly improved content and organisation scores over time. Pedagogical and research implications regarding implementing the two feedback processing conditions are discussed.

Verbatim source use (VSU) in integrated argumentative writing tasks may enhance linguistic complexity of writing performance. This assistance might present an unequal advantage for test-takers across levels of writing proficiency, engendering validity and fairness concerns. While previous research has mostly examined the relationships between source use characteristics and proficiency levels, the relationship between VSU and linguistic complexity remains underexplored. To further unpack these relationships, this study examined both the direct impact of VSU on linguistic complexity of writing performances and its indirect impact through interaction with writing proficiency. Using natural language processing tools and techniques, we examined 34 linguistic complexity features and three VSU features of 3250 argumentative writing performances on a university-level English Placement Test (EPT). We performed exploratory factor analysis to identify linguistic complexity dimensions and applied mixed-effect models to examine how VSU features and proficiency level impacted these dimensions. Post-hoc analyses suggested weak direct impacts of different VSU features on linguistic complexity, which might reflect different essay writing strategies. However, no meaningful indirect impact was found. The findings help unravel the impact of VSU on argumentative writing and provide empirical evidence for validity arguments for integrated writing assessments.

The current study investigated the relationships among large-grained and fine-grained aspects of absolute syntactic complexity (SC) and expert-assessed writing quality of 446 argumentative writing samples of college-level Chinese EFL learners. Computational indices tapping into large-grained and fine-grained aspects of absolute SC were computed by TAASSC and L2SCA. Drawing upon rigorous SEM analyses, this paper demonstrated the utility of computational indices that tap into absolute SC. Overall, the measurements of absolute SC accounted for 42 % of the variance in human-judged overall writing scores. The results revealed that (1) noun phrase (NP) complexity was the underlying cause that determined trained raters’ judgement on argumentative writing quality; (2) among traditional large-grained indices, MLC, CN/C, and CN/T, were dependable metrics in representing SC and predicting writing quality; (3) among fine-grained indices, prepositional phrases and relative clauses as noun modifiers were prominent in representing NP complexity; (4) relative clause and adjectival modifiers had unique and complementary effects to large-grained NP complexity in affording explanations for human judgement; (5) the use of prepositions in NP was the most prominent contributor to the increase of large-grained NP complexity among the noun phrase modifiers in this specific corpus. Situated in previous research, the results provide an opportunity to evaluate L2 writing within the theoretical framework of absolute syntactic complexity.

The study examined the frequency and accuracy of four discourse competence (DC) parameters (i.e., establishment of a smooth old-to-new information flow, stance display, engagement with readers, and mastery of knowledge of discourse structure) in Hong Kong English-as-a-second-language (ESL) secondary students’ disciplinary research reports. The results showed that: (1) elements of establishment of a smooth old-to-new information flow constituted, on average, less than 9 % of the reports’ length, with mean accuracy rates ranging from 64.31 % to 80.33 %; (2) elements of stance display constituted, on average, no more than 3 % of the reports’ length, with mean accuracy rates ranging from 58.33 % to 98.17 %; (3) elements of engagement with readers constituted, on average, no more than 2 % of the reports’ length, with mean accuracy rates ranging from 33.33% to 88.64 %; (4) the mean accuracy rates of mastery of knowledge of discourse structure ranged from 0 % to 100 %. The findings reveal the secondary students’ DC in disciplinary research writing and pinpoint weak areas that instructors can work on. The proposed DC framework forms a foundation for future research on DC in discipline-specific research writing.

The study explored EFL students' development of syntactic complexity by employing the Latent Class Growth Analysis (LCGA) approach. A total of 214 tertiary EFL students from Southwest China were invited to write four argumentative essays over an academic semester. The unconditional models of LCGA were utilized to explore the optimal latent classes of students' development trajectories of syntactic complexity. The conditional models of LCGA were employed to investigate the predictive effect of English proficiency on the optimal latent classes. Results of the unconditional models revealed different latent classes of development trajectories for six indices of syntactic complexity rather than the remaining ones, which offers tentative evidence for the heterogeneity of L2 development trajectories. Results of the conditional models showed that English proficiency did not predict the membership in these latent classes. These results are discussed and implications for L2 instruction are attempted.

A systematic review prepared by the researchers or users is transparent, complete, and accurate information that gives deep insights into why the review was done, what they did, and what they found. Various tools can be utilized to conduct a systematic review of the research papers, which is essential for researchers or users who wish to write and publish papers and those who review and evaluate them. By conducting a systematic review, users can gain valuable insights into the landscape of existing literature and identify key trends and influential works in their field from references they used in the article. Many researchers enhance the strength of their systematic review articles by manually preparing and including the analysis of the preferred reporting items for systematic reviews and meta-analyses (PRISMA)-2020 checklist and flow diagram. Motivated by the aforementioned discussion, this paper uses a web-based application to provide a graphical visualization of the analysis and flow diagram for the PRISMA. This application is valuable for stakeholders or users involved in the research process, including researchers, students, editors, reviewers, academic institutions, funding agencies, and publication houses. The proposed application enables stakeholders or users to assess the quality of the articles considered for systematic review more effectively along with the student writing assessment. Offering a visual display of the results from individual studies and syntheses enhances their ability to analyze and interpret the findings of the research articles by analyzing reference of research papers concerning year, entry type, authors, keywords, and journals.

This research methods article introduces the open source PERSUADE 2.0 corpus. The PERSUADE 2.0 corpus comprises over 25,000 argumentative essays produced by 6th-12th grade students in the United States for 15 prompts on two writing tasks: independent and source-based writing. The PERSUADE 2.0 corpus also provides detailed individual and demographic information for each writer. The goal of the PERSUADE 2.0 corpus is to advance research into relationships between discourse elements, their effectiveness, writing quality, writing tasks and prompts, and demographic and individual differences.