Assessing Writing最新文献

While a large body of research has been accumulated that provides reliability and validity evidence for L2 integrated writing tasks, relatively little research has been conducted to examine integrated writing tasks as a means to provide diagnostic insights for teachers and learners. The current study aims to fill in this lacuna by applying a log-linear cognitive diagnostic model (LCDM) to reading-to-write integrated writing data collected from 315 Chinese college-level English as a Foreign Language (EFL) examinees. For this study, the integrated writing task was conceptualized as consisting of language use, source use, and content, with each of these unobservable attributes measured by surrogate indicators. Results showed that all the pairs of postulated attributes were positively correlated. However, the association between language use and content (r = 0.36) was not as strong as that of either language use and source use (r = 0.74) or source use and content (r = 0.90). Also, item parameters indicated that language use is more important than other attributes for obtaining a passing score for writing features. Lastly, the test-taker classification showed that it is impossible to master source use without other attributes, demonstrating the dependence of source use on other attributes. Implications for teaching are discussed.

Upon receiving constructive feedback, students may experience unpleasant emotions from critical comments about their writing or the realization that their work is unfinished. Few studies have focused on how learners are able to manage such emotions, one aspect of feedback literacy. Regulating these emotions may involve practicing self-kindness and avoiding self-judgment, two subcomponents of self-compassion. Self-compassionate individuals may move past any feelings of failure and direct their attention to what needs improvement. The question addressed was whether undergraduates’ level of self-compassion would affect their perceptions of the constructiveness of researcher-created feedback statements. At a U.S. southwest university, students (N = 508) rated the constructiveness of 56 statements that had been created to represent different levels of constructiveness in feedback to a fictitious writing assignment. Results indicated that students’ self-kindness positively predicted feedback constructiveness, whereas self-judgment was a negative predictor. Additionally, students higher in self-compassion (high in self-kindness in one analysis and those low in self-judgment in a second) rated the least constructive statements as more constructive than did students low in self-compassion. We end with implications for feedback literacy and writing assessment research and for application of self-compassion in the context of feedback on writing.

As writing on computers has become increasingly common in L2 assessment and learning activities, it is crucial to understand the mediation effects induced by the computer on writing performance and to compare them with those of handwriting. This is especially important for L2 Chinese learning, given that handwriting characters has been claimed to play an essential role in the development of Chinese literacy. The current study extends the scope of writing modality investigation by examining the linguistic, metadiscourse, and organizational properties of handwritten and typed essays by L2 Chinese learners. Furthermore, predictors of holistic ratings of writing quality were identified in the two modes to understand whether the focal points of raters’ evaluations may differ between the two mediums. The results yielded moderate to strong evidence about how the two modalities allow for distinct affordances, interact differently with the L2 (i.e., Chinese), and consequently affect writing performance in various dimensions.

This study investigated how common raters’ experiences of uncertainty in high-stakes testing are before, during, and after the rating of writing performances, what these feelings of uncertainty are, and what reasons might underlie such feelings. We also examined if uncertainty was related to raters’ rating experience or to the quality of their ratings. The data were gathered from the writing raters (n = 23) in the Finnish National Certificates of Proficiency, a standardized Finnish high-stakes language examination. The data comprise 12,118 ratings as well as raters’ survey responses and notes during rating sessions. The responses were analyzed by using thematic content analysis and the ratings by descriptive statistics and Many-Facets Rasch analyses. The results show that uncertainty is variable and individual, and that even highly experienced raters can feel unsure about (some of) their ratings. However, uncertainty was not related to rating quality (consistency or severity/leniency). Nor did uncertainty diminish with growing experience. Uncertainty during actual ratings was typically associated with the characteristics of the rated performances but also with other, more general and rater-related or situational factors. Other reasons external to the rating session were also identified for uncertainty, such as those related to the raters themselves. An analysis of the double-rated performances shows that although similar performance-related reasons seemed to cause uncertainty for different raters, their uncertainty was largely associated with different test-takers’ performances. While uncertainty can be seen as a natural part of holistic ratings in high-stakes tests, the study shows that even if uncertainty is not associated with the quality of ratings, we should constantly seek ways to address uncertainty in language testing, for example by developing rating scales and rater training. This may make raters’ work easier and less burdensome.

In writing instruction, specifying writing assignments in terms of purpose, audience, and medium is considered good practice. Earlier studies that found positive effects of such rhetorical specification were usually conducted with older participants. The benefits of rhetorical specification for novice writers are not yet clear, especially in the context of assessing writing. Thus, this study examined the effects of rhetorical specification on text quality of descriptions in an assessment prompt for fourth graders. Austrian fourth graders were assessed with the same paper-pencil-based L1-writing prompt but were randomly assigned within classrooms to one of three different conditions: high-level rhetorical specification (n = 78), medium-level rhetorical specification (n = 44), or no rhetorical specification (n = 44). The texts written by participants were rated holistically and analytically. The analysis revealed no differences between texts written by students under these three different conditions of rhetorical specification levels except for one single analytic indicator of text quality. Texts written in response to medium-level rhetorical specification scored higher on the rating of the criterion Adaptation to the audience than texts written under the other two conditions. The pros and cons of (high-level) rhetorical specification and good assessment practice with novice writers are being discussed in the findings.

What is involved in source use and how to assess it have been key concerns of research on L2 integrated writing assessment. However, raters’ ability to reliably assess the construct remains scarcely investigated, as do the relations among different types of integrated writing tasks. To partially address this gap, the present study had a sizeable sample (N = 204) of undergraduates from three Hong Kong universities write a summary and an integrated reading-to-write argumentative essay task in a test-like condition. Then, focusing on the criteria of source use, it analysed raters’ application of analytical rubrics in assessing the writing outputs. Rater variability and scale structures were examined through the Multi-Facet Rasch Measurement and compared across the two writing tasks. Both similarities and differences were found. In the summary task, the criteria for source use were applied similarly to the criteria for language use and discourse features. In the essay task, however, the application of the source use criteria was much less consistent. Diagnostic statistics indicate that fewer levels on the scale would be more advisable. For both tasks, the criterion of source language use was found not to fit the overall model nor to align with the criteria for source ideas or language use, indicating that this criterion may represent a trait different from the other. The statistical relations between source use and the other subconstructs of integrated writing tasks are also reported herein. Implications are discussed in the interest of refining the assessment of the source use construct in the future.

This study investigated the dimensionality and determinants of Korean-speaking sixth-grade students’ narrative writing. The sample included 113 Korean children in one elementary school, aged 12.48 years. The Story Composition component of the Test of Written Language (Hammill & Larsen, 2009) was employed to evaluate writing quality and the Writing Assessment Measure (Dunsmuir et al., 2015) rubric was used to assess seven component skills of narrative writing: handwriting, spelling, punctuation, sentence structure and grammar, vocabulary, organization and overall structure, and ideas. Results of a confirmatory factor analysis supported both a one-factor and a three-factor model of narrative writing among Korean students. Further, hierarchical linear modeling analysis revealed that better handwriting and sentence structure and grammar skills significantly predicted higher writing quality. Gender also uniquely contributed to students’ narrative writing quality, with females performing better than males. Overall, the effects on narrative writing quality were largest for gender, followed by handwriting, and then by sentence structure and grammar.

The present study focused on the assessment of how task complexity and learner variables (English proficiency level, self-regulated writing strategies, and writing self-efficacy belief) influence English academic writing for students in a foreign language context. The participants were 270 students from a medium-sized university in China. All participants completed measures on self-regulated writing strategies, self-efficacy, and an academic writing test. Guided research questions aimed to explore the extent to which task complexity and English proficiency level influenced writing performance along with how learners’ self-efficacy and self-regulated writing strategies mediated the role of task complexity in academic writing performance. Structural equation modelling results showed that task complexity and English proficiency level influenced learners’ writing performance. Self-efficacy beliefs and the use of self-regulated writing strategies mediated the role of task complexity on academic writing performance. Implications related to the assessment of task complexity, self-regulated writing strategies, self-efficacy, and academic writing were discussed.

The study utilized a mixed methods approach to compare the scoring of raters in assessing writing performance across three modes: paper-based, on-screen marking of scanned images, and online word-processed versions. six experienced raters evaluated the performances of 39 test-takers in each mode. The many-facet Rasch model was employed to analyze scoring differences among the rating modes; the semi-structured interview was used to collect raters' perceptions towards performance under the three modes. The findings indicated that the difficulty level was ranked in ascending order of on-screen marking of scanned images, paper-based text, and online word-processed text. Bias analysis revealed interactions between the rater and the mode, as well as between the criterion and the mode. Verbal reports from the raters highlighted four construct-irrelevant factors that could potentially influence scoring under the three modes: convenience for essay overview and word recognition, potential underestimation of word count, and raters' preference for essays in handwriting. Based on the results, recommendations were provided for rater training and essay scoring across different modes.

The mainstream approach to teacher assessment literacy seems to be founded on a (post)positivist paradigm leading to an autonomous model of literacy comprised of generic knowledge and skills. This paradigm obscures the non-cognitive, embodied, and affective dimensions of assessment practices. In this conceptual inquiry, I use the New Materialist philosophy to make sense of writing assessment literacy and feedback practices. In New Materialisms, the materiality of everything is emphasized, ontology is flat, reality is becoming, agency is relational, knowledge is entangled practice, and language is a resource in communicative assemblage. Using the noted conceptual tools, I try to provide a materialized conceptualization of writing assessment and feedback practice arguing that from a New Materialist perspective, feedback practices are an assemblage of rhetoric, IELTS, institution, materiality, art, cross-lingual resources, social relations, affect, and embodiment.