Amplifying test-taker voices in the validation of L2 writing assessment tasks

Studies have documented how L2 writers utilize digital tools to mediate the process of writing, yet little is known about test-takers’ perceptions of the utility of digital tools in writing assessment tasks. In a previous investigation (Pusey & Butler, in press), we found that international graduate students’ writing performance was different on assessment tasks that varied in terms of access to external writing resources (e.g., spell-check, dictionaries, internet content searches). To better understand the mechanisms underlying these different outcomes, participants’ (n = 20) perceptions of the two task conditions (with or without access to external resources) were examined based on a questionnaire. Results indicated that participants tended to perceive writing tasks that permit access to external resources as more similar to university writing and as better vehicles for demonstrating their academic writing ability. However, they also perceived this task condition as more difficult than the one which disallowed use of external resources. Regarding enjoyment, test-takers’ perceptions were almost evenly divided. Moreover, additional construct-irrelevant factors (e.g., topic, time, task characteristics and processes) appeared to influence perceptions of the tasks. The findings demonstrate the value of listening to test-taker voices to identify construct-irrelevant factors in writing assessments within the context of an increasingly digitalized world.