Evaluating the Use and Interpretation of the TOEIC® Listening and Reading Test Score Report: Perspectives of Test Takers in Japan

Abstract

Researchers suggest that claims about the meaningfulness of test score interpretations and consequences of test use should be backed by evidence that stakeholders understand the definition of the construct assessed (meaningfulness) and score reports (consequences). Evaluation of stakeholders' actual uses and interpretations of score reports in large-scale standardized language proficiency tests, however, remains limited in the score reporting literature. This study investigates how test takers, as an important stakeholder group, use and interpret the score report of the TOEIC® Listening and Reading (TOEIC L&R) test. Data were collected from 834 TOEIC L&R test takers in Japan, who represented a wide range of English language proficiency based on their TOEIC L&R total scores. The participants responded to an online survey that asked about their uses and interpretations of the test results and their comprehension of the performance feedback presented in the score report. The results showed that the participants used the test results largely as intended, providing an important piece of validity evidence to support the proposed uses of the TOEIC L&R test. Some of the score reporting elements such as the performance feedback and footnote message, however, were not easy to understand for all participants, revealing a need to improve the interpretability of the score report. The study findings have implications for designing informative score reports and the usefulness around reporting test performance feedback.