Topic Familiarity Effects on Performance in Speaking Assessment Tasks

Abstract

Speaking assessment is believed to be difficult in its expansion and execution. Thus, it is a challenge for teachers in preparing students for speaking tests. This study’s purpose was to identify whether topic familiarity affects speaking performance among students who were preparing to sit for the high-stakes Malaysian University English Test speaking test. The study aimed to investigate the validity and reliability of the speaking assessment measures and, subsequently, to examine the speaking tests’ item difficulty measures differences according to topic familiarity level. Data were collected from 40 non-native speakers of English among Malaysian Form Six pre-university students who were preparing for their MUET test. The researcher conducted two practice speaking tests, which used retired papers of the MUET CEFR-aligned speaking tests, to the 40 participants who were grouped into 10 speaking test groups. The practice speaking tests were video and audio-recorded. Topic familiarity was measured using self-report questionnaires. In the second phase, each of the seven appointed raters scored all 40 students’ responses in two speaking tests consisting of two speaking tasks assessed across four criteria: task fulfillment, language, communicative ability, and group discussion. Many-facet Rasch measurement results revealed significant differences translating to significant influence of topic familiarity on speaking performance. The present study’s results not only confirmed the significance of topic familiarity in the preparation for speaking assessments, but also highlighted the need for formal teaching on potential topic themes that are commonly encountered in such assessments, particularly those that are at the post-secondary level. These findings imply importance in designing speaking tests taking into account test-taker topic familiarity.