Developing a New Task to Measure Speech Perception Ability: Is the Word Count Task Valid and Reliable?

Abstract

To become an advanced second language listener, a learner needs to have good speech perception ability. Previous research that focused on measuring this ability often utilized integrated-skills tasks (e.g., repetition tasks and dictation tasks), but their validity and reliability are questionable because learners’ productive skills affect their task performance. This study attempted to develop an original discrete-point task called the word count task, in which learners count and report the number of words in blanks. To evaluate the task’s validity and reliability, two comparable studies with dictation tasks were conducted with university students in Japan. The second study, which was revised based on the first study, revealed that the reliability coefficient of the word count task expressed by Cronbach’s (cid:302)(cid:3) reached .85, slightly exceeding the reliability of the dictation tasks. Moreover, correlations with dictation tasks were found to be significantly positive with moderate to strong relationships, meaning the word count task demonstrated sufficient criterion-related validity. Moreover, the listening strategy survey conducted to explore cognitive processes involved in the task showed that phonological processing is more dominant than meaning processing in the word count task. These findings seem to corroborate the applicability of the word count task to research and classroom assessment, but further research is necessary to reevaluate its validity using other methods mentioned in this study. The first study attempted to examine the validity and reliability of a prototype of word count task (WCT) and reveal how the task should be revised. As explained, there is no established task to measure speech perception ability; therefore, the validity of WCT was tested based on its relationship with a dictation task (DT). As already stated, DTs have shortcomings as a measure of speech perception ability, but due to their popularity in educational settings in Japan, using one is the best counterpart to the WCT.