A comparison of contextualized and non-contextualized meaning-recall vocabulary test formats

Meaning-recognition and meaning-recall are two commonly-used test modalities to assess second language vocabulary knowledge for the purpose of reading. Although considerable variation in item format exists within each modality, previous research has examined this variation almost exclusively among meaning-recognition item types. This article reports on two exploratory studies, each comparing a fully-contextualized and a non-contextualized meaning-recall variant for one specific testing purpose: coverage-comprehension research. The fully-contextualized test utilized the same 622-word passage in each study. In the non-contextualized tests, target words appeared in short, non-defining sentences; in Study A, the elicited response was a translation of only the target item while in Study B, it was the entire prompt sentence. Scores on the compared tests differed significantly only in Study A. In both studies, the consistency with which the compared item formats yielded the same outcome (correct or incorrect) when the same target word was encountered by the same learner was rather low. The provision of relatively authentic context sometimes seemed to aid lexical inferencing, but other times it increased task difficulty relative to the limited-context formats. These findings suggest that different meaning-recall formats could lead to different conclusions regarding knowledge of specific words, and this could impact coverage-comprehension research findings.