The Effect of Linguistic Factors on Assessment of English Language Learners’ Mathematical Ability: A Differential Item Functioning Analysis

Abstract

ABSTRACT Increasing linguistic diversity in classrooms has led researchers to examine the validity and fairness of standardized achievement tests, specifically concerning whether test score interpretations are free of bias and score use is fair for all students. This study examined whether mathematics achievement test items that contain complex language function differently between two language subgroups: native English speakers (EL1, n= 1 000), and English language learners (ELL, n= 1 000). Confirmatory Differential Item Functioning (DIF) analyses using a SIBTEST were performed on 28 mathematics assessment items. Eleven items were identified to have complex language features, and DIF analyses revealed that seven of these items (63%) favored EL1s over ELLs. Effect sizes were moderate (0.05 ≤ <0.10) for six items, and marginal ( <0.05) for one item. This paper discusses validity issues with math achievement test items assessing ELLs and calls for careful test development and instructional accommodation in the classroom.