Evaluating the Language- and Culture-Related Construct-Irrelevant Variance and Reliability of the Sense of School Belonging Scale: Suggestions for Revision

Abstract

Using expert reviews and item response theory (IRT), this study evaluated the language- and culture-related construct-irrelevant variance and reliability of the 2019 TIMSS sense of school belonging scale (SSBS) for grades 4 and 8. The five items of the SSBS, which were identical for both grades, were reviewed for the language- and culture-related construct-irrelevant variance evidence by 10 assessment experts who have expertise in Chinese and English languages and cultures. Further, the existing TIMSS 2019 SSBS data associated with the countries of China (Hong Kong), Singapore, Australia, and America were analyzed within the framework of IRT for estimating the item and scale information function values (i.e., reliability) of the SSBS. The experts did find language- and culture-related evidence in each of the five items that might lead to the construct variance of the SSBS across these four countries. The IRT results further indicated that the SSBS item and scale information function values varied considerably across these four countries. Suggestions for revising the SSBS are proposed for TIMSS program developers.