Journal of applied measurement最新文献

Research using the National Teacher and Principal Survey (NTPS) has consistently demonstrated that teachers' reported working conditions are related to both intentions to leave the profession and attrition (Tickle, Chang, and Kim, 2011). However, limited research evaluates teacher appraisals of job-related demands and resources as an antecedent to job dissatisfaction. We tested for differential item functioning (DIF) using a partial credit model approach within a Rasch modeling context to examine whether elementary and secondary teachers with similar overall stress levels respond to the NTPS Demands and Resources items in similar ways. For the Demands items, seven of the items displayed differences that were negligible, four were intermediate, and three items indicated large DIF contrasts. For the Resources items, 10 items displayed differences that were negligible, two were intermediate, and zero items indicated large DIF contrasts. These results indicate elementary and secondary teachers exhibit different appraisal patterns, suggesting implications for the development and use of survey data in public school settings in general, and for the use of the NTPS data in particular.

This paper describes the process of assessing the unidimensionality and validity of egalitarian education (EE) items based on the Rasch measurement model. Egalitarian education was measured by a self-developed 5 EE items of Likert-scale format. The process of assessing the validity of EE items involved a collection of data from 400 Malay teachers, who are teaching in government school around peninsular of Malaysia where the measurement of construct validity for the overall EE items were established using Winsteps. Various Rasch measurement tools were utilized to demonstrate the true unidimensionality and validity measure of the EE items and in meeting the needs of the Rasch measurement model. The findings show that the validity and unidimensionality of EE items can be truly established and can satisfy the characteristics of the Rasch measurement model.

In previous studies, researchers have focused on the development and interpretation of measurement tools related to self-efficacy. However, researchers have seldom investigated whether these instruments demonstrate acceptable psychometric properties, including similar item interpretations between subgroups of respondents. The purpose of this study was to explore the extent to which a self-efficacy measure has a consistent interpretation for two self-reported gender subgroups. The researchers utilized Rasch analysis to explore differences in item difficulty between the subgroups. Results suggested differences in item difficulty ordering for certain self-efficacy items. Implications for research and practice are discussed.

In this study we investigate whether transformations between different representations of mathematical objects constitute a suitable framework for the assessment of students' comprehension of fraction addition. Participants (N = 164) solved a set of 20 fraction addition problems constructed on the basis of Duval's (2017) theory of the role of representational transformations in mathematical comprehension. Using Rasch measurement theory and principal component analysis, we found that the items could be separated into three levels of difficulty based on the transformation involved. This large-scale structure was consistent across gender and across subgroups of preservice teachers and middle-grade students. On a finer scale, the production of diagrammatic representations, and the type of diagrammatic representation involved, constitute potential subdimensions of the instrument. We conclude that transformations between representations can be productive for the assessment of fraction addition comprehension as long as care is taken to curtail the potential effects of multidimensionality.

To understand the role of fit statistics in Rasch measurement is simple: applied researchers can only benefit from the desirable properties of the Rasch model when the data fit the model. The purpose of the current study was to assess the Q-Index robustness (Ostini and Nering, 2006), and its performance was compared to the current popular fit statistics known as MSQ Infit, MSQ Outfit, and standardized Infit and Outfit (ZSTDs) under varying conditions of test length, sample size, item difficulty (normal and uniform), and dimensionality utilizing a Monte Carlo simulation. The Type I and Type II error rates are also examined across fit indices. This study provides applied researchers guidelines the robustness and appropriateness of the use of the Q-Index, which is an alternative to the currently available item fit statistics. The Q-Index was slightly more sensitive to the levels of multidimensionality set in the study while MSQ Infit, Outfit, and standardized Infit and Outfit (ZSTDs) failed to identify the multidimensional conditions. The Type I error rate of the Q-Index was lower than the rest of the fit indices; however, the Type II error rate was higher than the anticipated beta = .20 across all fit indices.

Most research on multistage testing (MST) uses simulated data. This study adds to the literature by using both operational test data and simulated data to compare two different MST designs with regard to proficiency estimation accuracy and module exposure rates and by investigating whether simulation studies and operational test studies yield similar results. Two MST designs (1-2 and 1-3-4 designs) from one state's sixth-grade summative mathematics assessment across two years were compared in this study. Both simulation and operational test studies demonstrate similar results: the two MST designs yield no significant performance differences with regard to estimation accuracy and module exposure. These results provide evidence that simulation studies can provide adequate results to inform decisions about MST designs.

An examinee- and an item-centered procedure are proposed to set cut scores for counts data. Both procedures assume that the counts data are modelled according to the Rasch Poisson counts model (RPCM). The examinee-centered method is based on Longford's (1996) approach and links contrasting-groups judgements to the RPCM ability scale using a random logistic regression model. In the item-centered method, the judges are asked to describe the minimum performance level of the minimally competent student by giving the minimum number of correct responses (or, equivalently, the maximum number of admissible errors). On the basis of these judgements for each subtest, the position of the minimally competent student on the RPCM ability scale is estimated. Both procedures are illustrated with a standard-setting study on mental arithmetic for students at the end of primary education.

Item difficulty and discrimination index are often used to evaluate test items and diagnose possible issues in true score theory. The two statistics are more related than the literature suggests. In particular, the discrimination index can be mathematically determined by the item difficulty and the correlation between the item performance and the total test score.

This simulation study explores the effects of missing data mechanisms, proportions of missing data, sample size, and test length on the biases and standard errors of item parameters using the Rasch measurement model. When responses were missing completely at random (MCAR) or missing at random (MAR), item parameters were unbiased. When responses were missing not at random (MNAR), item parameters were severely biased, especially when the proportion of missing responses was high. Standard errors were primarily affected by sample size, with larger samples associated with smaller standard errors. Standard errors were inflated in MCAR and MAR conditions, while MNAR standard errors were similar to what they would have been, had the data been complete. This paper supports the conclusion that the Rasch model can handle varying amounts of missing data, provided that the missing responses are not MNAR.

The main objective of this study is to develop and validate a sources of mathematics self-efficacy (SMSE) scale to be used in a polytechnic adopting Problem Based Learning (PBL) as its main instructional strategy. Based on socio-constructivist learning approach, PBL emphasizes collaborative and self-directed learning. A non-experimental cross-sectional design using a questionnaire was employed in this study. The validation process was conducted over three phases. Phase 1 was the initial development stage to generate a pool of items in the questionnaire. In Phase 2, a pilot test was performed to obtain qualitative and quantitative feedback to refine the initial pool of items in the questionnaire. Finally, in Phase 3, the revised scale was administered to the main student cohort taking the mathematics module. The collected data from the questionnaire was subjected to empirical scrutiny, including exploratory factor analysis (EFA) and Rasch analysis. The participants for this study were first year polytechnic students taking a mathematics module. There were 29 participants taking part in Phase 2 of the study, comprising 12 (41%) females and 17 (59%) males. For Phase 3, there were 161 participants, comprising 91 (57%) males and 70 (43%) females. The EFA yielded a three-factor solution, comprising (a) personal experience; (b) vicarious experience; and (c) psychological states. The items in the SMSE scale demonstrated good internal consistency and reliability. The results from the Rasch rating scale analysis showed an acceptable item and person fit statistics. The final 23-item SMSE scale was found to be invariant across gender. Finally, the study showed that the SMSE scale is a psychometrically reliable and valid instrument to measure the sources of mathematics self-efficacy among students. PBL educators could use the results from the SMSE scale in the study to adopt appropriate interventions in curriculum design and delivery to boost self-efficacy of students and hence improve their mathematics achievement.