Journal of applied measurement最新文献

Confidence marking is increasingly used in multiple choice testing situations, but when the Rasch measurement model is applied to the data, only the binary data is used, discarding the information given by the confidence marking. This study shows how Wilson's ordered partition model (OPM), a member of the Rasch family of models, can be used to model the confidence information. The result is a model which is in strict relation to the binary Rasch model, since the Rasch ICC's are "split" into a set of curves each representing a confidence level. The new model provides a set of item parameters that map the probability of being in each confidence level in relation to the test-taker's ability. The study provides a powerful diagnostic tool to assess item difficulty, overconfidence or misuse of confidence levels but also the fact that a question is particularly tricky or creates a lot of doubt.

This study describes an approach for examining model-data fit for the dichotomous Rasch model using Tukey-Hann item response functions (TH-IRFs). The procedure proposed in this paper is based on an iterative version of a smoothing technique proposed by Tukey (1977) for estimating nonparametric item response functions (IRFs). A root integrated squared error (RISE) statistic (Douglas and Cohen, 2001) is used to compare the TH-IRFs to the Rasch IRFs. Data from undergraduate students at a large university are used to demonstrate this iterative smoothing technique. The RISE statistic is used for comparing the item response functions to assess model-data fit. A comparison between the residual based Infit and Outfit statistics and RISE statistics are also examined. The results suggest that the RISE statistic and TH-IRFs provide a useful analytical and graphical approach for evaluating item fit. Implications for research, theory and practice related to model-data fit are discussed.

New mathematics standards were adopted broadly across the United States of America between 2011-2013. Problem solving is a central facet of these new standards. Given new standards and the prominence of mathematical problem solving, there is a need for valid and reliable assessments that measure students' abilities related to those standards. Moreover, Rasch measurement techniques support psychometric analyses during validation studies, effectively measuring students' and items' properties in ways not afforded by true score theory. This manuscript builds upon past research (see Bostic and Sondergeld, 2015a, 2015b) with a validity study of two related problem-solving measures for grades seven and eight. Results from this validation study indicated that the problem-solving measures for grades seven and eight had sufficient evidence for their use.

Resilience is the personality trait of having positive dispositions which enable individuals to cope with stressful situations. Hence, a reliable resilience scale can provide useful information on understanding and treating individuals suffering from stress and trauma. The 10-item Connor-Davidson Resiliance Scale (CD-RISC-10) is a candidate scale. However, very little psychometric research has been conducted on this scale and, moreover, psychometric analyses to date have not been conclusive. To attain further evidence of the scale's psychometric properties, we tested the CD-RISC-10 on 288 adult Education major students at an Australian University using both traditional (factor analyses) and modern (Rasch) measurement approaches. Factor analyses indicated good psychometric functioning of the scale. However, Rasch modelling revealed evidence of item misfit and multiple dimensions. Optimal performance was achieved after the removal of two misfitting items indicating a well-functioning 8-item scale.

Scale anchoring is a method to provide additional meaning to particular scores at different points along a score scale by identifying representative items associated with the particular scores. These items are then analyzed to write statements of what types of performance can be expected of a person with the particular scores to help test takers and other stakeholders better understand what it means to achieve the different scores. This article provides simple formulas that can be used to identify possible items to serve as scale anchors with the Rasch model. Specific attention is given to practical considerations and challenges that may be encountered when applying the formulas in different contexts. An illustrative example using data from a medical imaging certification program demonstrates how the formulas can be applied in practice.

The purpose of this research was to estimate the reliability of the scores produced from and validity of the inferences drawn from the revised 90-item Teachers' Emotion Questionnaire consisting of three measures: frequency of emotional expressivity, self-efficacy for regulation of emotional expressivity when teaching, and self-efficacy for regulation of context-specific emotional expressivity. A void exists in an instrument assessing teachers' regulation and communication of their emotions. One-hundred seventeen practicing teachers participated in this study at Time 1 and 46 at Time 2. Rasch rating scale analyses indicated sufficient item and person separation and reliability and some support for the construct validity of the inferences drawn from the measures. Test re-test reliability for the person estimates was supported for all three measures over a four-week period: r=.592, p<.001, r=.473, p<.01, and r=.641, p<.001, respectively. Concurrent validity for the self-efficacy for regulation of emotional expressivity when teaching measure with the re-appraisal and suppression sub-scales on the Emotional Regulation Questionnaire (Gross and John, 2003) was supported at Time 1. Modifications to rating scales and future directions for assessing teachers' emotions based on these results are discussed.

Students may experience considerable fear and stress in school settings, and based on Dweck's (2006) notion of "mindset" we hypothesized that fear introduces qualitative changes in students' self-concepts. Hypotheses were tested on 3847 third-grade students from nine Iberoamerican countries (Bolivia, Chile, Colombia, Cuba, Ecuador, Panama, Peru, Spain, and Venezuela), who completed Murillo's (2007) adaptation of Marsh' (1988) SDQ-I. Rasch scaling indicated that the information-content of High-Fear students' ratings was more localized across the latent dimension than was that of Low-Fear students, and their ratings also showed less cognitive variety. The resulting measurement distortions could be captured via logistic regression over the ratings' residuals. Also, using training and validation samples (with respectively 60 and 40% of all cases), it proved possible to predict students' fear levels and their gender. We see the present findings as a first step towards implementing an online warning and detection system for signs of bullying among students.

Residual-based fit statistics are commonly used as an indication of the extent to which the item response data fit the Rash model. Fit statistic estimates are influenced by sample size and rules-of thumb estimates may result in incorrect conclusions about the extent to which the model fits the data. Estimates obtained in this analysis were compared to 250 simulated data sets to examine the stability of the estimates. All INFIT estimates were within the rule-of-thumb range of 0.7 to 1.3. However, only 82% of the INFIT estimates fell within the 2.5th and 97.5th percentile of the simulated item's INFIT distributions using this 95% confidence-like interval. This is a 18 percentage point difference in items that were classified as acceptable. Fourty-eight percent of OUTFIT estimates fell within the 0.7 to 1.3 rule- of-thumb range. Whereas 34% of OUTFIT estimates fell within the 2.5th and 97.5th percentile of the simulated item's OUTFIT distributions. This is a 13 percentage point difference in items that were classified as acceptable. When using the rule-of- thumb ranges for fit estimates the magnitude of misfit was smaller than with the 95% confidence interval of the simulated distribution. The findings indicate that the use of confidence intervals as critical values for fit statistics leads to different model data fit conclusions than traditional rule of thumb critical values.

This study explored optimization of item-attribute matrices with the linear logistic test model (Fischer, 1973), with optimal models explaining more variance in item difficulty due to identified item attributes. Data were 8th-grade mathematics test item responses of two TIMSS 2007 booklets. The study investigated three categories of attributes (content, cognitive process, and comprehensive cognitive process) at two grain levels (larger, smaller) and also compared results with random attribute matrices. The proposed attributes accounted for most of the variance in item difficulty for two assessment booklets (81% and 65%). The variance explained by the content attributes was very small (13% to 31%), less than variance explained by the comprehensive cognitive process attributes which explained much more variance than the content and cognitive process attributes. The variances explained by the grain level were similar to each other. However, the attributes did not predict the item difficulties of two assessment booklets equally.

The authors investigated the effect of missing completely at random (MCAR) item responses on partial credit model (PCM) parameter estimates in a longitudinal study of Positive Affect. Participants were 307 adults from the older cohort of the Notre Dame Study of Health and Well-Being (Bergeman and Deboeck, 2014) who completed questionnaires including Positive Affect items for 56 days. Additional missing responses were introduced to the data, randomly replacing 20%, 50%, and 70% of the responses on each item and each day with missing values, in addition to the existing missing data. Results indicated that item locations and person trait level measures diverged from the original estimates as the level of degradation from induced missing data increased. In addition, standard errors of these estimates increased with the level of degradation. Thus, MCAR data does damage the quality and precision of PCM estimates.