Journal of outcome measurement最新文献

In order to apply the Rasch model to multiple-choice items, incorrect responses to distractors are usually aggregated to a single category. In doing so, information of individual distractors disappears. In this paper, a Rasch-type analysis is proposed where one parameter is assigned to each distractor. The information is thus preserved. The proposed distractor model can be applied to investigate the performance of distractors, which is useful for item revision. This model is a necessary condition of the Rasch model, that is, fitting the distractor model will fit the Rasch model, but not vice versa. The results of a small simulation study show that parameter recovery of the distractor model is very satisfactory. A real data set of twenty multiple-choice items was analyzed. Some items were found to fit the Rasch model rather than the distractor model. It is this diagnostic value that makes the distractor model suitable for multiple-choice items.

Most calibration programs designed for the family of Rasch psychometric models report the asymptotic standard errors for person and item measure estimates resulting from the calibration process. Although these estimates are theoretically correct, they may be influenced by any number of factors, e.g., restrictions due to the loss of degrees of freedom in estimation, targeting of the instrument, i.e., the degree of offset between mean item difficulty and mean person ability, and the presence of misfit in the data. The effect of these factors on the standard errors reported for the person has not been previously reported. The purpose of this study was to investigate the effects of these three factors on the asymptotic standard errors for person measures using simulated data. The results indicate that asymptotic errors systematically underestimate the observed standard deviation of ability in simulated data, though this underestimation is usually small for targeted instruments with reasonable sample size. However, the underestimation can easily be corrected with a simple linear function. These simulations use only dichotomous data and the results may not generalize to the rating scale and partial credit models.

The psychometric properties of the OARS ADL scale, comprised of seven physical activities of daily living (PADL) and seven instrumental activities of daily living (IADL) items, were examined using a Rasch measurement approach. Two of the PADL items failed to demonstrate acceptable goodness-of-fit with the measurement model but the remaining 12 items could be combined into a single measure of ADL ability. Although the OARS ADL scale was designed to identify those community-dwelling elderly who need supports and services to continue to live in the community, the scale items were found to be poorly targeted to community-dwelling elderly since almost half of our sample received maximal scores. Rasch analysis identified how we might improve the sensitivity of the OARS ADL scale but its utility in outcome and longitudinal studies remains questionable.

Throughout the mid to late 1970's considerable research was conducted on the properties of Rasch fit mean squares. This work culminated in a variety of transformations to convert the mean squares into approximate t-statistics. This work was primarily motivated by the influence sample size has on the magnitude of the mean squares and the desire to have a single critical value that can generally be applied to most cases. In the late 1980's and the early 1990's the trend seems to have reversed, with numerous researchers using the untransformed fit mean squares as a means of testing fit to the Rasch measurement models. The principal motivation is cited as the influence sample size has on the sensitivity of the t-converted mean squares. The purpose of this paper is to present the historical development of these fit indices and the various transformations and to examine the impact of sample size on both the fit mean squares and the t-transformations of those mean squares. Because the sample size problem has little influence on the person mean square problem, due to the relatively short length (100 items or less), this paper focuses on the item fit mean squares, where it is common to find the statistics used with sample sizes ranging from 30 to 10,000.

Factor analysis is a powerful technique for investigating multidimensionality in observational data, but it fails to construct interval measures. Rasch analysis constructs interval measures, but only indirectly flags the presence of multidimensional structures. Simulation studies indicate that, for responses to complete tests, construction of Rasch measures from the observational data, followed by principal components factor analysis of Rasch residuals, provides an effective means of identifying multidimensionality. The most diagnostically useful residual form was found to be the standardized residual. The multidimensional structure of the Functional Independence Measure (FIMSM) is confirmed by means of Rasch analysis followed by factor analysis of standardized residuals.

Item response models have been developed to explore change measurement, including those proposed by Fischer and his colleagues (e.g., Fischer & Pazer, 1991; Fischer & Ponocny, 1994), Andersen (1985) and Embretson (1991). In this article, we propose another multidimensional Rasch model, the multidimensional random coefficient multinomial logit (MRCML) model (Adams, Wilson, & Wang, 1997). All these models are briefly reviewed and compared. The MRCML can be applied to not only polytomous items but also investigation of variations in item difficulties. Based on variations in difficulties across occasions and items, five kinds of models are proposed. Some simulation studies were conducted to examine parameter recovery of the MRCML model under various testing situations. All the parameters were recovered very well. A real data set was analyzed to show applications of the MRCML to measuring individual differences in change.

Objectives: This study examines the stability of a physical disability construct across instruments and samples. The purpose is not to report a formal equating of instrument calibrations, but to indicate whether such an effort would be likely to succeed. Theory. The economics transforming health care from its orientation toward crisis-driven disease reactions to population- and evidence-based preventive health management and individualized disease management demand general scale-free measures of functional independence.

Methods: A new method, pseudo-common item equating, is demonstrated. Similar, but not identical items, from different instruments, calibrated on different samples, are compared.

Data: More than 30 articles presenting Rasch analyses of physical functioning scales were reviewed. Four instruments provided data from ten of these articles, for eleven different calibrations (two instruments are both included in one article).

Results: The final overall average correlation disattenuated for error is .93, with an average of 7 pseudo-common items, and an average p-value of .01, meaning that measures based on these calibrations should be linearly transformable versions of the same metric. Scientific importance. The quantitative stability of different areas of physical functional independence across instruments and samples suggests that the development and deployment of a universal metric is a realizable goal.

The purpose of this paper is to report on the development of the on-line performance assessment instrument--the Assessment of Motor and Process Skills (AMPS). Issues that will be addressed in the paper include: (a) the establishment of the scoring rubric and its implementation in an extended Rasch model, (b) training of raters, (c) validation of the scoring rubric and procedures for monitoring the internal consistency of raters, and (d) technological implementation of the assessment instrument in a computerized program.

The different chi-square statistics reported in the many-faceted Rasch model analysis are presented and interpreted. In addition, other chi-square summary values are computed and presented for interpretation of facets. The chi-square values are useful for determining: (1) the significance of a facet in the Rasch model; (2) the significant contribution of facet main and interaction effects; (3) differences among facet elements; and (4) identifying the specific facet interaction adjustments to the subjects' calibrated logit ability measure.

The purpose of this study was to determine the optimal categorization of a self-efficacy ordered-response scale using the Rasch analysis and compare the performance of the Rasch statistics and parameter estimates with conventional statistics. A 50-item scale to measure psychomotor self-efficacy was administered to a total of 2,022 children, including 1,009 boys and 1,013 girls. The data analysis started by collapsing the original five adjacent categories into two, three, and four categories, and a total of 14 data sets were derived. Each of these data sets, including the original one, was analyzed using the Rasch rating scale model, and a set of Rasch model-data fit, category, and separation statistics and parameter estimates, as well as three conventional statistics, were computed and compared. It was found that, instead of the five-category construct designed, the best order of category meanings of the scale in respondents' perceptions was a three-category construct. The Rasch threshold estimates were sensitive indexes in determining the order of the categorization, and that item separation statistics were useful in determining the optimal categorization after its order was confirmed. The commonly used coefficient alpha was found not helpful at all in determining the optimal categorization. The Rasch analysis was demonstrated to be a useful post-hoc analytic approach in determining the optimal categorization of an ordered-response scale.