Journal of outcome measurement最新文献

Considerable uncertainty exists over the benefit that patients receive from surgical decompressive treatment for cervical spondylotic myelopathy (CSM). Such difficulties might be addressed by accurate quantification of CSM severity as part of a trial determining the outcome of surgery in different patient groups. This study compares the applicability of various existing quantitative severity scales to measurement of CSM severity and the effects on severity of surgical decompression. Scores on the following scales were determined on 100 patients with CSM preoperatively and then again six months following surgical decompression: Odom's Criteria, Nurick grade, Ranawat grade, Myelopathy Disability Index (MDI), Japanese Orthopaedic Association (JOA) Score, European Myelopathy Score (EMS) and Short Form-36 Health Survey (SF36). All the scales showed significant improvement following surgery. However, each had differing qualities of reliability, validity and responsiveness that made them more or less suitable. The MDI showed the greatest sensitivity between different severity levels, sensitivity to operative change and reliability. However, analysis of all the questionnaire scales into components that looked at different aspects of function revealed potential problems with redundancy and a lack of consistency. This prospective observational study provides a rational basis for determining the advantages and disadvantages of different existing scales in measurement of CSM severity and for making adaptations to develop a scale more specifically suited to a comprehensive surgical trial.

This paper describes a comparative analysis of (ADL) and (IADL) items administered to two samples, 4,430 persons representative of older Americans, and 605 persons representative of patients with rheumatoid arthrisit (RA). Responses are scored separately using both Likert and Rasch measurement models. While Likert scoring seems to provide information similar to Rasch, the descriptive statistics are often contrary if not contradictory, and estimates of reliability from Likert are inflated. The test characteristic curves derived from Rasch are similar despite differences between the levels of disability with the two samples. Correlations of Rasch item calibrations across three samples were .71, .76, and .80. The fit between the items and the samples, indicating the compatibility between the test and subjects, is seen much more clearly with Rasch with more than half of the general population measuring the extremes. Since research on disability depends on measures with known properties, the superiority of Rasch over Likert is evident.

Constructed-response or open-ended tasks are increasingly used in recent years. Since these tasks cannot be machine-scored, variability among raters cannot be completely eliminated and their effects, when they are not modeled, can cast doubts on the reliability of the results. Besides rater effects, the estimation of student ability can also be impacted by differentially weighted tasks/items that formulate composite scores. This simulation study compares student ability estimates with their true abilities under different rater scoring designs and differentially weighted composite scores. Results indicate that the spiraled rater scoring design without modeling rater effects works as well as the nested design in which rater tendencies are modeled. As expected, differentially weighted composite scores have a confounding effect on student ability estimates. This is particularly true when open-ended tasks are weighted much more than the multiple-choice items and when rater effects interact with weighted composite scores.

A hierarchical multiple linear regression approach (N = 761) was used to identify pertinent factors which influence health-related quality of life (HRQL) reports among Hispanic and African-American cancer patients. The independent variables include: performance status, disease site, disease stage, mode of administration, socio-economic status (SES), gender, age, living arrangement, race/ethnicity, religious affiliation, insurance status, and spiritual beliefs. The outcome measures, five subscales of HRQL (physical well-being, social well-being, satisfaction with treatment, emotional well-being, functional well-being) and overall HRQL (sum of the five subscales), were estimated using the Functional Assessment of Cancer Therapy (FACT) Scales. This study identified performance status and spiritual beliefs as consistent predictors of overall HRQL. This study also found no significant effects of SES, mode of administration, gender age, living arrangement and insurance status on the reporting of overall HRQL. Spiritual beliefs and performance status are important determinants of HRQL across a diverse group of cancer patients.

When latent trait models are used to measure change across time, it is difficult to disentangle changes in one facet of the measurement context from changes in other facets. Hence, it is difficult to diagnose change. Wright (1999b) proposed an algorithm for disentangling change, and previously the authors applied this algorithm to measuring change across two occasions (Wolfe and Chiu, 1999). In this article we extend Wright's algorithm to disentangle changes in measures across three occasions. We describe a standard Rasch rating scale analysis of a multi-occasion evaluation that produces confusing results when subjected to a series of "separate" calibrations. Then, we apply Wright's correction to the same data to show that the algorithm reveals changes that are more similar to ones that would be expected. Our demonstration shows that Wright's procedure can reduce misfit to the Rasch Rating Scale Model as well as changing the interpretation of change within the measurement context.

The purpose of this study is to examine the alternate forms reliability of the AMPS (Assessment of Motor and Process Skills) (Fisher, 1997a) where alternate forms means different pairs of AMPS tasks. The participants for this study were persons selected from the AMPS database who had performed four AMPS tasks. The participants varied in age, gender, diagnosis, and level of assistance needed to live in the community. The AMPS was administered by trained and calibrated occupational therapists according to standardized procedures. The data for the 91 participants were subjected to 12 many-faceted Rasch analyses to generate ADL motor and ADL process ability measures for each task and each set of paired tasks. Repeated measures ANOVAs revealed no time effect across the four AMPS tasks. Pearson product moment correlations between Tasks 1 and 2 combined and Tasks 3 and 4 combined were r = .91 and r = .86 for the ADL motor and ADL process scales, respectively. Calculation of the standardized difference (z) revealed that no more than 8% of the participants had ADL motor or ADL process ability measures that differed significantly between observations once we accounted for real differences in a persons performance; 80% of the paired ADL motor and ADL process ability measures remained stable within +/- 0.5 logits when the participants performed two tasks. The AMPS ADL motor and ADL process scales can be used reliably in clinical practice and for research purposes.

This study examined five Rasch-model-based item-fit indices: unweighted and weighted standardized indices (denoted UWz and Wz), standardized likelihood index (denoted Lz), and Extended Caution Indices (denoted ECI2z and ECI4z), in terms of their distributional properties and the power of detecting item bias or Differential Item Functioning (DIF). The results indicated that although these five standardized item-fit indices did not depart significantly from a normal distribution, it appeared that the Type I error rates were not reasonable. For the power of five standardized item-fit indices to detect DIF, the results showed that all indices did perform poorly across various conditions. These findings lead to the conclusion that all indices used in this study are inadequate fit measures for detecting DIF.

Jackson and Marsh (1996) reported the development of a Flow State Scale (FSS) for use in sport and physical activity. The FSS contains 36 items measuring the nine dimensions of flow described by Csikszentmihalyi (1990, 1993). Jackson and Marsh reported high internal consistency estimates for the subscales and evidence for nine first-order factors and one second-order factor when confirmatory factor analytic techniques were used. The present study extended this validation work by subjecting the data from the original sample (N = 394) of elite younger athletes and a subsequent sample (N = 398) of older athletes to Rasch analysis. These Rasch analyses showed quite clearly that the flow dimensions may be conceptualised as a continuum with "autotelic experience" being experienced more readily than dimensions such as "transformation of time", a state that may only be encountered at the height of a deep flow experience. The Rasch analyses provide useful additional information about the areas of the flow continuum tapped by the items and scales of the FSS and, in so doing, help to confirm the construct validity and generalisability of the scale itself.

One of the benefits of item response theory (IRT) applications in scale development is the greater transparency of resulting scores. This feature allows translation of a total score on a particular scale into a profile of probable item responses with relative ease, for example by using the variable map that often is part of IRT analysis output. Although there have been a few examples in the literature using variable maps to enrich clinical interpretation of individual functional assessment scores, this feature of IRT output has received very limited application in rehabilitation research. The present paper illustrates the application of variable maps to support more in-depth interpretation of functional assessment scores in research and clinical contexts. Two examples are presented from an outcome prediction study conducted during the standardization of a new functional assessment for elementary school students with disabilities, the School Function Assessment. Two different applications are described: creating a dichotomous outcome variable using scores from a continuous scale, and interpreting the meaning of a classification cut-off score identified through Classification and Regression Tree (CART) analysis.

When measures are taken on the same individual over time, it is difficult to determine whether observed differences are the result of changes in the person or changes in other facets of the measurement situation (e.g., interpretation of items or use of rating scale). This paper describes a method for disentangling changes in persons from changes in the interpretation of Likert-type questionnaire items and the use of rating scales (Wright, 1996a). The procedure relies on anchoring strategies to create a common frame of reference for interpreting measures that are taken at different times and provides a detailed illustration of how to implement these procedures using FACETS.