Journal of outcome measurement最新文献

The Daily Activities Questionnaire (DAQ) was developed to assess activities of daily living (ADL) independence in people with Alzheimer's disease. After administering it to 276 people diagnosed with Alzheimer's disease, we examined the quality of the rating scale and its structure using a Rasch measurement approach. Results indicated that the original 10-point rating scale should be restructured to a 5-point rating scale to improve the quality of the instrument. In addition, we found that all but two ADL items defined the same construct and could be combined into a single summary measure of ADL independence. The remaining items were positioned along a hierarchical continuum, with IADL tasks more difficult than PADL tasks. Furthermore, the tasks were logically ordered by difficulty. We therefore report that the DAQ is a valid scale and conclude that it is a viable measure of ADL independence for studies of Alzheimer's disease.

An alternative to dichotomous scoring of multiple items anchored to a common stem is scoring these items as a single polytomous item (testlet scoring). This study systematically compared the partial credit model (PCM), the generalized partial credit model (GPCM), and the graded response model (GRM) in the context of testlet scoring. Data sets included a sample from the fall 1994 administration of the SAT I (N = 2,548) and a simulated data set. Theta estimation, information, and model fit were analyzed. Correlations among theta estimates ranged from 0.9748 to 0.9921. The relationship among the information functions of the PCM, GPCM and the GRM reflected the discrimination parameter estimates for the latter two models. Suggestions are made with regard to model selection.

This paper reports the use of the Extended Logistic Model (ELM) of Rasch (Andrich, 1988), based on Item Response Theory, to validate the reduction of continuous motor skill data to categories of performance. The data were gathered from the performances of 5 and 6 year old children on 24 fundamental movement skills and involved different measurement units such as seconds, centimetres, scores and counts. In order to compare results across all skills the data were collapsed into discrete sets of categories. Several alternative cut-off locations based on normative data were considered. A feature of the ELM is that it can account for correct scoring of the response categories, but only if the threshold estimates derived from the data by the measurement model are correctly ordered in a hierarchical fashion, from lowest to highest. Should this be the case, a valid scoring function has been established. In this study, the data were successfully reduced to three categories based on the 15th and 85th percentile allowing further analysis to proceed.

Rasch parameter estimation methods can be classified as non-interative and iterative. Non-iterative methods include the normal approximation algorithm (PROX) for complete dichotomous data. Iterative methods fall into 3 types. Datum-by-datum methods include Gaussian least-squares, minimum chi-square, and the pairwise (PAIR) method. Marginal methods without distributional assumptions include conditional maximum-likelihood estimation (CMLE), joint maximum-likelihood estimation (JMLE) and log-linear approaches. Marginal methods with distributional assumptions include marginal maximum-likelihood estimation (MMLE) and the normal approximation algorithm (PROX) for missing data. Estimates from all methods are characterized by standard errors and quality-control fit statistics. Standard errors can be local (defined relative to the measure of a particular item) or general (defined relative to the abstract origin of the scale). They can also be ideal (as though the data fit the model) or inflated by the misfit to the model present in the data. Five computer programs, implementing different estimation methods, produce statistically equivalent estimates. Nevertheless, comparing estimates from different programs requires care.

Adding the items of the Functional Assessment Measure (FAM) to the Functional Independence Measure (FIM instrument) has been proposed as a method to extend the range of the FIM, particularly when assessing functional status in rehabilitation patients with brain injury, including stroke. It has been proposed that this approach is especially helpful in ameliorating ceiling effects when brain-injured patients have reached the end of their inpatient rehabilitation stay or are being seen in outpatient settings. In the present study, 376 consecutive stroke patients on a Canadian inpatient rehabilitation unit were concurrently administered the FIM and the FAM. Rasch analysis was used to evaluate how well the FAM items extended the difficulty range of the FIM for both the Motor and Cognitive domains. Within the Motor domain, only the FAM item assessing Community Access was found to be more difficult than extant FIM items, and this item showed some tendency to misfit with the other motor items. In the Cognitive domain, the only FAM item with a higher difficulty level than the FIM items was that assessing Employability. Notably, strict adherence to scoring guidelines for these two FAM items requires taking patients out into the community to evaluate their actual performances, a practice unlikely in the typical inpatient stroke rehabilitation unit. Results indicate that use of the entire FAM as an adjunct to the FIM reduces test efficiency while providing only minimal additional protection against ceiling effects.

This report describes an implementation of the Rasch model during the longitudinal evaluation of a federally-funded early childhood preschool intervention program. An item bank is described for operationally defining a psychosocial construct called community life-skills competency, an expected teenage outcome of the preschool intervention. This analysis examined the position of teenage students on this scale structure, and investigated a pattern of cognitive operations necessary for students to pass community life-skills test items. Then this scale structure was correlated with nationally standardized reading and math achievement scores, teacher ratings, and school records to assess its validity as a measure of the community-related outcome goal for this intervention. The results show a functional relationship between years of early intervention and magnitude of effect on the life-skills competency variable.

Previous measures of Atopic Dermatitis (AD) have not been adequate for research purposes. This paper describes a study conducted in dermatology clinics of the Royal Children's Hospital, Melbourne, Australia, to develop a reliable, valid and practical measure. A pool of items to describe both site and morphology of AD was generated from a literature survey and expert opinion. Selected items were incorporated into a measure with each item rated on a four point scale. The measure was piloted and revised to a simpler format and called the Atopic Dermatitis Assessment Measure (ADAM). Unidimensionality was established. Reliability was determined by comparing two doctors blind ratings on 51 patients (mean age = 70 months). Agreement varied depending upon site and morphology with more agreement on "mild" AD than on "severe" AD. These results imply that operational definitions of the scales need to be defined more clearly. The measure satisfies the assumptions for a partial credit analysis.

Eight guidelines are suggested to aid the analyst in investigating whether rating scales categories are cooperating to produce observations on which valid measurement can be based. These guidelines are presented within the context of Rasch analysis. They address features of rating-scale-based data such as category frequency, ordering, rating-to-measure inferential coherence, and the quality of the scale from measurement and statistical perspectives. The manner in which the guidelines prompt recategorization or reconceptualization of the rating scale is indicated. Utilization of the guidelines is illustrated through their application to two published data sets.

The purpose of this study was to refine an instrument designed to measure a single construct, the effectiveness of school-based partnerships. The instrument was designed to measure the "health" of the partnership teams and to identify specific problems for which intervention might be appropriate. The items were based on four theoretical models of partnering efforts. The partnerships studied were created to enhance the teaching of elementary school science and involved elementary teachers and resource professionals in school-based programs over a six-year period. The results show how Rasch analysis, using the item and person fit statistics, bias analysis using separate calibration groups for contrasts of interest, and principal component analysis can be used to evaluate the unidimensionality of a scale.

This purpose of this paper is to describe the Rasch calibration of a portion of the Watkins-Farnum Performance Scale (WFPS), using a sample of 218 sixth graders from a middle school in Rhode Island. The WFPS is a test of instrumental music performance and consists of fourteen exercises of increasing difficulty, of which students play as many as possible until they fail two consecutive exercises. The WFPS has demonstrated reliability and validity. However, classical test theory did not allow its authors to calculate a measure of difficulty for each bar (because some students did not play all bars), or to allow judges the flexibility to shorten the scale (because of low reliability). Using Rasch scaling, item difficulties can be estimated, the test can be administered more efficiently, and perhaps most importantly, diagnostic information can be easily obtained.