Development and Validity of the Rating Scales of Academic Skills for Reading Comprehension

Abstract

The development and psychometric qualities of a measure using teacher judgment to rate performance in reading comprehension for narrative text is described—the Rating Scales for Academic Skills—Reading Comprehension Narrative (RSAS-RCN). Sixty-five teachers from the third, fourth, and fifth grades of 8 elementary schools completed the measure on 177 students. Each teacher rated students who had been identified through school-based universal screening to be below the 25th percentile, between the 25th and 74th percentile, and at or above the 75th percentile on national normative standards. Results indicated the RSAS-RCN has strong to moderate evidence of (a) 1-week test–retest reliability, (b) concurrent validity with the Group Reading Assessment and Diagnostic Evaluation (GRADE) and end of year state assessment in reading, and (c) significant classification accuracy across student ability levels. Principal component analysis and item response theory (Rasch modeling) indicate the RSAS-RCN is comprised of a single general dimension. Overall, this examination of the RSAS-RCN suggests teacher judgment may be a potentially valuable tool in assessing reading comprehension among upper elementary school students.