Development and preliminary validation of the evaluation use scale for evaluation systems (EUS-ES)

Abstract

ABSTRACTThe usefulness of evaluation is critical. Evaluation use occurs when, from its results or process, decisions are made about the program, it changes people's mindsets, or persuasive or legitimation actions happen (instrumental, conceptual, and symbolic uses respectively). Few quantitative evaluation use studies have been conducted in recent years. Most existing evaluation use scales are outdated due to theoretical advances and were project or program specific, not designed for evaluation systems. This study was designed to develop the Evaluation Use Scale for Evaluation Systems (EUS-ES) and gather initial validity evidence. We collected and analyzed data from Brazilian professors and program coordinators who were consultants of the Brazilian Graduate Evaluation System. Validity evidence was collected based on instrument content and internal structure. CFA results supported a three-factor-correlated model with adequate score reliability estimates. The EUS-ES is valuable for diagnostic purposes to identify low/high evaluation uses in an evaluation system setting.KEYWORDS: Evaluation useinstrumental useconceptual usesymbolic usescale Disclosure statementNo potential conflict of interest was reported by the author(s).Additional informationFundingThis work was supported in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil (CAPES) under Grant 88887.373319/2019-00.Notes on contributorsRoberto Brazileio PaixãoRoberto Brazileiro Paixão is an associate professor of educational evaluation.Michael C. RodriguezMichael C. Rodriguez is a professor of quantitative methods in education and dean of the College of Education and Human Development.