How Do You Define Success? Evaluative Criteria for Informal STEM Education

Abstract

Abstract Evaluative criteria define a “successful” intervention and direct what evaluation questions to ask, data to collect, and conclusions to draw. This study analyzes the criteria that underpin a sample of summative evaluation reports from InformalScience.org to understand how success was defined and investigated. Summative evaluation is often framed as an examination of effectiveness in achieving desired outcomes, and every report in the sample did investigate this criteria domain. Further, nearly all studies addressed multiple outcome categories, reflecting a mix of cognitive, affective, and behavioral outcomes. In addition, rather than focusing exclusively on the intervention’s effectiveness, most reports addressed criteria from other domains. Particular attention was directed to intervention design and implementation and visitor experience, and some reports examined equity and relevance. Findings reveal complex, multidimensional definitions of success associated with informal STEM education and raise implications for evaluation capacity building related to design, methods, and reporting.