How Difference Tasks Are Affected by Probability Format, Part 1: A Making Numbers Meaningful Systematic Review.

Background. To develop guidance on the effect of data presentation format on communication of health probabilities, the Making Numbers Meaningful project undertook a systematic review. Purpose. This article, one in a series, covers evidence about "difference tasks," in which a reader examines a stimulus to evaluate differences between probabilities, such as the effect of a risk factor or therapy on the chance of a disease. This article covers the effect of format on 4 outcomes: 1) identifying a probability difference (identification) or recalling it (recall), 2) identifying the largest or smallest of a set of probability differences (contrast outcome), 3) placing a probability difference into a category such as "elevated" or "below average" (categorization outcome), and 4) performing computations (computation outcome). Data Sources. MEDLINE, Embase, CINAHL, the Cochrane Library, PsycINFO, ERIC, ACM Digital Library; hand search of 4 journals. Finding Selection. Pairwise screening to identify experimental/quasi-experimental research comparing 2 or more formats for quantitative health information. This article reports on 53 findings derived from 35 unique studies reported in 32 papers. Data Extraction. Pairwise extraction of information on stimulus (data in a data presentation format), cognitive task, and perceptual, affective, cognitive, or behavioral outcomes. Data Synthesis. Most evidence involving outcomes of difference-level cognitive tasks was weak or insufficient. Evidence was strong that 1) computations involving differences are easier with rates per 10ⁿ than with percentages or 1 in X rates and 2) adding graphics to numbers makes it easier to perform difference-level computations. Limitations. A granular level of evidence syntheses leads to narrow guidance rather than broad statements. Conclusions. Although many studies examined differences between probabilities, few were comparable enough to generate strong evidence.

Highlights: Most evidence about the effect of format on ability to evaluate differences in probabilities was weak or insufficient because of too few comparable studies.Strong evidence showed that computations relevant to differences in probabilities are easier with rates per 10ⁿ than with 1 in X rates.Adding graphics to probabilities helps readers compute differences between probabilities.