The Impact of Forecast Inconsistency and Probabilistic Forecasts on Users’ Trust and Decision-Making

Abstract

When forecasts for a major weather event begin days in advance, updates may be more accurate but inconsistent with the original forecast. Evidence suggests that resulting inconsistency may reduce user trust. However, adding an uncertainty estimate to the forecast may attenuate any loss of trust due to forecast inconsistency as has been shown with forecast inaccuracy. To evaluate this hypothesis, the experiment reported here, tested the impact on trust of adding probabilistic snow accumulation forecasts to single value forecasts in a series of original and revised forecast pairs (based on historical records) that varied in both consistency and accuracy. Participants rated their trust in the forecasts and used them to make school closure decisions. Half of participants received single-value forecasts and half also received the probability of 6 or more inches (decision threshold in the assigned task). As with previous research, forecast inaccuracy was detrimental to trust although probabilistic forecasts attenuated the effect. Moreover, the inclusion of probabilistic forecasts allowed participants to make economically better decisions. Surprisingly, in this study, inconsistency increased, rather than decreased trust, perhaps because it alerted participants to uncertainty and led them to make more cautious decisions. Furthermore, the positive effect of inconsistency on trust was enhanced by the inclusion of probabilistic forecast. This work has important implications for practical settings, suggesting that both probabilistic forecasts and forecast inconsistency provide useful information to decision makers. Therefore, members of the public may well benefit from well-calibrated uncertainty estimates and newer, more reliable information.