Enhancing crowd wisdom using measures of diversity computed from social media data

Abstract

"Wisdom of Crowds" (WoC) refers to a form of collective intelligence in which the aggregate judgment of a group of individuals is, in most instances, superior to that of any one group member. For a crowd to be wise, its members must possess diverse knowledge and viewpoints. Such diversity leads to uncorrelated judgment errors that cancel out in aggregate. Yet despite the fact that diversity is known to be an essential ingredient in WoC, little research aims to measure and exploit diversity in human social systems for the purpose of maximizing crowd intelligence. Here we quantify the diversity of a group of individuals through semantic analysis of their social media (Twitter) communications. Focusing on the domain of fantasy sports, we show that virtual crowds of fantasy team owners selected based on the diversity of their tweet content can outperform both non-diverse and randomly sampled crowds. Our results suggest a new approach for intelligent crowd assembly in which measures of diversity extracted from online social media communications can guide the selection of crowd members. These results have implications for numerous domains that utilize aggregated judgments - from consumer reviews, to econometrics, to geopolitical forecasting and intelligence analysis.