Reducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case Study

Abstract

Many existing fairness metrics measure group-wise demographic disparities in system behavior or model performance. Calculating these metrics requires access to demographic information, which, in industrial settings, is often unavailable. By contrast, economic inequality metrics, such as the Gini coefficient, require no demographic data to measure. However, reductions in economic inequality do not necessarily correspond to reductions in demographic disparities. In this paper, we empirically explore the relationship between demographic-free inequality metrics -- such as the Gini coefficient -- and standard demographic bias metrics that measure group-wise model performance disparities specifically in the case of engagement inequality on Twitter. We analyze tweets from 174K users over the duration of 2021 and find that demographic-free impression inequality metrics are positively correlated with gender, race, and age disparities in the average case, and weakly (but still positively) correlated with demographic bias in the worst case. We therefore recommend inequality metrics as a potentially useful proxy measure of average group-wise disparities, especially in cases where such disparities cannot be measured directly. Based on these results, we believe they can be used as part of broader efforts to improve fairness between demographic groups in scenarios like content recommendation on social media.