Fair Ranking Metrics

Information access systems such as search engines and recommender systems often display results in a sorted ranked list based on their relevance. Fairness of these ranked list has received attention as an important evaluation criteria along with traditional metrics such as utility or accuracy. Fairness broadly involves both provider and consumer side fairness at both group and individual levels. Several fair ranking metrics have been proposed to measure group fairness for providers based on various “sensitive attributes”. These metrics differ in their fairness goal, assumptions, and implementations. Although there are several fair ranking metrics to measure group fairness, multiple open challenges still exist in this area to consider. In my thesis, I work on the area of fair ranking metrics for provider-side group fairness. I am interested in understanding the fairness concepts and practical applications of these metrics to identify their strength and limitations to aid the researchers and practitioner by pointing out the gaps. Moreover, I will contribute to this research area by focusing on some of the limitations like considering different browsing models and bias in relevance information.