Sensitivity and Robustness Analyses in Social Multi-Criteria Evaluation of Public Policies

Abstract

In policy arenas, the major virtue of Multiple Criteria Decision Analysis (MCDA) is the possibility of dealing with a plurality of multidimensional features both at technical and social levels. However, in this process there is always the danger of oversimplifying complex issues by creating false certainties. MCDA outputs may seem a precise result, while they are not, frequently. In this article, we introduce various improvements of the state of the art, in particular with reference to Social Multi-Criteria Evaluation (SMCE), which has been explicitly developed for public policies. From the theoretical point of view, local and global sensitivity analyses are considered as complementary, while habitually they are considered as separate analyses; this is particularly relevant for criterion weights, which are one of the most sensitive input parameters in real-world applications. Algorithmically, our approach allows to perform exhaustive sensitivity and robustness analyses in the context of the Kemeny median ranking aggregation rule by solving its computational time issue. From an empirical point of view, we propose an approach, based on frequency matrices, to make output uncertainty transparent and easy to communicate; this helps improving the policy learning process, too. Finally, we present an illustrative example, where we summarise the whole approach and put emphasis on the role of sensitivity analysis as a tool for better understanding the decision model and explore its informative content.