Comparing Default Predictions in the Rating Industry for Different Sets of Obligors

We generalize the refinement ordering for well calibrated probability forecasters to the case were the debtors under consideration are not necessarily identical. This ordering is consistent with many well known skill scores used in practice. We also add an illustration using default predictions made by the leading rating agencies Moody’s and S&P.