Selective conformal inference with false coverage-statement rate control

Abstract

Conformal inference is a popular tool for constructing prediction intervals. We consider here the scenario of post-selection/selective conformal inference, that is prediction intervals are reported only for individuals selected from unlabelled test data. To account for multiplicity, we develop a general split conformal framework to construct selective prediction intervals with the false coverage-statement rate control. We first investigate benjamini2005false's false coverage rate-adjusted method in the present setting, and show that it is able to achieve false coverage-statement rate control but yields uniformly inflated prediction intervals. We then propose a novel solution to the problem called selective conditional conformal prediction. Our method performs selection procedures on both the calibration set and test set, and then constructs conformal prediction intervals for the selected test candidates with the aid of conditional empirical distribution obtained by the post-selection calibration set. When the selection rule is exchangeable, we show that our proposed method can exactly control the false coverage-statement rate in a model-free and distribution-free guarantee. For non-exchangeable selection procedures involving the calibration set, we provide non-asymptotic bounds for the false coverage-statement rate under mild distributional assumptions. Numerical results confirm the effectiveness and robustness of our method in false coverage-statement rate control and show that it achieves more narrowed prediction intervals over existing methods across various settings.