A sharper definition of alignment for Panoptic Quality

Abstract

The Panoptic Quality metric, developed by Kirillov et al. in 2019, makes object-level precision, recall and F1 measures available for evaluating image segmentation, and more generally any partitioning task, against a gold standard. Panoptic Quality is based on partial isomorphisms between hypothesized and true segmentations. Kirillov et al. desire that functions defining these one-to-one matchings should be simple, interpretable and effectively computable. They show that for $t$ and $h$, true and hypothesized segments, the condition stating that there are more correct than wrongly predicted pixels, formalized as $IoU(t,h)>.5$ or equivalently as $|t\cap h|>.5|t\cup h|$ has these properties. We show that a weaker function, requiring that more than half of the pixels in the hypothesized segment are in the true segment and vice-versa, formalized as $|t\cap h|>.5\left|t\right|$ and $|t\cap h|>.5\left|h\right|$, is not only sufficient but also necessary. With a small proviso, every function defining a partial isomorphism satisfies this condition. We theoretically and empirically compare the two conditions.