Person Search by Uncertain Attributes

Abstract

This paper presents a person search system by uncertain attributes. Attribute-based person search aims at finding person images that are the best matched with a set of attributes specified by a user as a query. The specified query attributes are inherently uncertain due to many factors such as the difficulty of retrieving characteristics of a target person from brain-memory and environmental variations like light and viewpoint. Also, existing attribute recognition techniques typically extract confidence scores along with attributes. Most of state-of-art approaches for attribute-based person search ignore the confidence scores or simply use a threshold to filter out attributes with low confidence scores. Moreover, they do not consider the uncertainty of query attributes. In this work, we resolve this uncertainty by enabling users to specify a level of confidence with each query attribute and consider uncertainty in both query attributes and attributes extracted from person images. We define a novel matching score to measure the degree of a person matching with query attribute conditions by leveraging the knowledge of probabilistic databases. Furthermore, we propose a novel definition of Critical Point of Confidence and compute it for each query attribute to show the impact of confidence levels on rankings of results. We develop a web-based demonstration system and show its effectiveness using real-world surveillance videos.