Enriching Image Archives via Facial Recognition

Abstract

The digitization of image archives across the globe has opened up vast collections of libraries, museums, and cultural heritage institutions. These collections provide valuable historical information to the public and researchers. Many image collections have little metadata describing who or what is depicted in a structured format, making it difficult to search for specific persons. This work presents a facial recognition pipeline to enrich these collections by recognizing the persons in each image. A reference dataset of over 6000 known persons was constructed and facial recognition was performed on a dataset of over 150 thousand images. Detected faces were matched with the known faces using a similarity score on the face embeddings. We developed an interactive labeling tool to efficiently validate the face recognition predictions. A total of 182 thousand detected faces were labeled with this tool. Using a minimum similarity score of 0.5, the face recognition model achieved a precision of 0.936 and identified over 62 thousand persons from the image archives. We show how clustering can be used to identify new persons that were not included in the reference dataset. Furthermore, we highlight the potential of facial recognition to enhance the accessibility of the collections and offer new insights.