Connoisseur: Provenance Analysis in Paintings

Abstract

Authorship attribution and matching have become paramount activities in current digital art repositories and communities, which seek to efficiently catalog and authenticate the ever-growing number of digitized paintings, uploaded in professional and casual capturing setups, by their own authors or enthusiasts alike. In this work, we employ convolutional network-based strategies to identify and classify art-related digital artifacts over the Painter by Numbers dataset. Firstly, we propose to exploit the authorship, style and genre annotated information in a multi-task setup, in which patches of paintings are encoded through a multiple outputs network and, in a second stage, used in an Siamese discriminating network to solve the authorship matching problem. Secondly, we combine the available annotated information in a more efficient manner, by posing the Painter by Numbers challenge as a multi-label problem. Empirical results show a substantial increase in class-balanced accuracy and ROC AUC score for both multi-task solutions, compared with their simpler counterparts trained using only authorship annotation. Furthermore, a slight increase in ROC AUC score is observed in the multi-label setup, indicating that this simple combination strategy is beneficial to training convergence.