Similarity learning networks uniquely identify individuals of four marine and terrestrial species

Abstract

Estimating the size of animal populations plays an important role in evidence-based conservation and management. Some methods for estimating population size rely on animals being individually identifiable. Traditionally, this has been done by marking physically captured animals, but increasingly, animals with distinctive natural markings are surveyed noninvasively using cameras. Animal reidentification from photographs is usually done manually, which is expensive, laborious, and requires considerable skill. An alternative is to develop computer vision methods that can support or replace the manual identification task. We developed an automated approach using deep learning to identify whether a pair of photographs is of the same individual or not. The core of the approach is a similarity learning network that uses paired convolutional neural networks with a triplet loss function to summarize image pairs and decide whether they are from the same individual. Prior to the main matching step, two additional convolutional neural networks perform image segmentation, cropping the animal object within the image, and orientation prediction, deciding which side of the animal was photographed. We applied the approach to four species, with images of the same individual often spanning several years: systematic surveys of bottlenose dolphins (Tursiops truncatus, 2008–2019) and harbor seals (Phoca vitulina, 2015–2019), a citizen science dataset of western leopard toads (Sclerophrys pantherina, unknown dates), and a publicly available repository of humpback whale images (Megaptera novaeangliae, unknown dates). For these species, our best-performing models were able to identify whether a pair of images were from the same individual or different individuals in 95.8%, 94.6%, 88.2%, and 83.8% of the cases, respectively. We found that triplet loss functions outperformed binary cross-entropy loss functions and that data augmentation and additional manual curation of training data provided small but consistent improvements in performance. These results demonstrate the potential of deep learning to replace or, more likely, support and facilitate manual individual identification efforts.