Deep sea spy: An online citizen science annotation platform for science and ocean literacy

Abstract

The recent development of deep-sea observatories has enabled the acquisition of high temporal resolution imagery for studying the dynamics of deep-sea communities on hourly to multi-decadal scales. These unprecedented datasets offer valuable insight into the variation of species abundance and biology in relation to changes in environmental conditions. Since 2010, camera systems deployed at hydrothermal vents have acquired over 11 terabytes (TB) of data that cannot be processed by research labs only. Although deep learning offers an alternative to human processing, training algorithms requires large annotated reference datasets. The Deep Sea Spy project allows citizens to contribute to the annotation of pictures acquired with underwater platforms. Based on approximately 4000 photos, each annotated 10 times by independent participants, we were able to develop a data validation workflow that can be applied to similar databases. We compared these annotations with expert-annotated data and analysed the agreement rate among participants for each of the 15,000 annotated individual organisms to optimise the robustness and confidence level in non-expert citizen science. The optimal number of repeat annotations per photo was also analysed to guide the definition of a trade-off between the accuracy and amount of data. An agreement rate of 0.4 (i.e., 4 out of 10 participants detecting one given individual) was established as an efficient threshold to reach counts similar to that obtained from an expert. One important result lies in the robustness of the temporal trends of species abundance as revealed by time-series analyses. Regarding the number of times a photo needs to be annotated, results varied greatly depending on the target species and the difficulty of the associated task. Finally, we present the communication tools and actions deployed during the project and how the platform can serve educational and decision-making purposes. Deep Sea Spy and the proposed workflow have a strong potential to enhance marine environmental observation and monitoring.