Creating synthetic data sets for training of neural networks for automatic catch analysis in fisheries

Abstract

Accurate estimation of catch is essential for sustainable fisheries. It ensures precise catch reporting, provides a better basis for stock assessment, and helps prevent overfishing. With recent advances in deep learning, this could be solved using computer vision, however, collecting and annotating data for different fisheries, all with diverse catch distributions and different imaging equipment, is expensive and time-consuming and is currently limiting the adoption of the technology. To address this issue, we propose the use of synthetic data sets, created in simulation, for training of neural networks for the task of automatic catch analysis. Although the domain is subject to large amounts of variation in the image data, we hypothesize that much of this variation is due to clutter and variations in the appearance of the fish as captured by the camera, rather than inherent variations in the raw material itself. As such, the variation can be covered effectively in data sets generated in simulation, without the need for large data sets of 3D-models for each species, which are also costly to produce. This is demonstrated by training a neural network for instance segmentation, instance classification and key point detection, solely on synthetic data created with only five 3D-models of fish. The neural network is evaluated on real data, gathered with a variety of sensors onboard different fishing vessels, demonstrating that it generalizes across different domains. This evaluation concludes that synthetic data can be a valuable addition to real data for computer vision applications for catch analysis.