Social media image classification for jellyfish monitoring

Abstract

The Portuguese man-of-war is responsible for the most common and severe stings worldwide. Jellyfish monitoring is essential to manage stings, and social media is a valuable data source for obtaining observations of this species. This study reports on using Convolutional Neural Networks for Portuguese man-of-war image classification extracted from social media posts. We created a suitable dataset and trained three different neural networks: VGG-16, ResNet50, and InceptionV3, with and without a pre-trained step with the ImageNet dataset. The pre-trained ResNet50 network presented the best results, obtaining 94% accuracy and 95% precision, recall, and F1 score. We conclude that Convolutional Neural Networks can be very effective for recognizing Portuguese man-of-war images from social media, helping in obtaining data about its occurrence and distribution.