In vivo localization of larval zebrafish’s cardiac chambers in lightsheet fluorescence microscopy using a customized reward function module-incorporated Deep-Q-Network model for reinforcement learning

Abstract

Integrating deep learning techniques into optical imaging systems has ramped up the procedures of biomedical investigation and allowed an unprecedented number of ways to acquiring experimental data. This study attempts to apply a Deep-Q Network (DQN)-based reinforcement learning technique to automatically locate the cross section of the ventricular chamber of zebrafish larvae in a light-sheet fluorescence microscopy. Experimentally, a total of 920 cardiac images of zebrafish larvae were acquired, pre-formatted and manually annotated to ensure the maximal utilization and precise interpretation of the data. Subsequently, a YOLOv5 algorithm was used for image recognition, which were implemented as the state of the custom-created environment in reinforcement learning. A reward score can then be derived through the controlling of the motorized moving platform and used to train a positioning model based on the Deep-Q Network (DQN). This reinforcement learning model not only can learn and improve from the past experience, but also optimize its dynamic behavior on the basis of the designs of a reward formula and an architecture of the learning strategy. Such reward feedback system dynamically directs the sample-loaded moving platform toward an image cross-section of larval zebrafish's cardiac chamber with the best contrast and clarity. The training results indicate that the object-searching DQN model is capable of precisely allocating the optimal cross-section of the ventricular chamber, can achieve the task with a success rate of 96 % in 600 training episodes, and has the advantages of efficiency and precision when compared with the Greedy method, a random strategy and a modified reward formula.