Dynamic spectral fluorescence microscopy via event-based & CMOS image-sensor fusion.

Abstract

We present a widefield fluorescence microscope that integrates an event-based image sensor (EBIS) with a CMOS image sensor (CIS) for ultra-fast microscopy with spectral distinction capabilities. The EBIS achieves a temporal resolution of ∼10μs (∼ 100,000 frames/s), while the CIS provides diffraction-limited spatial resolution. A diffractive optical element encodes spectral information into a diffractogram, which is recorded by the CIS. The diffractogram is processed using a deep neural network to resolve the fluorescence of two beads, whose emission peaks are separated by only 7 nm and exhibit an 88% spectral overlap. We validate our microscope by imaging the capillary flow of fluorescent beads, demonstrating a significant advancement in ultra-fast spectral microscopy. This technique holds broad potential for elucidating foundational dynamic biological processes.