Advancing Electron Microscopy using Deep Learning

Electron microscopy, a sub-field of microanalysis, is a victim of its own success. The widespread use of electron microscopy for imaging molecules and materials has had an enormous impact on our understanding of countless systems and has accelerated impacts in drug discovery and materials design, for electronic, energy, environment and health applications. With this success a bottleneck has emerged, as the rate at which we can collect data has significantly exceeded the rate at which we can analyse it. Fortunately, this has coincided with the rise of advanced computational methods, including data science and machine learning. Deep learning, a sub-field of machine learning capable of learning from large quantities of data such as images, is ideally suited to overcome some of the challenges of electron microscopy at scale. There are a variety of different deep learning approaches relevant to the field, with unique advantages and disadvantages. In this review, we describe some well-established methods, with some recent examples, and introduce some new methods currently emerging in computer science. Our summary of deep learning is designed to guide electron microscopists to choose the right deep learning algorithm for their research and prepare for their digital future.