The road from MLE to EM to VAE: A brief tutorial

Variational Auto-Encoders (VAEs) have emerged as one of the most popular genres of generative models, which are learned to characterize the data distribution. The classic Expectation Maximization (EM) algorithm aims to learn models with hidden variables. Essentially, both of them are iteratively optimizing the evidence lower bound (ELBO) to maximize to the likelihood of the observed data.

This short tutorial joins them up into a line and offer a good way to thoroughly understand EM and VAE with minimal knowledge. It is especially helpful to beginners and readers with experiences in machine learning applications but no statistics background.