Characterization and Analysis of Text-to-Image Diffusion Models

Abstract

Diffusion models have rapidly emerged as a prominent AI model for image generation. Despite its importance, however, little have been understood within the computer architecture community regarding this emerging AI algorithm. We conduct a workload characterization on the inference process of diffusion models using Stable Diffusion. Our characterization uncovers several critical performance bottlenecks of diffusion models, the computational overhead of which gets aggravated as image size increases. We also discuss several performance optimization opportunities that leverage approximation and sparsity, which help alleviate diffusion model's computational complexity. These findings highlight the need for domain-specific hardware that reaps out the benefits of our proposal, paving the way for accelerated image generation.