Computer Vision for Fashion: From Individual Recommendations to World-wide Trends

Abstract

The fashion domain is a magnet for computer vision. New vision problems are emerging in step with the fashion industry's rapid evolution towards an online, social, and personalized business. Style models, trend forecasting, interactive search, and recommendation all require visual understanding with rich detail and subtlety. As a result, research in this area is poised to have great influence on how people shop, how the fashion industry analyzes its enterprise, and how we model the cultural trends revealed by what people wear. In this talk, I will present our work over the last few years developing computer vision methods for fashion. To begin, we explore how to discover styles from Web photos, learning how people assemble their outfits and the latent themes they share. Leveraging such styles, we show how to infer compatibility of new garments, optimize personalized mix-and-match capsule wardrobes, suggest minimal edits to make an outfit more fashionable, and recommend clothing that flatters diverse human body shapes. Next, turning to the world stage, we investigate fashion forecasting and influence. Given photos of fashion products, we learn to forecast what looks and styles will be popular in the future. We further boost those forecasts by modeling the spatio-temporal style influences between 44 major world cities. Throughout, by learning models from unlabeled Web photos, our approaches sidestep subjective manual annotations in favor of direct observations of what people choose to wear.