Using Block Coordinate Descent to Learn Sparse Coding Dictionaries with a Matrix Norm Update

Abstract

Researchers have recently examined a modified approach to sparse coding that encourages dictionaries to learn anomalous features. This is done by incorporating the matrix I-norm, or $\ell_{1,\infty}$ mixed matrix norm, into the dictionary update portion of a sparse coding algorithm. However, solving a matrix norm minimization problem in each iteration of the algorithm causes it to run more slowly. The purpose of this paper is to introduce block coordinate descent, a subgradient-like approach to minimizing the matrix norm, to the dictionary update. This approach removes the need to solve a convex optimization program in each iteration and dramatically reduces the time required to learn a dictionary. Importantly, the dictionary learned in this manner can still model anomalous features present in a dataset.