Online structural SVM learning by dual ascending procedure

Abstract

We propose online learning algorithms for structural SVM that has promising applications in large-scale learning. A framework is introduced for analyzing the online learning of structural SVM from primal perspective to dual perspective. The task of minimizing the primal objective function is converted to incremental increasing of the dual objective function. The model's parameter is learned through updating dual coefficients. We propose two update schemes: all outputs update scheme and most violated output update scheme. The first scheme updates dual coefficients of all the outputs, while the second schemes only updated dual coefficients of the most violated output. The performance of structural SVM is improved in online learning process. Experimental results on multiclass classification task and sequence tagging task show that our online learning algorithms achieve satisfying accuracy while reducing the computational complexity.