Optimal methods for convex nested stochastic composite optimization

Abstract

Recently, convex nested stochastic composite optimization (NSCO) has received considerable interest for its applications in reinforcement learning and risk-averse optimization. However, existing NSCO algorithms have worse stochastic oracle complexities, by orders of magnitude, than those for simpler stochastic optimization problems without nested structures. Additionally, these algorithms require all outer-layer functions to be smooth, a condition violated by some important applications. This raises a question regarding whether the nested composition make stochastic optimization more difficult in terms of oracle complexity. In this paper, we answer the question by developing order-optimal algorithms for convex NSCO problems constructed from an arbitrary composition of smooth, structured non-smooth, and general non-smooth layer functions. When all outer-layer functions are smooth, we propose a stochastic sequential dual (SSD) method to achieve an oracle complexity of \(\mathcal {O}(1/\epsilon ^2)\) (resp., \(\mathcal {O}(1/\epsilon )\)) when the problem is convex (resp., strongly convex). If any outer-layer function is non-smooth, we propose a non-smooth stochastic sequential dual (nSSD) method to achieve an \(\mathcal {O}(1/\epsilon ^2)\) oracle complexity. We provide a lower complexity bound to show the latter \(\mathcal {O}(1/\epsilon ^2)\) complexity to be unimprovable, even under a strongly convex setting. All these complexity results seem to be new in the literature, and they indicate that convex NSCO problems have the same order of oracle complexity as problems without nested composition, except in the strongly convex and outer non-smooth cases.