Zeroth-order stochastic compositional algorithms for risk-aware learning

DS Kalogerias, WB Powell - SIAM Journal on Optimization, 2022 - SIAM
SIAM Journal on Optimization, 2022SIAM
We present \itFree-MESSAGE^,p, the first zeroth-order algorithm for (weakly) convex mean-
semideviation-based risk-aware learning, which is also the first-ever three-level zeroth-order
compositional stochastic optimization algorithm. Using a nontrivial extension of Nesterov's
classical results on Gaussian smoothing, we develop the \itFree-MESSAGE^,p algorithm
from first principles and show that it essentially solves a smoothed surrogate to the original
problem, the former being a uniform approximation of the latter, in a useful, convenient …
We present , the first zeroth-order algorithm for (weakly) convex mean-semideviation-based risk-aware learning, which is also the first-ever three-level zeroth-order compositional stochastic optimization algorithm. Using a nontrivial extension of Nesterov's classical results on Gaussian smoothing, we develop the algorithm from first principles and show that it essentially solves a smoothed surrogate to the original problem, the former being a uniform approximation of the latter, in a useful, convenient sense. We then present a complete analysis of the algorithm, which establishes convergence in a user-tunable neighborhood of the optimal solutions of the original problem for convex costs, as well as explicit convergence rates for convex, weakly convex, and strongly convex costs, in a unified way. Orderwise, and for fixed problem parameters, our results demonstrate no sacrifice in convergence speed as compared to existing first-order methods, while striking a certain balance among the condition of the problem, its dimensionality, and the accuracy of the obtained results, naturally extending previous results in zeroth-order risk-neutral learning.
Society for Industrial and Applied Mathematics
Showing the best result for this search. See all results