Stochastic Optimization with Constraints: A Non-asymptotic Instance-Dependent Analysis (2404.00042v1)
Abstract: We consider the problem of stochastic convex optimization under convex constraints. We analyze the behavior of a natural variance reduced proximal gradient (VRPG) algorithm for this problem. Our main result is a non-asymptotic guarantee for VRPG algorithm. Contrary to minimax worst case guarantees, our result is instance-dependent in nature. This means that our guarantee captures the complexity of the loss function, the variability of the noise, and the geometry of the constraint set. We show that the non-asymptotic performance of the VRPG algorithm is governed by the scaled distance (scaled by $\sqrt{N}$) between the solutions of the given problem and that of a certain small perturbation of the given problem -- both solved under the given convex constraints; here, $N$ denotes the number of samples. Leveraging a well-established connection between local minimax lower bounds and solutions to perturbed problems, we show that as $N \rightarrow \infty$, the VRPG algorithm achieves the renowned local minimax lower bound by H`{a}jek and Le Cam up to universal constants and a logarithmic factor of the sample size.
- A framework for estimation of convex functions. Statistica Sinica, pages 423–456, 2015.
- Asymptotic normality and optimality in nonsmooth stochastic approximation. arXiv preprint arXiv:2301.06632, 2023.
- Implicit functions and solution mappings: A view from variational analysis, volume 616. Springer, 2009.
- Local minimax complexity of stochastic convex optimization. Advances in Neural Information Processing Systems, 29, 2016.
- J. C. Duchi and F. Ruan. Asymptotic optimality in stochastic optimization. 2021.
- J. Dupacová and R. Wets. Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems. The annals of statistics, 16(4):1517–1549, 1988.
- J. Hájek. Local asymptotic minimax and admissibility in estimation. In Proceedings of the sixth Berkeley symposium on mathematical statistics and probability, volume 1, pages 175–194, 1972.
- J.-B. Hiriart-Urruty and C. Lemaréchal. Convex analysis and minimization algorithms I: Fundamentals, volume 305. Springer science & business media, 1996.
- R. Johnson and T. Zhang. Accelerating stochastic gradient descent using predictive variance reduction. Advances in neural information processing systems, 26, 2013.
- Is temporal difference learning optimal? an instance-dependent analysis. SIAM Journal on Mathematics of Data Science, 3(4):1013–1040, 2021.
- Instance-optimality in optimal value estimation: Adaptivity via variance-reduced q-learning. arXiv preprint arXiv:2106.14352, 2021.
- A. J. King. Asymptotic behaviour of solutions in stochastic optimization: nonsmooth analysis and the derivation of non-normal limit distributions (least squares). 1986.
- Asymptotic theory for solutions in statistical estimation and stochastic programming. Mathematics of Operations Research, 18(1):148–162, 1993.
- L. Le Cam et al. Limits of experiments. In Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, volume 1, pages 245–261. University of California Press, 1972.
- Asymptotics in statistics: some basic concepts. 2000.
- Y. Nesterov. Primal-dual subgradient methods for convex problems. Mathematical programming, 120(1):221–259, 2009.
- R. Poliquin and R. T. Rockafellar. Tilt stability of a local minimum. SIAM Journal on Optimization, 8(2):287–299, 1998.
- A. Shapiro. Asymptotic properties of statistical estimators in stochastic programming. The Annals of Statistics, 17(2):841–858, 1989.
- A. W. Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000.
- M. J. Wainwright. Stochastic approximation with cone-contractive operators: Sharp ℓ∞subscriptℓ\ell_{\infty}roman_ℓ start_POSTSUBSCRIPT ∞ end_POSTSUBSCRIPT-bounds for q𝑞qitalic_q-learning. arXiv preprint arXiv:1905.06265, 2019.
- M. J. Wainwright. Variance-reduced q𝑞qitalic_q-learning is minimax optimal. arXiv preprint arXiv:1906.04697, 2019.
- J. Wellner et al. Weak convergence and empirical processes: with applications to statistics. Springer Science & Business Media, 2013.
- L. Xiao and T. Zhang. A proximal stochastic gradient method with progressive variance reduction. SIAM Journal on Optimization, 24(4):2057–2075, 2014.