Papers
Topics
Authors
Recent
Gemini 2.5 Flash
Gemini 2.5 Flash
194 tokens/sec
GPT-4o
7 tokens/sec
Gemini 2.5 Pro Pro
45 tokens/sec
o3 Pro
4 tokens/sec
GPT-4.1 Pro
38 tokens/sec
DeepSeek R1 via Azure Pro
28 tokens/sec
2000 character limit reached

Hybrid Heavy-Ball Systems: Reset Methods for Optimization with Uncertainty (2009.13770v2)

Published 29 Sep 2020 in eess.SY and cs.SY

Abstract: Momentum methods for convex optimization often rely on precise choices of algorithmic parameters, based on knowledge of problem parameters, in order to achieve fast convergence, as well as to prevent oscillations that could severely restrict applications of these algorithms to cyber-physical systems. To address these issues, we propose two dynamical systems, named the Hybrid Heavy-Ball System and Hybrid-inspired Heavy-Ball System, which employ a feedback mechanism for driving the momentum state toward zero whenever it points in undesired directions. We describe the relationship between the proposed systems and their discrete-time counterparts, deriving conditions based on linear matrix inequalities for ensuring exponential rates in both continuous time and discrete time. We provide numerical LMI results to illustrate the effects of our reset mechanisms on convergence rates in a setting that simulates uncertainty of problem parameters. Finally, we numerically demonstrate the efficiency and avoidance of oscillations of the proposed systems when solving both strongly convex and non-strongly convex problems.

Citations (6)

Summary

We haven't generated a summary for this paper yet.