Improved Algorithms for Convex-Concave Minimax Optimization

This paper studies minimax optimization problems $\minx \maxy f(x,y)$, where $f(x,y)$ is $mx$-strongly convex with respect to $x$, $my$-strongly concave with respect to $y$ and $(Lx,L{xy},Ly)$-smooth. Zhang et al. provided the following lower bound of the gradient complexity for any first-order method: $\Omega\Bigl(\sqrt{\frac{Lx}{mx}+\frac{L{xy}^2}{m_x my}+\frac{Ly}{my}}\ln(1/\epsilon)\Bigr).$ This paper proposes a new algorithm with gradient complexity upper bound $\tilde{O}\Bigl(\sqrt{\frac{Lx}{mx}+\frac{L\cdot L{xy}}{mx my}+\frac{Ly}{my}}\ln\left(1/\epsilon\right)\Bigr),$ where $L=\max{Lx,L{xy},Ly}$. This improves over the best known upper bound $\tilde{O}\left(\sqrt{\frac{L^2}{mx my}} \ln^{3\left(1/\epsilon\right)\right)$} by Lin et al. Our bound achieves linear convergence rate and tighter dependency on condition numbers, especially when $L{xy}\ll L$ (i.e., when the interaction between $x$ and $y$ is weak). Via reduction, our new bound also implies improved bounds for strongly convex-concave and convex-concave minimax optimization problems. When $f$ is quadratic, we can further improve the upper bound, which matches the lower bound up to a small sub-polynomial factor.