Optimal Convergence Rate of Hamiltonian Monte Carlo for Strongly Logconcave Distributions

Abstract: We study Hamiltonian Monte Carlo (HMC) for sampling from a strongly logconcave density proportional to $e<sup>{-f}$ where $f:\mathbb{R}^d \to \mathbb{R}$ is $\mu$-strongly convex and $L$-smooth (the condition number is $\kappa = L/\mu$). We show that the relaxation time (inverse of the spectral gap) of ideal HMC is $O(\kappa)$, improving on the previous best bound of $O(\kappa<sup>{1.5})$; we complement this with an example where the relaxation time is $\Omega(\kappa)$. When implemented using a nearly optimal ODE solver, HMC returns an $\varepsilon$-approximate point in $2$-Wasserstein distance using $$\widetilde{O}((\kappa d)^{0.5} \varepsilon<sup>{-1})$$ gradient evaluations per step and $$\widetilde{O}((\kappa d)<sup>{1.5}\varepsilon<sup>{-1})$$ total time.