Papers
Topics
Authors
Recent
Search
2000 character limit reached

Iterative Refinement for ℓp\ell_p-norm Regression

Published 21 Jan 2019 in cs.DS, cs.NA, math.OC, stat.ML, and math.NA | (1901.06764v1)

Abstract: We give improved algorithms for the ℓp\ell_{p}-regression problem, min⁡x∥x∥p\min_{x} \|x\|_{p} such that Ax=b,A x=b, for all p∈(1,2)∪(2,∞).p \in (1,2) \cup (2,\infty). Our algorithms obtain a high accuracy solution in O~p(m∣p−2∣2p+∣p−2∣)≤O~p(m13)\tilde{O}_{p}(m^{\frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{1}{3}}) iterations, where each iteration requires solving an m×mm \times m linear system, mm being the dimension of the ambient space. By maintaining an approximate inverse of the linear systems that we solve in each iteration, we give algorithms for solving ℓp\ell_{p}-regression to 1/poly(n)1 / \text{poly}(n) accuracy that run in time O~p(mmax⁡{ω,7/3}),\tilde{O}_p(m^{\max\{\omega, 7/3\}}), where ω\omega is the matrix multiplication constant. For the current best value of ω>2.37\omega > 2.37, we can thus solve ℓp\ell_{p} regression as fast as ℓ2\ell_{2} regression, for all constant pp bounded away from $1.$ Our algorithms can be combined with fast graph Laplacian linear equation solvers to give minimum ℓp\ell_{p}-norm flow / voltage solutions to 1/poly(n)1 / \text{poly}(n) accuracy on an undirected graph with mm edges in O~p(m1+∣p−2∣2p+∣p−2∣)≤O~p(m43)\tilde{O}_{p}(m^{1 + \frac{|p-2|}{2p + |p-2|}}) \le \tilde{O}_{p}(m^{\frac{4}{3}}) time. For sparse graphs and for matrices with similar dimensions, our iteration counts and running times improve on the pp-norm regression algorithm by [Bubeck-Cohen-Lee-Li STOC`18] and general-purpose convex optimization algorithms. At the core of our algorithms is an iterative refinement scheme for ℓp\ell_{p}-norms, using the smoothed ℓp\ell_{p}-norms introduced in the work of Bubeck et al. Given an initial solution, we construct a problem that seeks to minimize a quadratically-smoothed ℓp\ell_{p} norm over a subspace, such that a crude solution to this problem allows us to improve the initial solution by a constant factor, leading to algorithms with fast convergence.

Citations (45)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.