Faster $p$-Norm Regression Using Sparsity

Abstract: For a matrix $A\in \mathbb{R}^{n\times d}$ with $n\geq d$, we consider the dual problems of $$\min |Ax-b|<em>p^p, \, b\in \mathbb{R}<sup>n$$ and $$\min</em>{A^\top x=b} |x|_p^p,\, b\in \mathbb{R}<sup>d$$. We improve the runtimes for solving these problems to high accuracy for every $p&gt;1$ for sufficiently sparse matrices. We show that recent progress on fast sparse linear solvers can be leveraged to obtain faster than matrix-multiplication algorithms for any $p &gt; 1$, i.e., in time $\tilde{O}(pn<sup>\theta)$ for some $\theta &lt; \omega$, the matrix multiplication constant. We give the first high-accuracy input sparsity $p$-norm regression algorithm for solving $\min |Ax-b|_p<sup>p$ with $1 &lt; p \leq 2$, via a new row sampling theorem for the smoothed $p$-norm function. This algorithm runs in time $\tilde{O}(\text{nnz}(A) + d<sup>4)$ for any $1&lt;p\leq 2$, and in time $\tilde{O}(\text{nnz}(A) + d<sup>\theta)$ for $p$ close to $2$, improving on the previous best bound where the exponent of $d$ grows with $\max{p, p/(p-1)}$.