Abstract
We consider the sparsification of sums $F : \mathbb{R}n \to \mathbb{R}$ where $F(x) = f1(\langle a1,x\rangle) + \cdots + fm(\langle am,x\rangle)$ for vectors $a1,\ldots,am \in \mathbb{R}n$ and functions $f1,\ldots,fm : \mathbb{R} \to \mathbb{R}+$. We show that $(1+\varepsilon)$-approximate sparsifiers of $F$ with support size $\frac{n}{\varepsilon2} (\log \frac{n}{\varepsilon}){O(1)}$ exist whenever the functions $f1,\ldots,fm$ are symmetric, monotone, and satisfy natural growth bounds. Additionally, we give efficient algorithms to compute such a sparsifier assuming each $fi$ can be evaluated efficiently. Our results generalize the classic case of $\ellp$ sparsification, where $fi(z) = |z|p$, for $p \in (0, 2]$, and give the first near-linear size sparsifiers in the well-studied setting of the Huber loss function and its generalizations, e.g., $fi(z) = \min{|z|p, |z|2}$ for $0 < p \leq 2$. Our sparsification algorithm can be applied to give near-optimal reductions for optimizing a variety of generalized linear models including $\ellp$ regression for $p \in (1, 2]$ to high accuracy, via solving $(\log n){O(1)}$ sparse regression instances with $m \le n(\log n){O(1)}$, plus runtime proportional to the number of nonzero entries in the vectors $a1, \dots, am$.
We're not able to analyze this paper right now due to high demand.
Please check back later (sorry!).
Generate a summary of this paper on our Pro plan:
We ran into a problem analyzing this paper.