Papers
Topics
Authors
Recent
Search
2000 character limit reached

Learning Sparse Additive Models with Interactions in High Dimensions

Published 18 Apr 2016 in cs.LG, cs.IT, math.IT, and stat.ML | (1604.05307v1)

Abstract: A function f:Rd→Rf: \mathbb{R}^d \rightarrow \mathbb{R} is referred to as a Sparse Additive Model (SPAM), if it is of the form f(x)=∑l∈Sϕl(xl)f(\mathbf{x}) = \sum_{l \in \mathcal{S}}\phi_{l}(x_l), where S⊂[d]\mathcal{S} \subset [d], ∣S∣≪d|\mathcal{S}| \ll d. Assuming ϕl\phi_l's and S\mathcal{S} to be unknown, the problem of estimating ff from its samples has been studied extensively. In this work, we consider a generalized SPAM, allowing for second order interaction terms. For some S1⊂[d],S2⊂([d]2)\mathcal{S}_1 \subset [d], \mathcal{S}_2 \subset {[d] \choose 2}, the function ff is assumed to be of the form: f(x)=∑p∈S1ϕp(xp)+∑(l,l′)∈S2ϕ(l,l′)(xl,xl′).f(\mathbf{x}) = \sum_{p \in \mathcal{S}_1}\phi_{p} (x_p) + \sum_{(l,l^{\prime}) \in \mathcal{S}_2}\phi_{(l,l^{\prime})} (x_{l},x_{l^{\prime}}). Assuming ϕp,ϕ(l,l′)\phi_{p},\phi_{(l,l^{\prime})}, S1\mathcal{S}_1 and, S2\mathcal{S}_2 to be unknown, we provide a randomized algorithm that queries ff and exactly recovers S1,S2\mathcal{S}_1,\mathcal{S}_2. Consequently, this also enables us to estimate the underlying ϕp,ϕ(l,l′)\phi_p, \phi_{(l,l^{\prime})}. We derive sample complexity bounds for our scheme and also extend our analysis to include the situation where the queries are corrupted with noise -- either stochastic, or arbitrary but bounded. Lastly, we provide simulation results on synthetic data, that validate our theoretical findings.

Citations (14)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.