Embedding Inequalities for Barron-type Spaces
(2305.19082)Abstract
An important problem in machine learning theory is to understand the approximation and generalization properties of two-layer neural networks in high dimensions. To this end, researchers have introduced the Barron space $\mathcal{B}s(\Omega)$ and the spectral Barron space $\mathcal{F}s(\Omega)$, where the index $s\in [0,\infty)$ indicates the smoothness of functions within these spaces and $\Omega\subset\mathbb{R}d$ denotes the input domain. However, the precise relationship between the two types of Barron spaces remains unclear. In this paper, we establish a continuous embedding between them as implied by the following inequality: for any $\delta\in (0,1), s\in \mathbb{N}{+}$ and $f: \Omega \mapsto\mathbb{R}$, it holds that [ \delta |f|{\mathcal{F}{s-\delta}(\Omega)}\lesssims |f|{\mathcal{B}s(\Omega)}\lesssims |f|{\mathcal{F}{s+1}(\Omega)}. ] Importantly, the constants do not depend on the input dimension $d$, suggesting that the embedding is effective in high dimensions. Moreover, we also show that the lower and upper bound are both tight.
We're not able to analyze this paper right now due to high demand.
Please check back later (sorry!).
Generate a summary of this paper on our Pro plan:
We ran into a problem analyzing this paper.