Deep Learning in High Dimension: Neural Network Approximation of Analytic Functions in $L^2(\mathbb{R}^d,γ_d)$

Abstract: For artificial deep neural networks, we prove expression rates for analytic functions $f:\mathbb{R}<sup>d\to\mathbb{R}$ in the norm of $L<sup>2(\mathbb{R}<sup>d,\gamma_d)$ where $d\in {\mathbb{N}}\cup{ \infty }$. Here $\gamma_d$ denotes the Gaussian product probability measure on $\mathbb{R}<sup>d$. We consider in particular ReLU and ReLU${}<sup>k$ activations for integer $k\geq 2$. For $d\in\mathbb{N}$, we show exponential convergence rates in $L<sup>2(\mathbb{R}<sup>d,\gamma_d)$. In case $d=\infty$, under suitable smoothness and sparsity assumptions on $f:\mathbb{R}<sup>{\mathbb{N}}\to\mathbb{R}$, with $\gamma_\infty$ denoting an infinite (Gaussian) product measure on $\mathbb{R}<sup>{\mathbb{N}}$, we prove dimension-independent expression rate bounds in the norm of $L<sup>2(\mathbb{R}<sup>{\mathbb{N}},\gamma_\infty)$. The rates only depend on quantified holomorphy of (an analytic continuation of) the map $f$ to a product of strips in $\mathbb{C}<sup>d$. As an application, we prove expression rate bounds of deep ReLU-NNs for response surfaces of elliptic PDEs with log-Gaussian random field inputs.