Structure Learning in Graphical Models from Indirect Observations

Abstract: This paper considers learning of the graphical structure of a $p$-dimensional random vector $X \in R<sup>p$ using both parametric and non-parametric methods. Unlike the previous works which observe $x$ directly, we consider the indirect observation scenario in which samples $y$ are collected via a sensing matrix $A \in R^{d\times p}$, and corrupted with some additive noise $w$, i.e, $Y = AX + W$. For the parametric method, we assume $X$ to be Gaussian, i.e., $x\in R^p\sim N(\mu, \Sigma)$ and $\Sigma \in R^{p\times p}$. For the first time, we show that the correct graphical structure can be correctly recovered under the indefinite sensing system ($d &lt; p$) using insufficient samples ($n &lt; p$). In particular, we show that for the exact recovery, we require dimension $d = \Omega(p<sup>{0.8})$ and sample number $n = \Omega(p^{0.8}\log³ p)$. For the nonparametric method, we assume a nonparanormal distribution for $X$ rather than Gaussian. Under mild conditions, we show that our graph-structure estimator can obtain the correct structure. We derive the minimum sample number $n$ and dimension $d$ as $n\gtrsim (deg)⁴ \log⁴ n$ and $d \gtrsim p + (deg\cdot\log(d-p))<sup>{\beta/4}$, respectively, where deg is the maximum Markov blanket in the graphical model and $\beta &gt; 0$ is some fixed positive constant. Additionally, we obtain a non-asymptotic uniform bound on the estimation error of the CDF of $X$ from indirect observations with inexact knowledge of the noise distribution. To the best of our knowledge, this bound is derived for the first time and may serve as an independent interest. Numerical experiments on both real-world and synthetic data are provided confirm the theoretical results.