The Phase Transition of Matrix Recovery from Gaussian Measurements Matches the Minimax MSE of Matrix Denoising (1302.2331v1)

Abstract: Let $X_0$ be an unknown $M$ by $N$ matrix. In matrix recovery, one takes $n < MN$ linear measurements $y_1,..., y_n$ of $X_0$, where $y_i = \Tr(a_i^T X_0)$ and each $a_i$ is a $M$ by $N$ matrix. For measurement matrices with Gaussian i.i.d entries, it known that if $X_0$ is of low rank, it is recoverable from just a few measurements. A popular approach for matrix recovery is Nuclear Norm Minimization (NNM). Empirical work reveals a \emph{phase transition} curve, stated in terms of the undersampling fraction $\delta(n,M,N) = n/(MN)$, rank fraction $\rho=r/N$ and aspect ratio $\beta=M/N$. Specifically, a curve $\delta^* = \delta^*(\rho;\beta)$ exists such that, if $\delta > \delta^*(\rho;\beta)$, NNM typically succeeds, while if $\delta < \delta^*(\rho;\beta)$, it typically fails. An apparently quite different problem is matrix denoising in Gaussian noise, where an unknown $M$ by $N$ matrix $X_0$ is to be estimated based on direct noisy measurements $Y = X_0 + Z$, where the matrix $Z$ has iid Gaussian entries. It has been empirically observed that, if $X_0$ has low rank, it may be recovered quite accurately from the noisy measurement $Y$. A popular matrix denoising scheme solves the unconstrained optimization problem $\text{min} | Y - X |F^2/2 + \lambda |X|* $. When optimally tuned, this scheme achieves the asymptotic minimax MSE $\cM(\rho) = \lim_{N \goto \infty} \inf_\lambda \sup_{\rank(X) \leq \rho \cdot N} MSE(X,\hat{X}_\lambda)$. We report extensive experiments showing that the phase transition $\delta^*(\rho)$ in the first problem coincides with the minimax risk curve $\cM(\rho)$ in the second problem, for {\em any} rank fraction $0 < \rho < 1$.