Papers
Topics
Authors
Recent
Search
2000 character limit reached

Sharp MSE Bounds for Proximal Denoising

Published 13 May 2013 in cs.IT, math.IT, and math.OC | (1305.2714v5)

Abstract: Denoising has to do with estimating a signal x0x_0 from its noisy observations y=x0+zy=x_0+z. In this paper, we focus on the "structured denoising problem", where the signal x0x_0 possesses a certain structure and zz has independent normally distributed entries with mean zero and variance σ2\sigma^2. We employ a structure-inducing convex function f(⋅)f(\cdot) and solve min⁡x{12∥y−x∥22+σλf(x)}\min_x\{\frac{1}{2}\|y-x\|_2^2+\sigma\lambda f(x)\} to estimate x0x_0, for some λ>0\lambda>0. Common choices for f(⋅)f(\cdot) include the ℓ1\ell_1 norm for sparse vectors, the ℓ1−ℓ2\ell_1-\ell_2 norm for block-sparse signals and the nuclear norm for low-rank matrices. The metric we use to evaluate the performance of an estimate x∗x^* is the normalized mean-squared-error NMSE(σ)=E∥x∗−x0∥22σ2\text{NMSE}(\sigma)=\frac{\mathbb{E}\|x^*-x_0\|_2^2}{\sigma^2}. We show that NMSE is maximized as σ→0\sigma\rightarrow 0 and we find the \emph{exact} worst case NMSE, which has a simple geometric interpretation: the mean-squared-distance of a standard normal vector to the λ\lambda-scaled subdifferential λ∂f(x0)\lambda\partial f(x_0). When λ\lambda is optimally tuned to minimize the worst-case NMSE, our results can be related to the constrained denoising problem min⁡f(x)≤f(x0){∥y−x∥2}\min_{f(x)\leq f(x_0)}\{\|y-x\|_2\}. The paper also connects these results to the generalized LASSO problem, in which, one solves min⁡f(x)≤f(x0){∥y−Ax∥2}\min_{f(x)\leq f(x_0)}\{\|y-Ax\|_2\} to estimate x0x_0 from noisy linear observations y=Ax0+zy=Ax_0+z. We show that certain properties of the LASSO problem are closely related to the denoising problem. In particular, we characterize the normalized LASSO cost and show that it exhibits a "phase transition" as a function of number of observations. Our results are significant in two ways. First, we find a simple formula for the performance of a general convex estimator. Secondly, we establish a connection between the denoising and linear inverse problems.

Authors (2)
Citations (75)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.