Papers
Topics
Authors
Recent
Search
2000 character limit reached

On the entropy of a noisy function

Published 6 Aug 2015 in cs.IT, math.CO, and math.IT | (1508.01464v4)

Abstract: Let 0<ϵ<1/20 < \epsilon < 1/2 be a noise parameter, and let TϵT_{\epsilon} be the noise operator acting on functions on the boolean cube {0,1}n\{0,1\}^n. Let ff be a nonnegative function on {0,1}n\{0,1\}^n. We upper bound the entropy of TϵfT_{\epsilon} f by the average entropy of conditional expectations of ff, given sets of roughly (1−2ϵ)2⋅n(1-2\epsilon)^2 \cdot n variables. In information-theoretic terms, we prove the following strengthening of "Mrs. Gerber's lemma": Let XX be a random binary vector of length nn, and let ZZ be a noise vector, corresponding to a binary symmetric channel with crossover probability ϵ\epsilon. Then, setting v=(1−2ϵ)2⋅nv = (1-2\epsilon)^2 \cdot n, we have (up to lower-order terms): H(X⊕Z)≥n⋅H(ϵ + (1−2ϵ)⋅H−1(E∣B∣=vH({Xi}i∈B)v)) H\Big(X \oplus Z\Big) \ge n \cdot H\left(\epsilon ~+~ (1-2\epsilon) \cdot H^{-1}\left(\frac{{\mathbb E}_{|B| = v} H\Big(\{X_i\}_{i\in B}\Big)}{v}\right)\right) As an application, we show that for a boolean function ff, which is close to a characteristic function gg of a subcube of dimension n−1n-1, the entropy of TϵfT_{\epsilon} f is at most that of TϵgT_{\epsilon} g. This, combined with a recent result of Ordentlich, Shayevitz, and Weinstein shows that the "Most informative boolean function" conjecture of Courtade and Kumar holds for high noise ϵ≥1/2−δ\epsilon \ge 1/2 - \delta, for some absolute constant δ>0\delta > 0. Namely, if XX is uniformly distributed in {0,1}n\{0,1\}^n and YY is obtained by flipping each coordinate of XX independently with probability ϵ\epsilon, then, provided ϵ≥1/2−δ\epsilon \ge 1/2 - \delta, for any boolean function ff holds I(f(X);Y)≤1−H(ϵ)I\Big(f(X);Y\Big) \le 1 - H(\epsilon).

Authors (1)
Citations (40)

Summary

No one has generated a summary of this paper yet.

Paper to Video (Beta)

No one has generated a video about this paper yet.

Whiteboard

No one has generated a whiteboard explanation for this paper yet.

Open Problems

We haven't generated a list of open problems mentioned in this paper yet.

Continue Learning

We haven't generated follow-up questions for this paper yet.