A Connection between Good Rate-distortion Codes and Backward DMCs

Abstract: Let $X<sup>n\in\mathcal{X}<sup>n$ be a sequence drawn from a discrete memoryless source, and let $Y<sup>n\in\mathcal{Y}<sup>n$ be the corresponding reconstruction sequence that is output by a good rate-distortion code. This paper establishes a property of the joint distribution of $(X<sup>n,Y<sup>n)$. It is shown that for $D&gt;0$, the input-output statistics of a $R(D)$-achieving rate-distortion code converge (in normalized relative entropy) to the output-input statistics of a discrete memoryless channel (dmc). The dmc is "backward" in that it is a channel from the reconstruction space $\mathcal{Y}<sup>n$ to source space $\mathcal{X}<sup>n$. It is also shown that the property does not necessarily hold when normalized relative entropy is replaced by variational distance.