Upper Bounds on the Relative Entropy and Rényi Divergence as a Function of Total Variation Distance for Finite Alphabets (1503.03417v4)

Abstract: A new upper bound on the relative entropy is derived as a function of the total variation distance for probability measures defined on a common finite alphabet. The bound improves a previously reported bound by Csisz\'ar and Talata. It is further extended to an upper bound on the R\'enyi divergence of an arbitrary non-negative order (including $\infty$) as a function of the total variation distance.