Spectral properties of sample covariance matrices arising from random matrices with independent non identically distributed columns

Given a random matrix $X= (x1,\ldots, xn)\in \mathcal M{p,n}$ with independent columns and satisfying concentration of measure hypotheses and a parameter $z$ whose distance to the spectrum of $\frac{1}{n} XX^T$ should not depend on $p,n$, it was previously shown that the functionals $\text{tr}(AR(z))$, for $R(z) = (\frac{1}{n}XX^T- zIp)^{-1}$ and $A\in \mathcal M{p}$ deterministic, have a standard deviation of order $O(|A|* / \sqrt n)$. Here, we show that $|\mathbb E[R(z)] - \tilde R(z)|F \leq O(1/\sqrt n)$, where $\tilde R(z)$ is a deterministic matrix depending only on $z$ and on the means and covariances of the column vectors $x1,\ldots, xn$ (that do not have to be identically distributed). This estimation is key to providing accurate fluctuation rates of functionals of $X$ of interest (mostly related to its spectral properties) and is proved thanks to the introduction of a semi-metric $ds$ defined on the set $\mathcal Dn(\mathbb H)$ of diagonal matrices with complex entries and positive imaginary part and satisfying, for all $D,D' \in \mathcal Dn(\mathbb H)$: $ds(D,D') = \max{i\in[n]} |Di - Di'|/ (\Im(Di) \Im(Di'))^{1/2}$. Possibly most importantly, the underlying concentration of measure assumption on the columns of $X$ finds an extremely natural ground for application in modern statistical machine learning algorithms where non-linear Lipschitz mappings and high number of classes form the base ingredients.