7

Consider two RV $X,Y$. If $d_{TV}(X,Y)=0$ you may couple them in such a way that $$\rho_{XY}=\frac {\operatorname{COV}(X,Y) }{\sigma_X\sigma_Y}=1.$$

So, is there any formula to bound $d_{TV}$ in terms of $\rho$?

Michael Hardy
  • 1
MR_BD
  • 6,307
  • 1
    Sorry. Just to clarify. What is $d_{TV}(X,Y)$? – Mostafa Ayaz Sep 03 '18 at 07:48
  • @MostafaAyaz https://en.wikipedia.org/wiki/Total_variation_distance_of_probability_measures – MR_BD Sep 03 '18 at 12:30
  • 3
    Well, if you take $Y=a X$ , you still have $\rho=1$ and you can make $d_{TV}$ as big as you wish. So I'd say no. – leonbloy Sep 04 '18 at 23:26
  • @leonbloy What about the other hand? Consider $\rho < \rho^*$ for each coupling. Is there any limit on $d_{TV}$? – MR_BD Sep 05 '18 at 08:07
  • I don't think your claim is even correct, we can have random variables without expectation or variance that are the same. Suppose we take the Cauchy distribution. In that case your claim does not hold since the covariance is undefined if no second moments exist. – Jan Sep 05 '18 at 15:04
  • Note that if we take $X,Z ~ N(0,1)$ and $Y=\rho X + \sqrt{1-\rho^2} Z$ we have $d_{TV}(X,Y)=0$ for all $\rho \in [-1,1]$. I think as the other comments have pointed out it is not possible to create such a bound. Perhaps you could give an easy example where the bound works and we can probably easily find a counterexample! – Jan Sep 05 '18 at 15:11
  • @Jan Sorry, but I didn't get the point of your example. Of course there is some coupling in which $\rho<1$ but there is some with $\rho =1$. BTW, my main concern is on discrete distributions. – MR_BD Sep 06 '18 at 08:03
  • 1
    nice to see you leila. + – Mikasa Sep 08 '18 at 18:10

1 Answers1

1

Here I do not provide a bound but a relation that can yield bounds.

Let the total variation distance between two probability measures defined on a space $(\Omega,\mathcal F)$ be $$ d_{TV}(P,Q)=2\sup_{A\in\mathcal F}|P(A)-Q(A)|. $$ The coupling inequality shows that for all coupling $(X,Y)$ such that: $$ d_{TV}(P,Q)\leq 2P(X\neq Y). $$ The maximal coupling theorem shows that the equality can be achieved by a coupling. So if $P(X\neq Y)$ can be related to $\rho$, then the problem is solved. Without loss of generality assume $X$ and $Y$ are zero-mean unit variance random variables: $$ \rho=E(XY)=P(X\neq Y)E(XY|X\neq Y)+P(X=Y)E(XY|X=Y). $$ But we have: $$ P(X=Y)E(XY|X=Y)+P(X\neq Y)E(X^2|X\neq Y)=E(X^2)=1 $$ and $$ P(X=Y)E(XY|X=Y)+P(X\neq Y)E(Y^2|X\neq Y)=E(Y^2)=1. $$ Combining three above equations, we get: $$ 2(1-\rho)=P(X\neq Y)E[(X-Y)^2|X\neq Y]. $$ which means that for maximal coupling of $(X,Y)$ we have: $$ 4(1-\rho)=d_{TV}(P,Q)E[(X-Y)^2|X\neq Y]. $$ Now if $X$ and $Y$ are bounded random variables with $|X|\leq M$ and $|Y|\leq M$, then:

$$ (1-\rho)\leq M^2 d_{TV}(P,Q). $$

There might be other ways of bounding $\rho$ however this is not clear at the moment to me.

Arash
  • 11,307