13

Let $X$ be a random variable and $f, g: \mathbb{R} \rightarrow \mathbb{R}$ be increasing functions. Show that $cov(f(X), g(X)) \ge 0$.

The following hint was also provided: Assume $X$, $Y$ are independent and identically distributed, then show $E[(f(X)-f(Y))(g(X)-g(Y))] \ge 0$.

My attempt: Since $f$ and $g$ are increasing, then $(f(x) - f(y))(g(x) - g(y)) \ge 0$ for all $x, y \in \mathbb{R}$. Thus, $E[(f(X) - f(Y))(g(X) - g(Y))] \ge 0$ by the monotonicity of expectations.

Expanding, we get $E[f(X)g(X)]-E[f(X)g(Y)]-E[f(Y)g(X)]+E[f(Y)g(Y)] \ge 0$. Now due to independence, the LHS becomes $E[f(X)g(X)]-E[f(X)]E[g(Y)]-E[f(Y)]E[g(X)]+E[f(Y)g(Y)]$. Then due to identically distributed, the LHS further becomes $2E[f(X)g(X)]-2E[f(X)]E[g(X)]$. So together we have $2cov(f(X), g(X)) \ge 0$ and we are done.

My main query: This whole proof relies on the fact that $X$ and $Y$ are i.i.d. How can we just simply assume this? If we didn't make this assumption, then we would never have been able to break up the expectations and collect like terms. Is this proof correct or do I need a proof that does not rely on the iid of $X$ and $Y$?

elbarto
  • 3,446
  • 2
    Since $Y$ does not appear in the original statement, you can assume it has whatever (possible) properties you want – Henry Sep 10 '17 at 10:05
  • To be formal, the existence of $Y$ is a corollary of Kolmogorov's existence theorem. – Zhanxiong Feb 11 '21 at 12:43

1 Answers1

9

Your proof is correct and complete.

Essential is that whenever there is a random variable $X$ defined on some probability space $\langle\Omega,\mathcal A,P\rangle$ then it is always possible to create a new probability space $\langle\Omega_1,\mathcal A_1,P_1\rangle$ together with random variables $X_1:\Omega_1\to\mathbb R$ and $Y_1:\Omega_1\to\mathbb R$ such that $X_1,Y_1$ are independent and both have the same distribution as $X:\Omega\to\mathbb R$.

(This must considered to be "knowledge" resting in our probability-backpack and also in the probability-backpack of others (e.g. our teachers) and does not have to be proved again and again.)

In that context it can be proved that $\mathsf{Cov}(f(X_1),g(X_1))\geq0$ the way you did.

And of course this also proves that $\mathsf{Cov}(f(X),g(X))\geq0$.

drhab
  • 153,781
  • 1
    Why is the last statement true?

    When we do coupling, i.e. create new random variables with same marginal on a new probability space, their correlation structures get affected. So, how can we extend the argument to the covariance of X,Y?

     – spandan madan Nov 08 '19 at 01:06
  • 1
    @spandanmadan Essential is that $\mathsf{Cov}(f(X),g(X))$ is completely determined by the distribution of $X$. For the created $X_1$ we proved that $\mathsf{Cov}(f(X_1),g(X_1))\geq0$. Then on base of the fact that $X$ and $X_1$ have the same distribution we are allowed to conclude that $\mathsf{Cov}(f(X),g(X))=\mathsf{Cov}(f(X_1),g(X_1))\geq0$. – drhab Nov 08 '19 at 07:39
  • 1
    So basically Cov(f(X),g(X)) is completely determined by the marginal of X? That makes sense. Good point! – spandan madan Nov 08 '19 at 23:13
  • 1
    @spandanmadan I am glad that you see the sense, but do not understand why you speak of "marginal of $X$". In this context $X$ is a random variable that has a distribution, and this distribution is decisive for the value of $\mathsf{Cov}(f(X),g(X))$. We normally speak of marginals if we are dealing with a joint distribution of more than one random variable. I do discern a distribution of $X$ but not a marginal of $X$. – drhab Nov 09 '19 at 09:09