1

I'm reading Larry Wasserman's "All of statistics", and I've come across a definition I can't "unpack".

Specifically the text defines $f(x, y)$ to be a PDF for the random variables $(X, Y)$, if:

  1. $f(x, y) \geq 0 $ for all $(x, y)$
  2. $\int_{-\infty}^{+\infty}\int_{-\infty}^{+\infty}f(x, y)dxdy = 1 $ and
  3. For any set $A \subset \mathbb{R}\times\mathbb{R}, \mathbb{P}((X, Y) \in A) = \iint_{A}f(x, y)dxdy$.

Intuitively it all makes sense, but what exactly is $\mathbb{P}((X, Y) \in A)$?

Earlier in the text, we have $\mathbb{P}(X=x)$ defined to be $\mathbb{P}(X^{-1}(x))$, and this makes sense, since the pre-image of $X$ is the sample space.

This definition can be trivially extended to other arithmetic operators, i.e. $<$, $>$, $\leq$, etc and even set operations $\mathbb{P}(X \in A)$, as long as $A$ is a subset of $\mathbb{R}$.

I'm struggling to see how exactly this definition can be extended to $\mathbb{P}((X, Y) \in A)$. The text is not helpful in that regard.

Adam Kurkiewicz
  • 309
  • There is an implicit probability triplet $(\Omega,\mathcal{F},\mathbb{P})$ with $X$ and $Y$ being measurable functions from $(\Omega,\mathcal{F})$ to $(\mathbb{R},\mathcal{B})$. Then $$\mathbb{P}((X,Y)\in A)=\mathbb{P}({\omega:(X(\omega),Y(\omega))\in A}).$$ As you can see, there is usually a notational abuse where the symbol $\mathbb{P}$ is used differently in 2 places. – Kim Jong Un Nov 12 '17 at 12:58
  • is $\mathfrac{F}$ a probability measure ? The text doesn't introduce them (leaving them to more advanced texts). I'd prefer not to know about them at the time (I'm doing research in computational biology, not maths or statistics proper). The text defines $X$ to be a function from $\Omega → \mathbb{R}$ – Adam Kurkiewicz Nov 12 '17 at 13:03
  • $\mathcal F$ is a $\sigma$-algebra. In my answer I denoted it as $\mathcal A$. – drhab Nov 12 '17 at 13:07
  • $A\subset \mathbb{P}[(X,Y)\in A] $ doesn't really make sense. A is a set and the right term is a number in the reals. The probability of the random variables being in A. – Felix Benning Nov 12 '17 at 13:11
  • @FelixB. True, but I never said that it does. $\mathbb{P}((X, Y) \in A)$ will be a real number from the interval [0, 1]. – Adam Kurkiewicz Nov 12 '17 at 13:15
  • @AdamKurkiewicz but you wrote that in the definition. Are you sure you didn't misread? – Felix Benning Nov 12 '17 at 13:22
  • Ah indeed, that's my typo. Apologies. – Adam Kurkiewicz Nov 12 '17 at 13:26

2 Answers2

2

Assuming that you are working with a probability space $(\Omega,\mathcal A,\mathbb P)$, the expression $\mathbb P((X,Y)\in B)$ must actually be read as:$$\mathbb P(\{\omega\in\Omega\mid (X(\omega),Y(\omega))\in B\})$$

Also the set $\{\omega\in\Omega\mid (X(\omega),Y(\omega))\in B\}$ can be recognized as the preimage of $B$ under the function $\Omega\to\mathbb R^2$ prescribed by $\omega\mapsto(X(\omega),Y(\omega))$.

Here $B\subseteq\mathbb R^2$ is a Borel set, and $X,Y:\Omega\to\mathbb R$ are random variables.

Did
  • 284,245
drhab
  • 153,781
  • Are $X$ and $Y$ always defined on the same probability space? I thought there could be coming from different probability spaces, and that's why I struggled with producing a good disambiguation of the notational abuse? – Adam Kurkiewicz Nov 12 '17 at 13:17
  • In a context that "mentions" $(X,Y)$ they must come from the same probability space. Else $(X,Y)$ has no proper meaning. Also it is quite rare (and inconvenient) in probability theory to be working with two random variables that come from different spaces. If they are, then there is always a possibility to build a new underlying space that captures them both. – drhab Nov 12 '17 at 13:19
  • @drhab you might be able to make it work $X:\Omega_1 \rightarrow \mathbb{R}$, $Y:\Omega_2 \rightarrow \mathbb{R}$. Then you could define $\Omega:=\Omega_1 \times \Omega_2$ and then $(X,Y):\Omega\rightarrow\mathbb{R}$ With abuse of notation: $X:\Omega \rightarrow \mathbb{R}$ makes sense (just put a projection in front) and you basically have one omega again. So it doesn't really matter. They end up being equivalent in a way anyway. – Felix Benning Nov 12 '17 at 13:29
  • @FelixB. Yes, I know. Then $\Omega=\Omega_1\times\Omega_2$ is the underlying space that captures them both. But in practice you actually never start with the (inconvenient) $X_i:\Omega_i\to\mathbb R$. – drhab Nov 12 '17 at 13:30
  • 2
    And @AdamKurkiewicz that is why you generally just assume they are from the same Omega, as you have no idea what omega is anyway, and you can easily construct a new Omega which captures any two random variables in the way you want. – Felix Benning Nov 12 '17 at 13:34
  • @FelixB. Related question – drhab Nov 12 '17 at 13:37
  • Thanks guys, this has been tremendously helpful. What I'm specifically working with is a rare muscle disease called "Myotonic Dystrophy Type 1", MD1 for short, and every human (healthy or not) can be assigned a value of random variable $X$, which, in great simplification, measures how much MD1 they have. Technically $X$ is an integer, but we model it as a real number. Then $Y_{G}$ is something else, called "intensity of probeset $G$", which is again specific to a human. Until now I assumed that the sample space behind $X$ and $Y_{G}$ was different, but maybe not: it's the space of all humans. – Adam Kurkiewicz Nov 12 '17 at 13:40
  • Glad to help you with it. – drhab Nov 12 '17 at 13:43
2

$\mathbb{P}(X,Y)$ doesn't exist. $\mathbb{P}[(X,Y)\in A]$ is the probability of the two dimensional random variable (X,Y) landing in A. I am not quite sure why you don't understand $\mathbb{P}[(X,Y)\in A]$ if you understand $\mathbb{P}[X\in A]$.

So let us look at the one-dimensional case first. Say X is modelling a dice. So $\mathbb{P}(X=k)=1/6$ for $k\in \{1,...6\}$. I can now define $A=\{2,4,6\}$. Then $\mathbb{P}[X\in A]=\mathbb{P}(\text{X is even})$.

Or if you struggle with the formal definition: $\mathbb{P}[X\in A]=\mathbb{P}[X^{-1}(A)]=\mathbb{P}[\{\omega\in\Omega:X(\omega)\in\Omega\}]$.

Now look at the two dimensional case. Say (X,Y) are two dice. $\mathbb{P}[(X,Y)=(k,l)]=1/36$ for $k,l\in\{1,...,6\}$. I can now define $A=\{(k,l)\in\{1,...6\}^2:k+l=5\}$. Then $\mathbb{P}[(X,Y)\in A]=\mathbb{P}[\text{Sum of the two dice is 5}]$.

Or if you struggle with the formal definition: $\mathbb{P}[(X,Y)\in A]=\mathbb{P}[(X,Y)^{-1}(A)]=\mathbb{P}[\{\omega\in\Omega:(X(\omega),Y(\omega))\in\Omega\}]$.

Of course this is the discrete case. But similarly if you can think of a continuous case for the one dimensional variable it shouldn't be too hard to find a example for the two dimensional continuous case. And discrete models are usually easier to grasp.

Felix Benning
  • 2,889