2

Let $(\Omega, \mathcal{F}, P)$ be a probability space s.t. $\Omega = \{0,1\}$. Let $X_1: \Omega \rightarrow \{0,1\}$, $X_2: \Omega \rightarrow \{0,1\}$ be two random variables over $\Omega$ (i.e., we're in the context of a Bernouli trial with, say, a coin flip). In fact, we can let $X_1$ and $X_2$ be $id$ in this context. We can suppose further that they are $i.i.d.$

Question: When people write $X_1 + X_2$, what do they mean? In particular, what is the domain and range of $X_1 + X_2$? I'm assuming that its probability distribution forms a convolution of $X_1$ and $X_2$. As I see it, the options for the domain and range are as follows:

  1. $X_1 +X_2$ is a function from $\Omega = \{0, 1\}$ to $\{0, 2\}$ s.t. $(X_1 + X_2)(0) = 0 + 0 = 0$ and $X_1 \rightarrow X_2(1) = 1 + 1 = 0$.

  2. $X_1 + X_2$ is a function from $\{(0,0), (0,1), (1,0), (1,1)\}$ to $\{0, 1, 2\}$ s.t. $(X_1 + X_2)(i,k) = i + k$.

Notice how in (2) the domain of $X_1 + X_2$ is no longer equal to the domain of $X_1$ and $X_2$.

So, with those options laid out, is $X_1 + X_2$ (1), (2), or something else entirely?

user1770201
  • 5,361
  • 6
  • 44
  • 80

2 Answers2

1

I think your second answer is closer. Let $S = X_1+X_2$, then $S$ is certainly has a 2-dimensional domain, i.e. $S : \Omega^2 \to \{0,1,2\}$ with $$ S(\omega) = X_1(\omega_1) + X_2(\omega_2), \quad \forall \ \omega = (\omega_1, \omega_2) \in \Omega^2. $$

gt6989b
  • 54,930
  • Is this how function addition in general is defined? I.E. if $f$ and $g$ are functions on $\Omega$, then $f+g$ has domain $\Omega^2$? Or is this a special convention used in probability theory? – user1770201 Apr 12 '16 at 03:31
  • 1
    @user1770201 this is a general definition – gt6989b Apr 12 '16 at 03:32
  • I changed this to no longer be the accepted answer. It seems there is mounting evidence this is not how $+$ is defined. See Artem's response. I'd be curious if you have any opinion on this if you disagree though. – user1770201 Apr 12 '16 at 18:51
  • @user1770201 no contradiction, his last paragraph states in 5 lines what i wrote in 1 – gt6989b Apr 12 '16 at 19:59
  • The strange idea to define each random variable on its own specific probability space pops up regularly on the site. A few remarks: 1. Their proponents usually stick to their gun no matter what. 2. This construction is supported by no source. 3. This construction is actually in contradiction with every source on the subject. 4. More importantly, this construction makes impossible every simple operation on random variables one can imagine. Once all this is said, I would be interested to understand where the idea comes from. – Did Apr 13 '16 at 06:32
  • @Did i honestly don't understand how you can do it any other way -- the construction in the accepted answer uses product spaces as well, which is what i did here, and so do Artem Mavrin's remarks, and your comment seems to indicate that you agree. I fail to see the difference between that construction and the one in my answer. – gt6989b Apr 14 '16 at 15:05
  • @Did perhaps i see the difference -- you want to define $S(\omega) = X_1(\omega) + X_2(\omega)$ where all variables map from $\Omega^n$, but each $X_i$ only uses the $i$th coordinate. Why would this make impossible every simple operation? We just defined addition, you can define integration and scalar multiple and everything else analogously -- why does this cause a problem? Please understand, I am not arguing, just asking for clarification for my own education. Thanks – gt6989b Apr 14 '16 at 15:08
1

The sum of two functions $X_1,X_2:\Omega\to\mathbb{R}$ is defined pointwise by $(X_1+X_2)(\omega) = X_1(\omega) + X_2(\omega)$.

In your example, there are only four possible functions $X_1,X_2:\{0,1\}\to\{0,1\}$, and contrary to what you said, this prevents them from being independent unless one of them is constant (the non-constant cases are $X_2 = X_1$ or $X_2 = 1 - X_1$). To see this, suppose $X_1,X_2:\{0,1\}\to\{0,1\}$ are the functions $X_1(\omega) = X_2(\omega) = \omega$ for $\omega \in \{0, 1\}$ (the other non-constant case is similar). For $X_1$ and $X_2$ to be independent, we need $$ P(X_1 \in A, X_2 \in B) = P(X_1 \in A) P(X_2 \in B) $$ for all Borel subsets $A, B \subseteq \mathbb{R}$. This does not hold, since $X_1$ and $X_2$ are literally equal functions, so $$ P(X_1 = 0, X_2 = 1) = P(\emptyset) = 0 \neq \frac{1}{4} = P(X_1 = 0) P(X_2 = 1). $$ The problem is that the sample space $\{0,1\}$ is not rich enough to handle non-constant independent random variables. Intuitively, $\{0,1\}$ only knows about a single coin flip, not multiple coin flips.

One way to create independent Bernoulli random variables is to consider the product space $\{0,1\}\times\{0,1\}$ (with the corresponding product probability measure), where each component represents a separate coin flip. Now if $X_1$ represents the first coin flip and $X_2$ the second, we can define $X_1((\omega_1,\omega_2)) = \omega_1$ and $X_2((\omega_1,\omega_2)) = \omega_2$. Now $X_1$ and $X_2$ truly represent independent coin flips, and their sum $X_1+X_2$ coincides with the definition above: $$ (X_1 + X_2)((\omega_1, \omega_2)) = X_1((\omega_1,\omega_2)) + X_2((\omega_1,\omega_2)) = \omega_1 + \omega_2. $$

Artem Mavrin
  • 2,662
  • This contradicts the answer provided by gt6989b, who claimed that $+$ is defined differently. Just to be clear: are you saying that he is wrong? – user1770201 Apr 12 '16 at 06:54
  • @user1770201 yes, I know. I have never seen the definition they gave used anywhere. I have only ever seen the definition I gave. I am not claiming they are wrong (a definition can't really be wrong), just that I think it is non-standard. – Artem Mavrin Apr 12 '16 at 06:55
  • What do you mean by "this prevents them from being independent unless one of them is constant"? – user1770201 Apr 12 '16 at 08:04
  • Also, your answer seems to contradict the accepted answer for this thread as well: http://math.stackexchange.com/questions/584313/about-a-domain-of-random-variable-s-n-x-1x-2-x-n?rq=1 – user1770201 Apr 12 '16 at 08:05
  • @user1770201 I have edited the paragraph about independence to hopefully answer your question. Also, the answer you linked supports me, not contradicts me. The answer suggests to take a product space ${H,T}^n$ as the sample space instead of just ${H,T}$. – Artem Mavrin Apr 12 '16 at 15:53
  • For another question of clarification: in the context of the law of large numbers, we typically consider the sequence of sums $\bar{X}_n = \frac{1}{n} (X_1 + \ldots + X_n)$ as $n \rightarrow \infty$. But under your definition of $+$ (which, at this point, I'm pretty much convinced is standard), we have to constantly change the domain for all of sample spaces $X_1, \ldots, X_n$ for each increment of $n$ as it gets larger. That seems very awkward! Am I missing something? Or is this just the way things are? – user1770201 Apr 12 '16 at 17:36
  • @user1770201 One way to get infinitely many independent random variables at once (for, e.g., the Law of Large Numbers) is to take an infinite product space as your sample space. For example, if we want to apply the law of large numbers to arbitrarily many coin flips, we can let the sample space $\Omega$ be the set of all sequences $(\omega_n){n=1}^\infty$, where $\omega_n\in{0,1}$ for all $n$. The $i$th coin flip will be represented by the random variable $X_i:\Omega\to{0,1}$ given by $X_i((\omega_n){n=1}^\infty)=\omega_i$. (continued) – Artem Mavrin Apr 12 '16 at 18:18
  • @user1770201 (continued) Then you can define the sequence of sums $n^{-1}(X_1+\cdots+X_n)$ for every $n$, and each such sum will be a random variable whose domain is $\Omega$. Also, to make the random variables $(X_n)_{n=1}^\infty$ independent, you need to put a suitable probability measure on the product sample space. It's not obvious that this can be done, but you can do it by using Kolmogorov's Extension Theorem. – Artem Mavrin Apr 12 '16 at 18:20
  • yes, that is a great way to think about it. It's just very bizarre that i.e. nowhere on the wikipedia page for the law of large numbers (and other related, basic statistics pages) is it specified what the domain/sample space is, and what, exactly $+$ denotes. If the domain is truly meant to be something like $(\omega_n)^\infty_{n=1}$, it's odd that it isn't specified (you would think it would be, as that's at all obvious IMO). Anyway, thanks for your help on this thread! – user1770201 Apr 12 '16 at 18:49
  • Not bizarre at all once one realizes that to specify the sample space is a useless task performed only in some ill-advised starting probability curricula. In "real life", one knows by a general theorem that some probability spaces exist that allow to define every random variable of interest, and one leaves it unspecified. – Did Apr 13 '16 at 06:35