Construct a random variable and probability space for a given distribution

Question

Given a set of real valued points $C:=\{x_i\colon\ i\in I\}$ where $I$ is a countable index set, I want to construct a probability space $(\Omega, \mathcal{A}, P)$ and a random variable $X\colon \Omega\rightarrow C$ such that $$P[X=x_i]=p_i$$ with $p_i\ge 0 ,\forall i\in I$ and $\sum_{i\in I} p_i=1$.

Here are my solutions:

1:

Set $(\Omega, \mathcal{A}, P)=(C,2^C, \mu)$ where $\mu(A):=\sum_{x_i\in A}p_i$ for a $A\in 2^C$. Then just pick $X=id$ and since the identity is always measurable we are done.

2:

Set $(\Omega,\mathcal{A},P)= (N|_I, \mathcal{B}(\mathbb{N}), \mu)$ with $\mu(A):=\sum_{i\in I} p_i\delta_{\{i\}}$ for $A\in\mathcal{B}(\mathbb{N)}$. I then define $X(i)=x_i$.

Are these examples correct? Is there a more systematic way to solve this problem (also for the continuous case)?

In my book I saw a theorem, that for every distribution function $F$ there is always a random variable and a probability space, such that $F_X=F$, meaning there is always a random variable which has the given distribution. Now the correspondence principle states that I can find a unique probability measure for every distribution function and vice versa. Do these two theorems combined give me the assurance, that for every probability distribution $P$ I can always find a random variable $X$ such that $P=P_X$?

Answer 1

Yes, your constructions make perfect sense and this is in fact the systematic way to do it. And it is very easy to adjust to a more general case, where the distribution is not necessarily discrete.

Consider a probability distribution $\mu$ on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$. Let $(\Omega, \mathcal{A}, P) = (\mathbb{R},\mathcal{B}(\mathbb{R}),\mu)$ and let $X := id$. It is quite easy to see, that $P_X = \mu$, since $P(X\in A) = P(X^{-1}(A))= P(A) = \mu(A)$.

We could even generalize to random vectors. So let's say we want to construct a probability space, where $X\sim \mu$ and $Y\sim \nu$ and $X,Y$ are independent. Then we could choose $(\Omega, \mathcal{A}, P) = (\mathbb{R^2},\mathcal{B}(\mathbb{R^2}),\mu \otimes \nu)$, where $\mu \otimes \nu$ is the product measure. Here we let $X(x,y) = x$ and $Y(x,y)=y$ and it is easy to check, that $P_{(X,Y)} = \mu \otimes \nu$, which shows that $X$ and $Y$ are independent with distribution $\mu$ and $\nu$.

Much more generally the construction can be extended to infinitely many (even uncountably many) random variables. This result is known as Kolmogorovs Consistency theorem: https://en.wikipedia.org/wiki/Kolmogorov_extension_theorem

For example the theorem guarantees the existence of sequences $X_1,X_2,X_3,...$ of $i.i.d.$ variables with some distribution $\mu$.

Answer 2

I think that your examples are okay but a bit far-fetched.

---

A random variable $X$ is a function that has $\mathbb R$ as codomain and is measurable wrt the $\sigma$-algebra $\mathcal B$ of Borel subsets of $\mathbb R$.

So the most evident/natural way here is to go for probability space $(\mathbb R,\mathcal B, P)$ where $P$ denotes the distribution involved and $X:\mathbb R\to\mathbb R$ is prescribed as identity $x\mapsto x$.

This works for every random variable $X$ and evidently we have $P_X=P$ and the answer on your last question is: "yes".

---

An alternative is the space $((0,1),\mathcal B(0,1),\lambda)$ where $\mathcal B(0,1)$ denotes the $\sigma$-algebra of Borel subsets of interval $(0,1)$ and $\lambda$ stands for the Lebesgue measure restricted to $\mathcal B(0,1)$.

For every random variable $X$ we have the Borel-measurable function $\Phi_X:(0,1)\to\mathbb R$ prescribed by:$$u\mapsto\inf(\{x\in\mathbb R\mid u\leq F_X(x)\}$$It is a sort of inverse of the CDF $F_X$ and it can be shown that random variable $\Phi_X$ has the same distribution as $X$ on base of:$$u\leq F_X(x)\iff \Phi_X(u)\leq x$$or equivalently $$\{\Phi_X\leq x\}=(0,F_X(x)]$$

---