5

In the most text book of advance probability theory, they always start from a probability space $(\Omega, F, P)$, and introduce the corresponding measure theory, then use measurable function to define a random variable. However, the probability space part is always vague. When we define a new random variable, we always directly define it by its density function or mass function. However, we cannot see the connection between probability space and random variable anymore.

My question is that can we always define a random variable from a probability space? For example, I want to define a Bernoulli distributed random variable. We can start from a prob. space that $\Omega=\{H,T\}$ and $P(H)=p, P(T)=1-p$, then define the measurable function between prob.space and $(R,B)$ as $X(T)=1$ and $X(H)=0$. Or a prob. space that $\Omega=[0,1]$, and $P$ is the Lebesgue measure, and $F$ the collection of Lebesgue measurable sets. Then define $X(\omega)=1$ if $\omega<0.5$ and so on.

Exponential distributed random variable is still OK, because the uniform distributed random variable can be defined by a identity function on the prob.space mentioned above, and we can apply the inverse function of the CDF of exponential distribution to define a exponential distributed random variable.

However, if we go further to define Gaussian random variable. Can we do similar procedure as above to define it? What can I image is that we can invoke CLT to define a Gaussian from a $\Omega =\{H,T\} $ or $\Omega=[0,1]$. But it is not very clear. Also, if we continue the thinking, how to construct a complicate stochastic process?

Xander Henderson
  • 32,453
ANuo
  • 141
  • 1
    Maybe Box and Muller algorithm also can help me to construct the Gaussian from the $[0,1]$ Leb meas. space. But it will go to bivariate normal first. Maybe that is also a problem. Anyway, the most important question is "can we always go back to a concrete prob. space when we try to define a random variable or random elements?" – ANuo Apr 04 '16 at 07:44
  • Or can we treat the Lebesgue measure space of $\Omega=[0,1]$ as the "atom" of the world of probability? Just like we have a bunch of Play-doh ($\Omega=[0,1]$), then we can use different mold (measurable function) to construct different shape (distribution) – ANuo Apr 04 '16 at 08:00
  • 3
    For a standard Gaussian, how about considering $\Omega=\mathbb{R}$ and $F=\mathcal{B}(\mathbb{R})$ and $P([a,b]) = \Phi(b)-\Phi(a)$ when $a \le b$? – Henry Apr 04 '16 at 08:54
  • @Henry Thank you for pointing this, I see it. When define a continuous random variable, we can define the prob measure by the distribution function (is called Lebesgue Stieltjes measue?) Then we always can start from the measure space (R, B(R)). – ANuo Apr 04 '16 at 09:13
  • @Henry Could you clarify what $F$ and $\mathcal{B}$ are there? – Learning stats by example Apr 21 '22 at 01:03
  • 1
    @Learningstatsbyexample $F$ is a sigma algebra of events in a probability space. $\mathcal{B}(\mathbb{R})$ is the Borel sigma algebra on the real numbers, i.e. the smallest sigma algebra that contains all the intervals on the real line plus complements and finite and countable unions and intersections – Henry Apr 21 '22 at 01:11

2 Answers2

2

$\def\ed{\stackrel{\text{def}}{=}}$ In one sense, the answer to your question is trivially "yes", because the very definition of the term "random variable" makes it a measurable function from the sample space of a probability space into some other measure space. Of course, this obviously doesn't really answer your question because you're using the term "random variable" in a somewhat looser sense, as being somehow specified by a probability density function or probability mass function.

To rephrase your question in more precise language, therefore, I take it to be "Given a probability density function or probability mass function, does there always exist a random variable on the sample space of some probability space whose density or mass function is the given one?" The answer to that question is "yes". In fact you can always take the probability space to be $\ \big((0,1), \mathscr{B}_{(0,1)},\ell\big)\ ,$ where $\ \mathscr{B}_{(0,1)}\ $ is the collection of Borel sets of the open unit interval, and $\ \ell\ $ is Lebesgue measure—i.e. the uniform distribution on the unit interval. Alternatively, it's always possible, and usually more convenient, to choose the sample space to be the set $\ \mathbb{R}\ $ of real numbers and the random variable to be the identity function on $\ \mathbb{R}\ .$ You can always use the given density function or mass function to define an appropriate probability measure on the Borel subsets of $\ \mathbb{R}\ $ so that this will work.

Suppose you're given a density function $\ \varphi:\mathbb{R}\rightarrow\mathbb{R}\ .$ If $\ X\ $ is the identity function on $\ \mathbb{R}\ $, and you define a probability measure $\ P\ $ on the Borel subsets $\ \mathscr{B}_{\mathbb{R}}\ $ of $\ \mathbb{R}\ $ by $$ P(A)\ed\int_A\varphi(x)\,dx $$ for $\ A\in\mathscr{B}_{\mathbb{R}}\ ,$ then \begin{align} P\big(X^{-1}(A)\big)&=P(A)\\ &=\int_A\varphi(x)\,dx\ , \end{align} so $\ \varphi\ $ is the density function of $\ X\ .$

Now suppose you're given a probability mass function $\ \mu:\mathscr{D}_\mu\rightarrow[0,1]\ ,$ where $\ \mathscr{D}_\mu\ $ is some countable subset of $\ \mathbb{R}\ .$ If you define a probability measure $\ P\ $ on the Borel subsets $\ \mathscr{B}_{\mathbb{R}}\ $ of $\ \mathbb{R}\ $ by $$ P(A)\ed\cases{0&if $\ \mathscr{D}_\mu\cap A=\varnothing$\\ \displaystyle\sum_{x\in \mathscr{D}_\mu\cap A}\mu(x)&otherwise} $$ for $\ A\in\mathscr{B}_{\mathbb{R}}\ ,$ then for $\ \mathscr{D}_\mu\cap A\ne\varnothing\ $ you have \begin{align} P\big(X^{-1}(A)\big)&=P(A)\\ &=\sum_{x\in \mathscr{D}_\mu\cap A}\mu(x)\\ &=\sum_{x\in \mathscr{D}_\mu\cap A}\mu(X^{-1}(x))\ , \end{align} so $\ \mu\ $ is the probability mass function of $\ X\ .$

To show that you can always take the probability space to be $\ \big((0,1), \mathscr{B}_{(0,1)},\ell\big)\ $ it's easier to work with the cumulative distribution function. The cumulative distribution function $\ F_\varphi\ $ corresponding to a density function $\ \varphi\ $ is given by $$ F_\varphi(x)\ed\int_{-\infty}^x\varphi(y)\,dy\ ,\tag{1}\label{e1} $$ and the cumulative distribution function $\ F_\mu\ $ corresponding to a probability mass function $\ \mu:\mathscr{D}_\mu\rightarrow[0,1]\ $ is given by $$ F_\mu(x)\ed\sum_{d\in \mathscr{D}_\mu: d\le x}\mu(d)\ .\tag{2}\label{e2} $$ In general, the cumulative distribution function $\ F_Y \ $ of a random variable $\ Y:\Omega\rightarrow\mathbb{R}\ $ on a probability space $\ (\Omega,\mathcal{F},P)\ $ is defined by $$ F_Y(x)\ed P\big(Y^{-1}((-\infty,x])\big)\ , $$ and is a more general concept than than probability density functions or probability mass functions, in that there exist cumulative distribution functions that cannot be obtained from density functions or probability mass functions via the equations \eqref{e1} or \eqref{e2}.

It follows from the properties of probability spaces that the cumulative distribution function of any random variable is an increasing, right-continuous function $\ F:\mathbb{R}\rightarrow[0,1]\ $ with $\ \lim_\limits{x\rightarrow-\infty}F(x)=0\ $ and $\ \lim_\limits{x\rightarrow\infty}F(x)=1\ .$ Conversely, an answer to your question is provided by the following:

Theorem. If $\ F:\mathbb{R}\rightarrow[0,1]\ $ with is an increasing, right-continuous function with $\ \lim_\limits{x\rightarrow-\infty}F(x)=0\ $ and $\ \lim_\limits{x\rightarrow\infty}F(x)=1\ ,$ then $\ F\ $ is the cumulative distribution function of a random variable defined on the probability space $\ \big((0,1), \mathscr{B}_{(0,1)},\ell\big)\ .$

Proof (outline). Let $$ X(\alpha)\ed\inf\{x\in\mathbb{R}\,|\,\alpha\le F(x)\,\} $$ for $\ \alpha\in(0,1)\ .$ It's not difficult to show that \begin{align} \big(0,F(x)\big)&\subseteq X^{-1}\big((-\infty,x\,]\big)\\ &\ed\{\alpha\in(0,1)\,|\,X(\alpha)\le x\,\}\\ &\subseteq\big(0,F(x)\big] \end{align} and therefore that $$ \ell\big(X^{-1}\big((-\infty,x\,]\big)\big)=F(x)\ . $$ Thus, $\ X\ $ is a random variable defined on the probability space $\ \big((0,1), \mathscr{B}_{(0,1)},\ell\big)\ $ whose cumulative distribution function is $\ F\ .$

lonza leggiera
  • 31,984
-1

I am not working in probability theory, but the answer, I believe, is the following. The random variable $X:\Omega\to\mathbb R$ on a probability space $(\Omega, F, P)$ is called a Gaussian random variable if the image measure $P\circ X^{-1}$ (aka pushforward measure) is Gaussian, that is, if $P(X^{-1}(A)) = \int_A \varphi(z)dz$ where $\varphi$ is the Gaussian probability density function.

Alexander Kurz
  • 139