how to derive the mean and variance of a Gaussian Random variable?

Question

How do we go about deriving the values of mean and variance of a Gaussian Random Variable $X$ given its probability density function ?

Answer 1

UPDATE 21-03-2017
A much faster way is to differentiate both sides of

$$\int_{-\infty}^{\infty}\frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx=1$$ with respect to the two parameters $\mu$ and $\sigma^2$ (RHS will then be zero).

---

The Gaussian pdf is defined as $$f_X(x) =\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}$$

where $\mu$ and $\sigma$ are two parameters, with $\sigma >0$. By definition of the mean we have $$E(X) = \int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$ which using integral properties can be written as

$$E(X) = \int_{-\infty}^{\infty}(x+\mu)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$=\int_{-\infty}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \;+\; \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx \qquad [1]$$

The first integral, call it $I_1$ equals zero, because we integrate an odd function (the product of an odd times an even function gives an odd function), over an interval centered at zero. Being pedantic, we have using additivity

$$I_1 = \int_{-\infty}^0x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$ Swapping the integration limits in the first we have

$$I_1 = -\int_{0}^{-\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

and using again integral properties we have

$$I_1 = \int_{0}^{\infty}(-x)\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(-x)^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx$$

$$\Rightarrow I_1 = -\int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx + \int_{0}^{\infty}x\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx = 0\qquad [2]$$

So we have that

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

Multiply by $\sigma \sqrt2$ to obtain

$$E(X) = \int_{-\infty}^{\infty}\mu\frac{1}{\sqrt{\pi}}e^{-x^2} dx = \mu\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx$$

...the last term because the integrand is an even function.

Now $$\frac{2}{\sqrt{\pi}}\int_{0}^{\infty}e^{-x^2} dx = \lim_{t\rightarrow \infty}\frac{2}{\sqrt{\pi}}\int_{0}^{t}e^{-x^2} dx = \lim_{t\rightarrow \infty} \text{erf}(t) = 1$$

where "erf" is the error function. So we end up with $$E(X) = \mu$$ i.e. that the parameter $\mu$ is the mean of the distribution.

VARIANCE
We have

$$\text {Var}(X) = \int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx$$

Applying the same tricks as before we have

$$\int_{-\infty}^{\infty}(x-\mu)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}dx = \int_{-\infty}^{\infty}x^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{x^2}{2\sigma^2}\right\}dx $$

$$=\sigma \sqrt2\int_{-\infty}^{\infty}(\sigma \sqrt2x)^2\frac{1}{\sigma\sqrt{2\pi}}\exp\left\{-\frac{(\sigma \sqrt2x)^2}{2\sigma^2}\right\}dx = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}x^2e^{-x^2}dx$$

Define $t=x^2\Rightarrow x= \sqrt t$ and $dt = 2xdx = 2\sqrt tdx \Rightarrow dx = (2\sqrt t)^{-1}dt$. Substituting

$$V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\int_{0}^{\infty}(\sqrt t)^2(2\sqrt t)^{-1}e^{-t}dt = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \int_{0}^{\infty}t^{\frac 32 -1}e^{-t}dt= \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \Gamma\left(\frac 32\right)$$

$$\Rightarrow V(X) = \sigma^2\frac{4}{\sqrt{\pi}}\frac 12 \frac {\sqrt \pi}{2} = \sigma^2$$

where $\Gamma()$ is the Gamma function. So the parameter $\sigma$ is the square-root of the variance, i.e. the standard deviation.

Answer 2

If $x\mapsto f(x)$ is the density function of a random variable $X$ with expected value $0$ and variance $1$, then $x\mapsto \frac1\sigma f\left(\frac{x-\mu}{\sigma}\right)$ is the density function of $\mu+\sigma X$, and thus of a random variable with expected value $\mu$ and variance $\sigma^2$.

That can be shown by thinking about the substitution $u = \dfrac{x-\mu}{\sigma}$ and $du=\dfrac{dx}\sigma$.

Therefore the problem reduces to this: How do we show that $$ x\mapsto \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right) $$ is the density of a distribution with expectation $0$ and variance $1$?

If you know that the expectation of a distribution with density $f$ is $\int_{-\infty}^\infty xf(x)\,dx$, then you know that you need to find $$ \int_{-\infty}^\infty x \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx. $$

Since this is the integral of an odd function over an interval that is symmetric about $0$, it must be equal to $0$ unless the positive and negative parts both diverge to $\infty$. The positive part is a constant times $$ \int_0^\infty x \exp\left(\frac{-x^2}{2}\right)\,dx = \int_0^\infty \exp(u)\,du, $$ where $u=-x^2/2$ so $du=x\,dx$. Clearly this is finite, and the negative part can be treated the same way.

The variance is \begin{align} & \int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2 \int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\int_0^\infty x^2 \frac{1}{\sqrt{2\pi}}\exp\left(\frac{-x^2}{2}\right)\,dx \\[8pt] = {} & 2\frac{1}{\sqrt{2\pi}} \int_0^\infty \Big(x\Big)\Big(x\exp\left(\frac{-x^2}{2}\right)\,dx\Big) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int x\,dv \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left( xv-\int v\,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \left(\left[-x\exp\left(\frac{-x^2}{2}\right)\right]_0^\infty -\int_0^\infty -\exp\left(\frac{-x^2}{2}\right) \,dx \right) \\[8pt] = {} & \frac{2}{\sqrt{2\pi}} \int_0^\infty \exp\left(\frac{-x^2}{2}\right) \,dx. \end{align}

You know that this is equal to $1$ if you know how the normalizing constant in the density function was found.

Answer 3

$$E(X)=E(x - \mu) + \mu$$ So we're done by showing that $E(x - \mu)=0$

$$E(x- \mu)= \int_{ -\infty }^\infty \frac{(x-\mu)}{\sqrt{2\pi}\sigma}\cdot e^{-\frac {(x-\mu)^2}{2\sigma^2}}\,dx$$

Using the subsitiution $z=\frac{x-\mu}{\sigma}$ we get

$$E(x- \mu)= \frac {\sigma}{\sqrt{2\pi}} \int_{ -\infty }^\infty z \cdot e^{z^2/2}\,dz$$ The function $f(z) = z\cdot e^{z^2/2}$ is obviously odd. Which makes $$\lim_{a \to \infty }\int_{ -a }^a z \cdot e^{z^2/2}\,dz=0 $$ and thus we're left with $$E(X)= \mu$$

Answer 4

Another way that might be easier to conceptualize:

As defined earlier, ()= $\int_{-∞}^∞ xf(x)dx$

To make this easier to type out, I will call $\mu$ 'm' and $\sigma$ 's'

f(x)= $\frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2}}$}. So, putting in the full function for f(x) will yield

E(x)= $\int_{-∞}^∞ x \frac{1}{\sqrt{(2πs^2)}}$exp{ $\frac{-(x-m)^2}{(\sqrt{2s^2} }dx$. Pretty gross to look at.

Looking at the exponent that e is being raised to, it seems like an ideal candidate for a u-subsitution. I will set u= $\frac {x-m}{s\sqrt2}$. That means that du= $\frac{dx}{s\sqrt2}$, or in other words dx= du•$s\sqrt2$.

Let's simplify the integral as much as possible using what we now know. Recalling the term $\frac{1}{\sqrt{(2πs^2)}}$, we shall rewrite it as $\frac{1}{s\sqrt{(2π)}}$.

E(x)= $\int_{-∞}^∞ x \frac{1}{s\sqrt{(2π)}}$ exp {$-u^2$}du•$s\sqrt2$.

The $s\sqrt2$'s from the first and last terms will cancel, leaving a constant of $\frac{1}{\sqrtπ}$. Let's put this in front of the integral for now to make this look a little better.

E(x)= $\frac{1}{\sqrtπ}$$\int_{-∞}^∞ x $ exp {$-u^2$}du.

The derivative of $\int_{-∞}^∞ $ exp {$-u^2$}du is in the form of a Gaussian integral. I'm not going to pretend that I can easily derive it (multivariable calc???) but it is widely known that the value of this integral is $\sqrtπ$.

We can use this knowledge to perform integration by parts to determine the value of the integral. I like to use u and v for this, but this u is not the same as the u from earlier. I hope this still makes sense.

$\int{udv}$=uv-$\int{ vdu}$

Set u=x. This means u'= 1dx. In terms of dx from earlier, u'= $s\sqrt2$ du. v'= $e^{(-u^2)}$du, and so by definition v= $\sqrtπ$.

uv= x$\sqrtπ$ and vdu=$\sqrtπ s\sqrt2 du$.

The overall integral is now equal to $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\int_{-∞}^∞ \sqrtπ s\sqrt2 du$] evaluated from ∞ to -∞.

Antideriving $\int vdu$ produces $s\sqrt2π$u. Recall our earlier value of u, u= $\frac {x-m}{s\sqrt2}$.

E(x)= $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $s\sqrt2π$$\frac{x-m}{s\sqrt2}$] or $\frac{1}{\sqrtπ}$ [x$\sqrtπ$- $\sqrtπ$(x-m)] from ∞ to -∞.

Within the brackets, the terms being evaluated from ∞ to -∞ are:

x$\sqrtπ$-x$\sqrtπ$ + m$\sqrtπ$= m$\sqrtπ$.

E(x)= $\frac{1}{\sqrtπ}$ • m$\sqrtπ$ = m, or the mean, just like the definition of the expected value says.