Functions of random variables result, where does it come from

Question

I have learned that if one has two random variables, say $X$ and $Y$ and if $Y=g(x)$, then we have that density of r.v. $Y$ is:

$$f_Y(y) = f_X(g^{-1}(y))\left| \frac{d(g^{-1}(y))}{dx}\right|$$

This result is obtained by looking at two cases when the $g(x)$ is monotonically decreasing and monotonically increasing and differentiating w.r.t. to $x$ in both cases, for example, for monotonically increasing case:

$$F(y) = P(Y \leq y) = P(X\leq x) = \int_{-\infty}^x f_X(\hat{x})d\hat{x} = \int_{-\infty}^{g^{-1}(y)}f_X(x) dx$$

Now differentiating the above w.r.t. $x$ and using the fundamental theorem of calculus, one obtains the required result in the first line.

My question is the following. I have seen my lecturer use the following notation:

$$f_X(x)|dx| = f_Y(g(x))|dg(x)|$$

is this an equivalent statement? And can one simply integrate both sides to obtain cumulative distribution functions (the second question is really: how to treat the absolute values to obtain the cumulative distribution function(s)). Thanks!

Answer 1

The true roots of this formula lie in the change of variables using absolute value of Jacobian in calculus. Recall that in 2D case, if you transform a small square area $dS=dxdy$ into small area $dU=dudv$, then $$\int f(x,y)dxdy = \int h(u,v)\cdot \big|J(u,v)\big|dudv, \ \ \ \big|J(u,v)\big|=\bigg|\frac{\partial(x,y)}{\partial(u,v)}\bigg|$$ where $h(u,v) = f(x(u,v),y(u,v))$. Here, in 1D, $u = g(x)$, so $x(u) = g^{-1}(u)$ and the Jacobian is just a $\big|\frac{dg^{-1}(u)}{du}\big|$. We have

$$\int f_X(x)dx = \int f_X(g^{-1}(u))\cdot \big|\frac{dg^{-1}(u)}{du}\big|du = \int f_U(u)du. $$ In the last equality we can already drop the integral sign. Now lets begin with his notation. At first, this is inverse transition, i.e. $y = g(x)$. We stay in $\mathbb{R}$, so $dS$=$dU$=$dx$. Thus $f_X(x)$ we replace with $f_Y(g(x)) \leftrightarrow h(u) $ and the Jacobian $\big|\frac{dg(x)}{dx}\big|$:

$$\int f_X(x)dx = \int f_Y(g(x))\cdot \big|\frac{dg(x)}{dx}\big|dx. $$

What's left is omitting the integrals - Ok, as we are working with infinitely small area and we can forget about the limits, - and separating the derivatives. Here I really don't know why do you need the last thing (so I can make a mistake), this is indeed a first order differential equation with separation of variables $dx$ and $dy = d(g(x))$. I think, you can really integrate both parts to get the probability of some event $A$, using that Lebesgue measure is always positive $|dx|=dx$ :

$$\int_A f_Y(g(x))\cdot |d(g(x))| = \int_A f_Y(g(x))\cdot |g'(x)|\cdot |dx|=\int_A f_Y(g(x))\cdot |g'(x)|\cdot dx. $$

Hope that it would help you somehow.

Answer 2

I believe you should be differentiating with respect to $y$, not $x$, both in the first displayed equation, and in the subsequent derivation. You should also be stating the assumption that $g$ is a monotonic function. If it isn't, you need to be summing over different monotonic pieces of $g$ in your first displayed equation.

You haven't shown the derivation in the case $g$ monotonically decreasing, but the issue there is that, for $y=g(x)$, we have $$ F_Y(y)=P(Y\le y)=P(X\ge x)=1-F_X(x)=1-F_X(g^{-1}(y)). $$ Differentiating both sizes with respect to $y$ gives the same formula as in the case of $g$ monotonically increasing, except for a minus sign. This explains the origin of the absolute value.

In practice, to compute the cumulative distribution function $F_Y(y)$ by integrating with respect to $x$, you have to know whether $g$ is increasing or decreasing. If it is the former, then you integrate from $-\infty$ to $g(x)$; if the latter, you integrate from $g(x)$ to $+\infty$. For a nonmonotonic function, you might have to integrate over several intervals and add the results. So in practice, "treating the absolute value to obtain the cumulative distribution function" means knowing enough about the shape of $g$ to figure out which intervals to integrate over.

(Added) To answer your questions more directly:

1. The equation $$ f_Y(y)=f_X(g^{-1}(y))\left\lvert\frac{dg^{-1}(y)}{dy}\right\rvert $$ is equivalent to $f_Y(y)\,\lvert dy\rvert=f_X(g^{-1}(y))\,\lvert dg^{-1}(y)\rvert$, which is equivalent to $f_Y(g(x))\,\lvert dg(x)\rvert=f_X(x)\,\lvert dx\rvert$.
2. Yes, you can simply integrate both sides to obtain the cumulative distribution function $F_X(x)$; doing so will tell you how $F_X(x)$ and $F_Y(g(x))$ are related. For example, suppose that $y=g(x)=-x$. Then $$ F_X(x)=\int_{-\infty}^x f_X(\hat x)\,d\hat x=\int_{-\infty}^x f_Y(-\hat x)\,d\hat x=\int_{\infty}^{-x} f_Y(\hat y)\,(-d\hat y)=\int_{-x}^{\infty} f_Y(\hat y)\,d\hat y=1-F_Y(-x). $$ In this calculation, the first equality is the definition of $F_X(x)$; the second equality follows from $f_Y(g(\hat x))=f_Y(-\hat x)$ and $\lvert dg(\hat x)\rvert=\lvert -d\hat x\rvert=d\hat x$, since $d\hat x$ represents a small positive interval when integrating from $-\infty$ to $x$; the third equality follows from the change of variable $\hat y=-\hat x$; the fourth equality is obtained by reversing the limits of integration; the final equality follows from the definition $F_Y(-x)=\int_{-\infty}^{-x}f_Y(\hat y)\,d\hat y$ and the property that $\int_{-\infty}^{\infty}f_Y(\hat y)\,d\hat y=1$.