1

Suppose $X$ is a random variable in $\mathbb{R}^n$ and $Y = f(X)$ where $f : \mathbb{R}^n \to \mathbb{R}^n$. $Y$ is a random variable as well and the probability distributions for $X$ and $Y$ are related by

$$ p_X(x) = \frac{p_{Y}(y)}{\left| det \frac{\partial(y_1,\ldots,y_n)}{\partial(x_1,\ldots,x_n)} \right|} $$

Is there a similar result when $m \neq n$? more specifically $m < n$ (strictly less)?

user8469759
  • 5,591
  • 4
  • 20
  • 54
  • 1
    I suspect $p_X(x)$ denotes a PDF for $X$. However if $f:\mathbb R^m\to\mathbb R^n$ where $m<n$ then generally $f(X)$ will not have a PDF. For instance let $m=1$ and $n=2$ and $f$ is prescribed by $x\mapsto(x,x)$. Then $P(f(X)\in\Delta)=1$ where $\Delta={(x,x)\mid x\in\mathbb R}$. But $\lambda(\Delta)=0$ so a PDF wrt Lebesguemeasure on $\mathbb R^2$ cannot exist. The problem arising is that $f(\mathbb R^{m})$ is a null-subset of $\mathbb R^n$. – drhab May 23 '19 at 11:16
  • But then we can't associate any pdf to a regular surface in R3. Is that the case? It seems a bit odd because there are sampling algorithm for surfaces anyway. – user8469759 May 23 '19 at 11:19
  • @user8469759 The point is that if the support of the random variable in $\mathbb R^3$ is not 3-dimensional then there is not a density in the usual sense. Yes you can still sample from the surface, but then you can sample from a discrete distribution in $\mathbb R$ without there being a density – Henry May 23 '19 at 11:23
  • There is some way out: PDF's not wrt Lebesguemeasures but wrt "local Lebesgue measures" on linear varieties of regular surfaces. Once I met them but that is a very long time ago, and I cannot be of any further help in this. – drhab May 23 '19 at 11:24
  • Not all distributions have density. The simplest example is discrete distribution. We usually work with either discrete (=with countable support) distributions or with absolutely continuous (=having density), but there are also singular (=any point has zero measure, but support has zero measure too) (and any measure can be decomposed in sum of discrete, continuous and singular). Singular components are quite counter-intuitive on $\mathbb R$, but in higher dimensions they are common - for example, distribution on some surface. – mihaild May 23 '19 at 11:27
  • @mihaild Not all distributions have density if we are only looking at densities wrt Lebesgue measures. But if we leave out that restriction then all distributions have density (e.g. wrt the probability measure itself). A discrete random variable has a density wrt a counting measure. – drhab May 23 '19 at 11:30
  • the other way around make more sense, that is, for $m>n$, changing the whole $\Bbb R^m$ by an $n$-dimensional submanifold: using an atlas to measure some probability in some submanifold of $\Bbb R^m$ –  May 23 '19 at 11:31
  • One small repair: I commented "...on linear varieties of regular surfaces.." Typo. I meant "...on linear varieties or regular surfaces.." – drhab May 23 '19 at 11:36
  • Sorry but intuitively... If I sample the $\sigma : U \subset \mathbb{R}^2 \to \mathcal{S} \subset \mathcal{R}^3$ is a parametrization of a regular surface, and I sample $(u,v)$ pairs from $U$ it seems clear to me that $\sigma(u,v)$ will be biased in $\mathcal{S}$, I find a bit strange we can't potentially define any distribution on $\mathcal{S}$. – user8469759 May 23 '19 at 12:22
  • Probably also worth to say that parameterization of regular surfaces are bijectives as well, this might help. – user8469759 May 23 '19 at 12:27

2 Answers2

2

If $m < n$ and $f$ is everywhere differentiable, then measure of $f(\mathbb R ^ m)$ is zero (see, for example, this answer - we can extend $f$ to $\mathbb R^n$ to apply it directly by using $g(x_1, \ldots, x_n) = f(x_1, \ldots, x_m)$ - then $\det D g$ is zero everywhere). So support of $f(X)$ has zero measure and thus $f(X)$ doesn't have density.

mihaild
  • 17,674
  • 1
    I agree on the technical aspects, but I also think this answer is too pessimistic. Can't we choose a density function on $\mathrm{img}(f)$ instead of using all of $\mathbb{R}^n$. – goblin GONE May 23 '19 at 11:24
  • 1
    @goblin what you are suggesting corresponds with my last comment on the question. – drhab May 23 '19 at 11:26
  • @goblin, you could for example go from $\mathbb R^2$ to the surface of a sphere in $\mathbb R^3$ but you may have to reparameterise down to two parameters and you really have $f: \mathbb R^2 \to S^2$ – Henry May 23 '19 at 11:30
0

I don't know the answer, so this is just a long comment.

If I understand correctly, your formula only works if $f$ is bijective. I also think we need for $f$ be (continuously?) differentiable with non-zero derivative everywhere. If someone could comment on these requirements, that would be great.

We could try firstly weakening this to allow $f$ to be a surjective continuously differentiable function. In this case, we expect that $p_Y(y)$ should be a sum over preimages of $y$. Like so:

$$p_{f(X)}(y) = \sum_{x \in f^{-1}(y)} p_X(x) \cdot |\det f'(x)|$$

For a function $f : \mathbb{R}^2 \rightarrow \mathbb{R}$, I expect that the appropriate variant on this would usually be some kind of an integral. Something like $$p_{f(X)}(y) = \int_{x \in f^{-1}(y)} p_X(x) \cdot |\det f'(x)|$$

with possibly some further subtleties involved.

Another direction this might be generalized is to allow $f$ to be injective but not necessarily surjective. This is to cover cases like $f : \mathbb{R} \rightarrow \mathbb{R}^2$. The notion of a Hausdorff measure seems relevant. For example, it might be possible to get a density function for $f(X)$ not with respect to $H^2_{\mathbb{R}^2}$ (the Lebesgue measure) but with respect to $H^1_{\mathbb{R}^2}$. In the comments, drhab suggests a different and more technical proposal of using "local Lebesgue measures". I'm not qualified to comment on such things, unfortunately, but that might be worth reading about.

goblin GONE
  • 69,385