2

Consider the following problem, from Tijms's Understanding Probability:

A receiver gets as input a random signal that is represented by a discrete random variable $X$, where $X$ takes on the value +1 with probability $p$ and the value -1 with probability $1- p$. The output $Y$ is a continuous random variable which is equal to the input $X$ plus random noise, where the random noise has an $N (0, \sigma^2 )$ distribution. You can only observe the output. What is the conditional probability of $X = 1$ given the observed value $Y = y$?

This problem is interesting because it seems a mix of discrete and continuous random variables. My attempt.

For the probability that $Y\le y$, we have two possibilities: either $X=1$ or $X=-1$, and $P(Y\le y) = P(Y\le y|X=1)P(X=1) + P(Y\le y|X=-1)P(X=-1)$, so $$ P(Y\le y)=p\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s-1)^2}ds + (1-p)\int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12 (s+1)^2}ds. $$ Here I used the fact that: $$ P(Y\le y|X=1) = \int_{-\infty}^y \frac{1}{\sqrt{2\pi}}e^{-\frac12(s-1)^2}ds. $$ Then, we might use: $$ P(X=1|Y=y) = P(Y=y|X=1)\frac{P(Y= y)}{P(X=1)}, $$ however if I replace the terms $P(Y\le y)$ and $P(Y\le y|X=1)$ I derived earlier, first I get a complicated simple expression, and then I am puzzled by having something like $P(Y=y)$ in the context of a continuous random variable.

J. D.
  • 815

3 Answers3

2

I know this sounds like nonsense, but for conditioning on a continuous variable $Y$ having a particular value $Y=y$, you can simply substitute the pdf where you would normally have $P(Y=y)$ (whose value is technically $0$).

Indeed, your situation is exactly Bayes theorem with one discrete variable and one continuous variable.

I don't know enough of the underlying theory (presumably measure theory?) to explain why this works. Sorry!

antkam
  • 15,638
2

Consider a mixed joint probability distribution

$$P(X=x, Y=y) \approx P(X=x)P(Y \in y+\delta y|X=x) =\\ P(X=x|Y \in y + \delta y)P(Y \in y+ \delta y) $$

For $\delta y \rightarrow 0$, we can write:

$$P(X=x)f_Y(y|X=x) = P(X=x|Y = y)f_Y(y) $$

Then, $$P(X=1|Y = y) = \frac{pf_Y(y|X=1)}{f_Y(y)}, $$

where $f_Y(y|X=1) = p N(1,\sigma^2)$ and $f_Y(y) = p N(1,\sigma^2) + (1-p)N(-1,\sigma^2)$, and $N(\mu,\sigma^2)$ is a density of a normal r.v. with mean $\mu$ and variance $\sigma^2$.

dnqxt
  • 1,352
0

Following the example by dnqxt, I suspect that we can interpret: $$ P(Y=y) = \lim_{\delta \to 0}P(y-\delta \le Y\le y) =\lim_{\delta\to 0} \left(p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2} + (1-p)\frac{\delta}{\sqrt{2\pi}}e^{-\frac12(y+1)^2}\right), $$ and similarly, $$ P(X=1,Y=Y)=\lim_{\delta\to0}P(X=1, y-\delta\le Y\le y)=\lim_{\delta\to0}p\frac{\delta}{\sqrt{2\pi}}e^{-\frac12 (y-1)^2}. $$ As expected, $P(Y=y)$ and $P(X=1,Y=y)$ are zero when we take the limit, but computing the conditional probability before taking the limit gives: $$ P(X=1|Y=y) = \frac{pe^{-\frac12 (y-1)^2}}{pe^{-\frac12 (y-1)^2}+(1-p)e^{-\frac12 (y+1)^2}}. $$

J. D.
  • 815