Double conditional probability

Question

Is it possible to compute $P(X\mid Y,Z)$ by calculating $P(X\mid Y)$ given the probability $P(\cdot\mid Z)$? Similarly, is it possible to get at the density $f_{X\mid Y,Z}$ by calculating the desity $f_{X\mid Y}$ given $P(\cdot\mid Z)$?

More precisely, let $(\Omega,\mathcal{A},P)$ be a probability space and let $X,Y,Z$ be random variables. Consider the conditional probability induced on $\mathcal{A}$ by conditioning on $Z$: $P(\cdot\mid Z=z)$. Suppose for each $z$ we calculate the conditional distribution $P(X\mid Y=y)$ in the modifed probability space $(\Omega,\mathcal{A},P(\cdot\mid Z))$. Is the resulting function of $(y,z)$ equal to the conditional distribution $P(X\mid Y=y, Z=z)$?

Suppose for each $z$ the conditional density $f_{X\mid Y}$ exists given the modified probability space described above. Is the resulting function of $(y,z)$ equal to the conditional density $f_{X\mid Y,Z}$?

Answer 1

For every $z$, let $Q_z=P(\ \mid Z=z)$, your question is whether for every $(x,y)$, $$ Q_z(X=x\mid Y=y)=P(X=x\mid Y=y,Z=z). $$ The answer is "yes", since, by definition, $$ Q_z(X=x\mid Y=y)=\frac{Q_z(X=x,Y=y)}{Q_z(Y=y)}=\frac{P(X=x,Y=y\mid Z=z)}{P(Y=y\mid Z=z)}, $$ that is, $$ Q_z(X=x\mid Y=y)=\frac{P(X=x,Y=y, Z=z)}{P(Y=y, Z=z)}=P(X=x\mid Y=y,Z=z). $$ In particular, if, for every $z$, $f_{X\mid Y}^{(z)}$ is the density of $X$ conditionally on $Y$ with respect to $Q_z$, then $f_{X\mid Y}^{(Z)}$ is the density of $X$ conditionally on $(Y,Z)$.

Answer 2

$P(X\cap Y\cap Z) = P(X|Y\cap Z)P(Y\cap Z) = P(X|Y\cap Z)P(Y|Z)P(Z)$

So

$P(X|Y\cap Z) = P(X\cap Y\cap Z)/(P(Y|Z)P(Z))$

Can also rearrange to get

$P(X\cap Y\cap Z) = P(Z|X\cap Y)P(X|Y)P(Y)$

So there's your $P(X|Y)$. Don't know if that helps.

Answer 3

The answer to both questions is: "Yes".

A Rigorous Statement of the Results to be Proved

Let $S=\left(\Omega_0,\mathcal{A}_0,P\right)$ be a probability space, let $\left(\Omega_i,\mathcal{A}_i\right)$ be measurable spaces for $i=1,2,3$ and let $X_i$ be $\left(\mathcal{A}_0/\mathcal{A}_i\right)$-measurable functions, respectively. Denote by $P_{X_i}$ $X_i$'s distribution function and by $P_{X_i,X_j}$ $\left(X_i, X_j\right)$'s distribution function ($i,j=1,2,3$) and let $P_*:\Omega_1\times\sigma\left(X_2,X_3\right)\rightarrow\left[0,1\right]$ be a regular version of the conditional distribution $P\left(\left(X_2,X_3\right)\in B_{2,3}\mid X_1=\omega_1\right)$. For every $\omega_1\in\Omega_1$ denote by $P_{\omega_1}$ the probability measure $P_{\omega_1}\left(B_{2,3}\right):=P_*\left(\omega_1,B_{2,3}\right)$.

1. Suppose $Q:\left(\Omega_1\times\Omega_2\right)\times\mathcal{A}_3\rightarrow\left[0,1\right]$ is a function such that for all $B_3\in\mathcal{A}_3$, $Q\left(\cdot,B_3\right)$ is $\left(\mathcal{A}_1\otimes\mathcal{A}_2/\mathfrak{B}\right)$-measurable ($\mathfrak{B}$ being the standard Borel field on the real line). For every $\omega_1\in\Omega_1$ denote by $Q_{\omega_1}$ the function

   $$Q_{\omega_1}:\Omega_2\times\mathcal{A}3\rightarrow\left[0,1\right],\space\space Q{\omega_1}\left(\omega_2,B_3\right):=Q\left(\left(\omega_1,\omega_2\right),B_3\right)$$

   and suppose that for every $\omega_1$, $Q_{\omega_1}$ is a version of the conditional distribution $P_{\omega_1}\left(X_3\in B_3\mid X_2=\omega_2\right)$, i.e. the distribution of $X_3$ conditional on $X_2$ given that the underlying probability space is $\left(\Omega_0, \sigma\left(X_2, X_3\right), P_{\omega_1}\right)$.

   Then $Q$ is a version of the conditional distribution $P\left(X_3\in B_3\mid\left(X_1,X_2\right)=\left(\omega_1,\omega_2\right)\right)$.

2. Let $\nu:\mathcal{A}_3\rightarrow\left[0,1\right]$ be some probability measure on $\mathcal{A}_3$. For every $\omega_1\in\Omega_1$ denote by $P_{X_2}^{\left(\omega_1\right)}$ the distribution of $X_2$ given the underlying probability $P_{\omega_1}$.

   Suppose $f:\left(\Omega_1\times\Omega_2\right)\times\Omega_3\rightarrow\left[0,\infty\right)$ is $\left(\left(\mathcal{A}_1\otimes\mathcal{A}_2\right)\otimes\mathcal{A}_3/\mathfrak{B}\right)$-measurable. For every $\omega_1\in\Omega_1$ denote by $f_{\omega_1}$ the function

   $$f_{\omega_1}:\Omega_2\times\Omega_3\rightarrow\left[0,\infty\right),\space\space f_{\omega_1}\left(\omega_2,\omega_3\right):=f\left(\left(\omega_1,\omega_2\right),\omega_3\right)$$

   and suppose that for every $\omega_1\in\Omega_1$, $f_{\omega_1}$ is a $\left(P_{X_2}^{\left(\omega_1\right)}\otimes\nu\right)$-density of $\left(X_2,X_3\right)$ given the underlying probability $P_{\omega_1}$.

   Then $f$ is the $\left(P_{X_1,X_2}\otimes\nu\right)$-density of $\left(\left(X_1,X_2\right),X_3\right)$ given the underlying probability $P$.

   Comment Note that a conditional density $f_{X_3\mid X_2}$ w.r.t. $\nu$ is simply a density $f_{X_2,X_3}$ w.r.t. $P_{X_2}\otimes\nu$.

Proof

1. Since it is given that $Q$ is $\left(\left(\mathcal{A}_1\otimes\mathcal{A}_2\right)/\mathfrak{B}\right)$-measurable, all that's left to check is that for all $B_3\in\mathcal{A}_3$ and all $B_{1,2}\in\mathcal{A}_1\otimes\mathcal{A}_2$,

   $$\int_{B_{1,2}}Q\left(\omega,B_3\right)\space P_{X_1,X_2}\left(d\omega\right)=P\left(\left(X_1,X_2\right)\in B_{1,2},X_3\in B_3\right)$$

   Fix $B_3\in\mathcal{A}_3$. First assume that $B_{1,2}$ is a rectangle: $B_{1,2}=B_1\times B_2$ for some $B_1\in\mathcal{A}_1,B_2\in\mathcal{A}_2$. Then

   $$\begin{array}{lcl}

   \int_{B_{1,2}}Q\left(\omega,B_3\right)\space P_{X_1,X_2}\left(d\omega\right) & = & \int_{B_1}\int_{B_2}Q\left(\left(\omega_1,\omega_2\right),B_3\right)\space P_{X_2}\left(d\omega_2\right)P_{X_1}\left(d\omega_1\right) \

   & = & \int_{B_1}\int_{B_2}Q_{\omega_1}\left(\omega_2,B_3\right)\space P_{X_2}\left(d\omega_2\right)P_{X_1}\left(d\omega_1\right) \

   & = & \int_{B_1}P_{\omega_1}\left(X_2\in B_2,X_3\in B_3\right)P_{X_1}\left(d\omega_1\right) \

   & = & P\left(X_1\in B_1, X_2\in B_2, X_3\in B_3\right) \

   & = & P\left(\left(X_1,X_2\right)\in B_{1,2}, X_3\in B_3\right)

   \end{array}$$

   where the first equation is by Tonelli's theorem, the second is by the definition of $Q_{\omega_1}$, the third is by the assumption that $Q_{\omega_1}$ is a conditional distribution and the fourth is by the definition of $P_*$.

   Since these rectangles form a generating $\pi$-system for $\mathcal{A}_1\otimes\mathcal{A}_2$, we can extend the result to all $B_{1,2}\in\mathcal{A}_1\otimes\mathcal{A}_2$ using Dynkin's $\pi$-$\lambda$ theorem.

2. Since $f$ is non-negative and $\left(\left(\mathcal{A}_1\otimes\mathcal{A}_2\right)\otimes\mathcal{A}_3/\mathfrak{B}\right)$-measurable, it remains to verify that for all $B_{1,2,3}\in\left(\mathcal{A}_1\otimes\mathcal{A}_2\right)\otimes\mathcal{A}_3$,

   $$\int_{B_{1,2,3}}f\space d\left(P_{X_1,X_2}\otimes\nu\right)=P\left(\left(\left(X_1,X_2\right),X_3\right)\in B_{1,2,3}\right)$$

   First assume that $B_{1,2,3}=\left(B_1\times B_2\right)\times B_3$ for some $B_i\in\mathcal{A}_i$, $i=1,2,3$. Then

   $$ \begin{array}{lcl}

   \int_{B_{1,2,3}}f\space d\left(P_{X_1,X_2}\otimes\nu\right) & = & \int_{B_1}\int_{B_2}\int_{B_3}f\left(\left(\omega_1,\omega_2\right),\omega_3\right)\space \nu\left(d\omega_3\right)\space P_{X_2}\left(d\omega_2\right)\space P_{X_1}\left(d\omega_1\right) \

   & = & \int_{B_1}\int_{B_2}\int_{B_3} f_{\omega_1}\left(\omega_2,\omega_3\right)\space\nu\left(d\omega_1\right)\space P_{X_2}\left(d\omega_2\right)\space P_{X_1}\left(d\omega_1\right) \

   & = & \int_{B_1} P_{\omega_1}\left(X_1\in B_1,X_2\in B_2\right)\space P_{X_1}\left(d\omega_1\right) \

   & = & P\left(X_1\in B_1, X_2\in B_2, X_3\in B_3\right) \

   & = & P\left(\left(\left(X_1,X_2\right),X_3\right)\in B_{1,2,3}\right)

   \end{array} $$

   where the first equation is by Tonelli's theorem, the second equation is by the definition of $f_{\omega_1}$, the third equation is by the assumption that $f_{\omega_1}$ is a density and the fourth equation is by the definition of $P_{\omega_1}$.

   Using Dynkin's $\pi$-$\lambda$ theorem we can extend the result to all $B_{1,2,3}\in\left(\mathcal{A}_1\otimes\mathcal{A}_2\right)\otimes\mathcal{A}_3$.

Q.E.D.