4

Let $U$ be a Uniform$[0,1]$ variate and conditioned on $U$, let $\{X_{n}\}_{n\geq 1}$ be iid $\operatorname{Bern}(U)$ variates. Then show that $E(U|\sigma(X_{1},...,X_{n}))\xrightarrow{a.s.} U$

My attempt(s): Firstly, I can immediately notice that $Y_{n}=E(U|\sigma(X_{1},...,X_{n}))$ is a Doob Martingale sequence that is uniformly bounded (by $1$) in $L^{\infty}$ . So, by Martingale convergence theorem $Y_{n}\xrightarrow{a.s\, ,\, L^{p},\,p<\infty} X $ for some $X\in L^{p}$ for all $p<\infty$. But I don't know how to show that $X$ is equal to $U$.

I also tried to use the method of moments as $U$ is compactly supported. I tried to show that $E(U^{m})=E(E(U^{m}|X_{1},...,X_{n}))\to E(X^{m})$ but the issue is that I need to consider $\bigg(E(U|X_{1},...,X_{n})\bigg)^{m}$ instead of $E(U^{m})$ and that we only have $\bigg(E(U|X_{1},...,X_{n})\bigg)^{m}\xrightarrow{L^{m}} X^{m}$. This does not give us that $X^{m}$ has the same moments as $U^{m}$.

I also know from the Convergence Theory that $Y_{n}=E(X|X_{1},...,X_{n})$ which would mean that $E(X|X_{1},...,X_{n})=E(U|X_{1},...,X_{n})$. From this can we conclude that $X=U$ as we have that $\int_{A} (X-U)\,dP=0$ for all $A\in \sigma(X_{1},...,X_{n})$. I am very unsure about this as I am not using any property of $X_{1},...,X_{n}$ let alone the fact that conditioned on $U$, they are iid Bernoulli$(U)$ variates.

Since $X_{n}$ are iid, I tried to think of using Kolmogorov's Zero-One law but I couldn't see how that could help.

Can anyone provide a hint or help me with this?

Davide Giraudo
  • 181,608
Dovahkiin
  • 1,610
  • 1
    $\int_{A} (X-U),dP=0$ for all $A\in \sigma(X_{1},...,X_{n})$ implies $X=U$ a.s. only when $X-U$ is measurable w.r.t. $ \sigma(X_{1},...,X_{n})$. We certainly don't have this. – Kavi Rama Murthy Jan 04 '24 at 08:48
  • @geetha290krm You are correct. I forgot about the measurability issue. – Dovahkiin Jan 04 '24 at 08:51
  • If you're familiar with estimation theory: $\mathbb{E}[U|\sigma(X_1^n)]$ is the optimiser of $\min \mathbb{E}[ (U - \tau)^2]$ over $\sigma(X_1^n)$ measurable $\tau$ (i.e. estimates of $U$). But its well known that the mean-square optimal Bernoulli parameter estimate is just $\sum X_i/n$, which converges to $U$ by the SLLN. Some $\omega$ chasing is probably necessary to make this precise (you need the SLLN over only $\omega : U(\omega) = u$). – stochasticboy321 Jan 04 '24 at 12:48
  • @stochasticboy321 Sorry I have not had a course in statistics. But I appreciate your comment and I think it will be helpful to other users who will come across this post. – Dovahkiin Jan 04 '24 at 13:18

1 Answers1

2

The $\sigma$-algebra generated by $X_1,\dots,X_n$ is generated by a partition hence in order to compute $\mathbb E\left[U\mid\sigma(X_1,\dots,X_n)\right]$, it suffices to determine for each $\delta_1,\dots,\delta_n\in\{0,1\}$, the quantities $$\tag{(*)} \mathbb E\left[U\mathbb{1}_{(X_1,\dots,X_n)=(\delta_1,\dots,\delta_n)}\right]\mbox{ and }\mathbb P\left((X_1,\dots,X_n)=(\delta_1,\dots,\delta_n)\right). $$ To do so, we use conditioning with respect to $U$ and the assumption, the result will be expressed in terms of $\ell(\delta)=\sum_{i=1}^n\delta_i$ and $n$. By assumption $$ \mathbb E\left[\mathbb{1}_{(X_1,\dots,X_n)=(\delta_1,\dots,\delta_n)}\mid U\right]=U^{\ell(\delta)}(1-U)^{n-\ell(\delta)}. $$ Integrate this to get the second part of (*). For the first one, multiply by $U$, put $U$ inside the conditional expectation and take the expectation. You will get that $$ \mathbb E\left[U\mid\sigma(X_1,\dots,X_n)\right]=\sum_{(\delta_i)_{i=1}^n\in\{0,1\}^n} c_{n,(\delta_i)_{i=1}^n}\mathbf{1}_{(X_i)_{i=1}^n=(\delta_i)_{i=1}^n}. $$

In order to show that $\mathbb E\left[U\mid\sigma(X_1,\dots,X_n)\right]$ converges to $U$ almost surely, we use the fact that if $\mathcal G$ is a $\sigma$-algebra and $U$ and $\mathbb E[U\mid\mathcal G]$ has the same distribution, then $U=\mathbb E[U\mid\mathcal G]$ almost surely.

Once the distribution of $\mathbb E\left[U\mid\sigma(X_1,\dots,X_n)\right]$ is determined, it suffices to check that it converges in distribution to a uniform random variable.

Davide Giraudo
  • 181,608
  • Thanks for your answer (+1) .Can you please elaborate on the second paragraph in your answer "To do so, we use conditioning with respect to $U$ and the assumption, the result will be expressed in terms of $\ell(\delta)=\sum_{i=1}^n\delta_i$ and $n$. " . I could not understand it. Also, I tried to prove convergence in distribution in my attempt using method of moments. So it is not clear to me how the terms $\ell(\delta)$ will help me in figuring out the distribution. – Dovahkiin Jan 04 '24 at 11:24
  • Additionally, can you also tell me why equality of distribution would imply equality almost surely? It is clear that $U$ and $E(U|\mathcal{G})$ are defined in the same probability space. If I am not wrong, $U$ will have to be $\mathcal{G}$ measurable or atleast measurable wrt the completion of $\mathcal{G}$ (i.e. G with the null sets). In that case, I can see that $U$ must equal $E(U|\mathcal{G})$. I hope I am making sense here. – Dovahkiin Jan 04 '24 at 11:29
  • @Dovahkiin I have added a bit more details and also a link. – Davide Giraudo Jan 04 '24 at 11:39
  • Thanks. Much clearer now. – Dovahkiin Jan 04 '24 at 11:42
  • Can you explain to me how to show that $\displaystyle\sum_{\delta\in {0,1}^{n}}f(c_{n,\delta})\dfrac{\Gamma(\ell(\delta)+1)\Gamma(n-\ell(\delta)+1)}{\Gamma(n+2)}\to \int_{0}^{1}f(x),dx$ where $c_{n,\delta}=\dfrac{\Gamma(\ell(\delta)+2)\Gamma(n-\ell(\delta)+1)}{\Gamma(n+3)}$ keeping with your notation ? Is this a real analytic result? I think this result must be true. (f is a continuous bounded function) – Dovahkiin Jan 04 '24 at 12:17
  • You can group the $\delta$ which have the same $\ell(\delta)$, which will give a sum over a single index, say $j$. The binomial coefficient which will appear multiplied by the quotient of gamma's will give something simpler. At some point, I believed that we would get something like a Riemann sum but it does seem so simple. Maybe an alternative would be to look at the cumulative distribution function. – Davide Giraudo Jan 04 '24 at 13:26
  • I thought that looking at the CDF would be more difficult that proving the convergence of a Riemann like sum. That's why I went forward with proving convergence of expectations for continuous bounded functions. I'll have to think more in terms of the cdf. Please update if you find a way out for the limit. I'll also update you in the comments. Anyways, a huge thanks for the help so far. I know that you are the go to expert in this site when it comes to Probability Theory. – Dovahkiin Jan 04 '24 at 13:52
  • An idea could be to use the fact that $(c_{n,\ell(\delta)}){\ell(\delta)\leqslant\lfloor (n-1)/2\rfloor}$ is increasing and we can approximate $f(c{n,\ell(\delta)})$ by something times the integral of $f$ over $[c_{n,\ell(\delta)},c_{n,\ell(\delta)}+1]$. For the $\delta$ such that $\ell(\delta)>\lfloor (n-1)/2\rfloor$, we use this time decreasingness. – Davide Giraudo Jan 04 '24 at 14:51