1

So say I have $U_1, U_2, \cdots$ be iid $U[0,1]$, and say $X_n = \prod_1^n U_i$, if $N = \inf\{n: X_n < 10^{-6}\}$, I need to find two expectations

1) $E[N]$

2) $E[X_N]$

My initial thought was convert the uniform random variables to exponentials by taking the negative logarithm, i.e. $-\ln(U_i) = W_i$, thus we have $Y_n = -\ln(X_n) = \sum_1^n W_i$ which transforms $N$ to $$N = \inf\{n: Y_n > 6 \ln(10) \}$$ which kind of screams coupon collector problem. But now the probabilities of the sampling time is not uniform any more so I am not sure on how to actually calculate the expectation.

Any hints/help on finding the first expectation would be greatly appreciated, in addition any hint on even starting the second expectation would also be phenomenal.

ManikSin
  • 107

1 Answers1

3

$\require{begingroup}\begingroup\renewcommand{\dd}[1]{\,\mathrm{d}#1}$Actually you were on the right track to solving the first question. Allow me to elaborate.

Sec.1$\quad$ Expectation of $N$

Each $W_i = -\ln U_i$ that follows the standard exponential distribution can be viewed as the waiting time between consecutive occurrences of a simple Poisson process with rate of unity.

In other words, the sum $Y_n = \sum_{i = 1}^n W_i$ that follows the Gamma distribution with shape parameter $n$ is the waiting time for the $n$-th occurrence of the Poisson process with unity rate.

You have a cutoff time length of $t = 6 \ln 10$. Consider a random variable $N_t$ that follows the discrete Poisson distribution with the probability mass function of

$$P(N_t = k) = e^{-\lambda t} \frac{ (\lambda t)^k }{ k! }~, \quad \text{where}~ \lambda = 1,~t = 6 \ln 10$$

That is, $N_t$ with the parameter $\lambda t$ counts the number of total occurrences within a time interval of length $t$ of a Poisson process with rate $\lambda$.

Now, $N$ is actually related to the random variable $N_t$ in that $$N = N_t + 1$$

Formally, this is the equivalence between the events $\{ Y_n > t\} = \{ N_t < n\}$ for the waiting time of Poisson process and the count. To be precise, we have the following equivalence of events:

\begin{alignat}{2} \{ N = 1 \} &= \{ Y_1 > t \} = \{ N_t < 1 \} &&= \{ N_t = 0\} \\ \{ N = 2 \} &= \{ Y_1 < t ~~\& ~~ Y_2 > t \} &&= \{ N_t = 1\} \\ \{ N = 3 \} &= \{ Y_2 < t ~~\& ~~ Y_3 > t \} &&= \{ N_t = 2\} \\ &\qquad\qquad \vdots\\ \{ N = k \} &= \{ Y_{k-1} < t ~~\& ~~ Y_k > t \} &&= \{ N_t = k-1 \} \\ &\qquad\qquad \vdots \end{alignat}

Thus the answer is simply given by one plus the expectation of a discrete Poisson distribution \begin{align} E[N] = 1+E[N_t] &= 1 + \lambda t \\ &= 1 + 6 \ln 10 \approx \bbox[1px,border:2px solid orange]{14.81551} \end{align}

Sec.2.1$\quad$ Expectation of $X_N$ as $E\bigl[ e^{-Y_N} \bigr]$

Consider a general threshold $0 < \tau < 1$ (and not just $\tau = 10^{-6}$ here) such that $$ t = -\ln \tau \quad \Longleftrightarrow \quad \tau = e^{-t}$$ The unit rate $\lambda = 1$ is understood to be always present implicitly, but it will be omitted.

Continue previous section's line of reasoning, note that breaking down the events like so \begin{align} \{ N = k \} &= \Big\{ Y_{k-1} < t ~~\& ~~ W_k > t - Y_{k-1} \Big\} \tag{1.1} \\ &= \left\{ X_{k-1} > \tau ~~\& ~~ U_k < \frac{\tau}{ X_{k-1} } \right\} \tag{1.2} \end{align} will be useful for our expectation calculation via conditioning.

\begin{align} E\bigl[ X_N \bigr] &= \sum_{k = 1}^{\infty} E\bigl[ e^{-Y_N} ~\big|~ N = k \bigr] \cdot P(N = k) \\ &= \sum_{k = 1}^{\infty} E\bigl[ e^{-Y_k} ~\big|~ Y_{k-1} < t ~~\& ~~ Y_k > t \bigr] \cdot P(N = k) \\ &= \sum_{k = 1}^{\infty} \frac{ E\bigl[ e^{ -(Y_{k-1} + W_k) } ~~ \scriptsize\color{blue}{\textbf{in region}}\normalsize~~ Y_{k-1} < t ~~\& ~~ W_k > t - Y_{k-1} \bigr] }{ P(Y_{k-1} < t ~~\& ~~ W_k > t - Y_{k-1}) } P(N = k) \\ &= \sum_{k = 1}^{\infty} \int\limits_{y=0}^t \left[ \int_{w=t-y}^{\infty} e^{-(y+w)} \cdot f_{ Y_{k-1} }(y) \cdot f_{ W_k }(w) \dd{w} \right] \dd{y} \tag{2} \end{align} where we have already invoked the independence between $Y_{k-1}$ and $W_k$ such that the joint is the direct product: $f_{Y_{k-1}, W_k}(y, w) = f_{Y_{k-1}}(y) \cdot f_{W_k}(w)$.

By design, the mass over the conditioning region $\{ (y,w)~|~ 0 < y < t, ~ w + y > t \}$ equals exactly the partitioning probability $P(N = k)$, which as stated before is also the mass of the discrete Poisson distribution $P(N_t = k-1)$. We can double check this afterwards.

Surely one can arrive directly at the final double integral expression and skip the whole "conditioning". It is after all just a partition into disjoint events $\{N = k\}$.

Continue Eq.(2) and implement the densities Gamma$[k\!-\!1, 1]$ for $Y_{k-1}$ and Exp$[1]$ for $W_k$. \begin{align} E\bigl[ e^{-Y_N} \bigr] &= \sum_{k = 1}^{\infty} \int\limits_{y=0}^t \left[ \int_{w=t-y}^{\infty} e^{-(y+w)} \cdot \frac{ e^{-y} y^{k-2} }{\Gamma(k-1)} \cdot e^{-w} \dd{w} \right] \dd{y} \\ &= \sum_{k = 1}^{\infty} \int\limits_{y=0}^t \frac{ e^{-2y} y^{k-2} }{(k-2)!} \left[ \int_{w=t-y}^{\infty} e^{-2w} \dd{w} \right] \dd{y} \\ &= \frac{ e^{-2t} }2 \sum_{k = 1}^{\infty} \int\limits_{y=0}^t \frac{ y^{k-2} }{(k-2)!} \dd{y} \\ &= \frac{ e^{-2t} }2 \sum_{k = 1}^{\infty} \frac{ t^{k-1} }{ (k-1)! } \\ &= \frac{ e^{-t} }2 \end{align} Recall that $\tau = e^{-t}$, namely, $\bbox[2px,border:2px solid orange]{E[X_N] = \tau /2}$. This is the answer to the second question.

For those who might be curious, this nice result is not a coincidence. As we shall see in the latter half of the next subsection, $X_N$ has a uniform distribution over $(0, \tau)$.

Now, a quick double check the probability mass of $\{ Y_{k-1} < t ~~\& ~~ W_k > t - Y_{k-1} \}$. \begin{align} P(N=k) &= \int\limits_{y=0}^t \int_{w=t-y}^{\infty} \frac{ e^{-y} y^{k-2} }{(k-2)! } \cdot e^{-w} \dd{w} \dd{y} \\ &= e^{-t} \int\limits_{y=0}^t \frac{ y^{k-2} }{(k-2)! } \dd{y} \\ &= e^{-t} \frac{ t^{k-1} }{(k-1)! } \end{align} Indeed, precisely the discrete Poisson mass $P(N_t = k-1)$.

Sec.2.2$\quad$ Expectation of $X_N$ as is

As an alternative to the previous subsection, here we directly handle the distribution of product of uniform, which derivation is the same approach of your logarithm transformation.

With the density for unconditional $X_n$ shown in the linked pages, note that we will be working with $X_{k-1}$ so the power and the factorial are of $k - 2$. The case of $k = 1$ will automatically work out fine as part of the sum, but let's be pedantic and pull it out.

Recall Eq.(1.2) for the region of each piece of the partition. Similar to the process shown in the previous section, we can skip the conditioning for $k=2$ and beyond. \begin{align} E\bigl[ X_N \bigr] &= \sum_{k = 1}^{\infty} E\bigl[ X_N ~\big|~ N = k \bigr] \cdot P(N = k) \\ &= E[X_1 ~\big|~ X_1 < \tau ] \cdot P(N = 1) \\ & \quad\quad {}+\sum_{k = 2}^{\infty} E\Bigl[ X_{k-1} \cdot U_k ~~ \scriptsize\color{blue}{\textbf{in region}}\normalsize~~ X_{k-1} > \tau ~~\& ~~ U_k < \frac{ \tau }{ X_{k-1} } \Bigr] \\ &= E[U_1 ~\big|~ U_1 < \tau ] \cdot P(N_t = 0) \\ & \quad\quad {}+\sum_{k = 2}^{\infty} \int_{x = \tau}^1 \int_{u = 0}^{ \frac{ \tau }x } x u \cdot \frac{ (-\ln x)^{k-2} }{ (k-2)! } \cdot \mathbb{1}_{0<u<1} \dd{u} \dd{x} \end{align} The conditional expectation of $U_1$ is easily the mid point between its lower bound (same, zero) and its conditionally restricted upper bound $\tau$. \begin{align} E\bigl[ X_N \bigr] &= \frac{\tau}2 \cdot e^{-t} + \sum_{k = 2}^{\infty} \int\limits_{x =\tau}^1 x \cdot \frac{ (-\ln x)^{k-2} }{ (k - 2)! } \left[ \int_{u = 0}^{ \frac{ \tau }x } u \dd{u} \right ] \dd{x} \\ &= \frac{ \tau^2 }2 + \sum_{k = 2}^{\infty} \int\limits_{x = \tau}^1 x \cdot \frac{ (-\ln x)^{k-2} }{ (k - 2)! } \cdot \frac12 \bigl( \frac{\tau}x \bigr)^2 \dd{x} \\ &= \frac{ \tau^2 }2 + \frac{ \tau^2 }2 \sum_{k = 2}^{\infty} \int_{x = \tau }^1 \frac{ (-\ln x)^{k-2} }{ (k - 2)! } \frac{ \dd{x} }x \\ &= \frac{ \tau^2 }2 + \frac{ \tau^2 }2 \sum_{k = 2}^{\infty} \frac{ (-\ln \tau)^{k-1} }{ (k - 1)! } \\ &= \frac{ \tau^2 }2 + \frac{ \tau^2 }2 \bigl( e^{ -\ln \tau } - 1 \bigr) \\ &= \frac{ \tau }2 \end{align} Necessarily we recover the same nice "average of uniform being the mid point". As promised, one could have ignored the formal mismatch of $k=1$ and carried out the sum as a whole.

Last but not least, let's derive the distribution of $X_N$ so as to understand "why" $E[X_N]=\tau/2$. For $0 < s < \tau$, consider the CDF as the probability:

\begin{align} \Pr\bigl\{ X_N < \mathbf{\color{magenta}s} \bigr\} &= \sum_{k = 1}^{\infty} \int\limits_{x = \tau }^1 \int_{u = 0}^{ \mathbf{\color{magenta}s}/ x } \frac{ (-\ln x)^{k-2} }{ (k-2)! } \cdot \mathbb{1}_{0<u<1}\dd{u} \dd{x} \\ &= \sum_{k = 1}^{\infty} \int\limits_{x =\tau }^1 \frac{ (-\ln x)^{k-2} }{ (k - 2)! } \left[ \int_{u = 0}^{ \mathbf{ \color{magenta}s}/x } 1 \dd{u} \right ] \dd{x} \\ &= \mathbf{\color{magenta}s} \sum_{k = 1}^{\infty} \int\limits_{x =\tau }^1 \frac{ (-\ln x)^{k-2} }{ (k - 2)! } \frac{\dd{x}}x \\ &= \mathbf{\color{magenta}s} \sum_{k = 1}^{\infty} \frac{ (-\ln \tau)^{k-1} }{ (k-1)! } \\ &= \mathbf{\color{magenta}s} \cdot e^{ -\ln \tau } \\ &= \mathbf{\color{magenta}s} / \tau \end{align} That is, the cumulative distribution function $F_{X_N}(s) = s / \tau$ is linear and the density is constant over $(0, \tau)$, the uniform distribution as we know it.$\endgroup$