What is the Bernoulli product measure's Radon-Nikodym derivative wrt Lebesgue measure?

Question

The Bernoulli product measure $\mu$ can be defined for each $p\in (0,1)$ on $\Omega = \{0,1\}^\mathbb N=\{\omega=(\omega_i)|\omega_i\in\{0,1\}, i\in\mathbb N\}=\Pi_{i=1}^\infty \{0,1\}$. The measure $\mu$ is essentially defined by a collection of Bernoulli random variables $X_i:\Omega\to\{0,1\}$ which take the value $1$ with probability $p$ and $0$ with probability $1-p$ depending only on the $i$th coordinate of $\omega\in\Omega$. The pre-images $X_i^{-1}(0)$ and $X_i^{-1}(1)$ are examples of cylinder sets in the product space and generate the $\sigma$-algebra that the product measure $\mu$ is defined on.

We can then define a map $\Omega\to [0;1]\subset\mathbb R$ by $\omega=(\omega_i)\mapsto \sum_{i=1}^\infty \omega_i 2^{-i}$. Up to a set of Lebesgue measure $0$ having to do with duplicate binary expansions of the dyadic rationals $k/2^n$ ($k,n\in\mathbb N$), this map identifies $\Omega$ with the unit interval $[0,1]\subset\mathbb R$. The duplicate issue is that dyadic rationals have both an infinite and a terminating binary expansion (in binary, $1/2=0.1=0.0\bar 1$), so the map is not a perfect bijection on that countable set.

We now have two measures on $[0;1]$, $\mu$ and Lebesgue measure $\cal L$. They are defined on slightly different $\sigma$-algebras since $\cal L$ is defined on the Borel sets and $\mu$ is defined on the $\sigma$-algebra generated by the sets that make the random variables $X_i$ measurable (generated by the cylinder sets).

Is there anything like a Radon-Nikodym derivative of $\mu$ with respect to $\cal L$ even though they are not defined on the same $\sigma$-algebras? For example, is it possible to extend or intersect the $\sigma$-algebras to obtain a useful integral formula with an actual density function so that we could write $$\mu=\nu + \int f d\cal L$$ on some meaningful collection of sets? ($\nu$ a measure singular wrt $\cal L$)

I've looked in a number of books including Billingsley, Chung, Folland, Rudin, Durrett, Cohn, and others. They discuss the construction of $\mu$, the differences of the $\sigma$-algebras that $\mu$ and $\cal L$ are defined on, but I haven't been able to find any kind of expression of one with respect to the other, if that's possible in some way.

My motivation is to get a better understanding of $\mu$ in more standard analytical terms.

PS: I originally posted this question over in

https://stats.stackexchange.com/questions/619732/what-is-the-bernoulli-product-measures-radon-nikodym-derivative-wrt-lebesgue-me

but it was recommended by a commenter that I post it here instead.

Answer 1

Let's first start by noting the measure $\mu$ is well defined on all Borel sets. Take

$$ X : \Omega \longmapsto \mathbb{R} $$ $$ X=\sum\limits_{i}\frac{X_{n}}{2^{n}} $$ Note that $\sum\limits^{N}_{i}\frac{X_{i}}{2^{i}}$ are random variables in $\Omega$ so its limit is in fact a random variable. We can then define a measure $\mu_{X}$ in all borel sets by setting $\mu_{X}(B)=\mu(X^{-1}(B))$. We can also get a little bit more intuition on how this measure works by looking on how it assigns measure to dyadic intervals.

Take any dyadic interval $I=[\frac{b}{2^{n}},\frac{b+1}{2^{n}}]$ of level $n$. By writing $b$ in base $2$ we may identify all dyadic intervals of level $n$ with strings of length $n$ in $\{0,1\}^{[[n]]}$. Now for any string $\sigma$ of length $n$ write $I_{\sigma}$ for the respective dyadic interval and $C_{\sigma}$ for the respective cylinder. We'll show by induction on $n$ that $X^{-1}(I_{\sigma})=C_{\sigma}\cup D_{\sigma}$ where $D_{\sigma}$ is a finite set (and thus of measure $0$ in $\Omega$).

The base case $n=1$ is clear if we notice that $X \in [0,\frac{1}{2}]=I_{0}$ if and only if $X_{1}=0$ or $X_{1}=1$ and $X_{i}=0$ for all $i > 1$, similarly for $I_{1}=[\frac{1}{2},1]$.

Now suppose we have proved our assertion for $n>1$. Take $\sigma$ any string of length of $n+1$. If $\sigma=0\sigma'$ for $\sigma'$ some string of length $n$, we have that $X \in I_{\sigma}$ if and only if $2X \in I_{\sigma'}$ and $X_{1}=0$. Write $\Omega_{0}=\{\omega \in \Omega: X_{1}=0\}$. The shift operator $$S(\omega)_{n}=\begin{cases} \omega_{0} & n=0 \\ \omega_{n+1} & \text{all other cases} \end{cases}$$

is a bijection from $\Omega$ to $\Omega_{0}$, furthermore, $S(C_{\rho})=C_{0\rho}$ for any string $\rho$. Now

$$2X|_{\Omega_{0}}=\sum\limits_{i}\frac{X_{i}}{2^{i-1}}=X \circ S^{-1}$$

So $$X^{-1}(I_{\sigma})=2X|_{\Omega_{0}}(I_{\sigma'})=(X \circ S^{-1})^{-1}(I_{\sigma'})=S(X(I_{\sigma'}))=S(C_{\sigma'}\cup D_{\sigma'})=C_{\sigma} \cup S(D_{\sigma'})$$

Here we used the inductive hypothesis on $X^{-1}(I_{\sigma})$. The case where $\sigma=1\sigma'$ can be done similarly by instead noting that $X^{-1}(I_{\sigma})=(2X-1)|_{\Omega_{1}}^{-1}(I_{\sigma'})$ where $\Omega_{1}=\{\omega \in \Omega : X_{1} = 1\}$. This completes the induction.

It is now clear that $\mu_{X}(I_\sigma)=p^{\sum\sigma_{i}}(1-p)^{\sum (1-\sigma_{i})}$

There is also a more "within $\mathbb{R}$" approach to finding the same measure through Carathéodory's extension theorem. By assigning this probability to each dyadic interval, we can construct exactly the same measure. Moreover in the special case where $p=\frac{1}{2}$, the uniqueness of the extension guarantees $\mu_{X}=\mathcal{L}$. Sadly that as far as we can go as for establishing relationships with $\mathcal{L}$.

For $p$ and $q$ write $\mu_{p}$ and $\mu_{q}$ for the measures constructed as we did above for Bernoulli random variables with parameters $p$ and $q$ respectively. We can prove that if $p \neq q$ then $\mu_{p}$ and $\mu_{q}$ are mutually singular, in particular if $p \neq \frac{1}{2}$, $\mu_{p}$ and $\mathcal{L}$ are mutually singular so Radon-Nikodym tells us little about $\mu_{p}$.

For a real number $x$ write $x_{i}$ for the $i$-th term in the binary expansion. By looking a carefully on how $\mu_{X}$ assigns probability to dyadic intervals, we can show that the functions $f_{i}(x)=x_{i}$ are independent Bernoulli random variables with parameter $p$ when looked at with respect to $\mu_{p}$ and of parameter $q$ with respect to the measure $\mu_{q}$. Thus, by the law of large numbers, the sets $A_{p}=\{ x \in \mathbb{R} : \lim\limits_{n \to +\infty}\frac{\sum^{n}x_{i}}{n}=p\}$ and $A_{q}=\{ x \in \mathbb{R} : \lim\limits_{n \to +\infty}\frac{\sum^{n}x_{i}}{n}=q\}$ fulfill

$$ \mu_{p}(A_{p})=1 $$ $$ \mu_{q}(A_{q})=1 $$

So $\mu_{p}$ and $\mu_{q}$ are mutually singular.

There is a little bit of discussion of the characteristics of these measures and of the sets $A_{p}$ from a fractal geometry perspective in first chapter of:

Bishop, Christopher J.; Peres, Yuval, Fractals in probability and analysis, Cambridge Studies in Advanced Mathematics 162. Cambridge: Cambridge University Press (ISBN 978-1-107-13411-9/hbk; 978-1-316-46023-8/ebook). ix, 402 p. (2017). ZBL1390.28012.