Exercise 2.3 (Prove Approximation lemma by Halmos ) Probability for Statistician by Galen R. Shorack

Question

Let the $\sigma$-finite meausre $\mu$ on the field $ \mathcal{C}$ be extended to $ \mathcal{A}=\sigma[ \mathcal{C}]$, and also refer to the extension as $\mu$. Then for each $A \in \mathcal{A}$ (or in $ \hat{ \mathcal{A}_\mu}$) such that $\mu(A)<\infty$, and for each $\epsilon>0$, we have $$\mu(A\triangle C)<\epsilon\text{ for some set } C\in \mathcal{C}.$$

Our teacher left a sketch of the proof in his lecture notes:

For any $\epsilon>0, \exists \bigcup^\infty_{n=1}A_n\supset A \ni$ $$\mu(A)\leq \sum^\infty_{n=1}\mu(A_n)<\mu(A)+\epsilon/2.$$ Take $N_0$ such that $\sum^\infty_{n=N_0+1}\mu(A_n)<\epsilon/2.$Define $C= \cup^{N_0}_{n=1}A_n.$ Then we have \begin{equation*} \begin{split} \mu(A\triangle C) & = \mu(A\backslash C)+\mu(C\backslash A) \\ & \leq \mu(\cup_n A_n\backslash C)+\mu(\cup_n A_n\backslash A)\\ & \leq \sum^\infty_{n=N_0+1}\mu(A_n)+(\sum^\infty_{n=1}\mu(A_n)-\mu(A))\\ & < \epsilon/2+\epsilon/2=\epsilon. \end{split} \end{equation*}

My questions are:

1. I think the proof is complete, what else left unproved in the exercise worth noting?
2. I think I don't quite get the conditions given in the question, I believe a $\sigma$-finite measure is different from a finite measure. If the question at first only provides the measure to be $\sigma$-finite. How come later on it also provides that $\mu(A)<\infty$, which means $A$ is a finite measure. (Stronger than $\sigma$-measure if I understand these two terms correctly).
3. What really this Approximation lemma is suggesting? Since originally we have a measure defined on a field but we managed to extend the measure on $\sigma$-field through outer measure thanks to Carathéodory's theorem and even manage to extend to completion. Does this lemma speak that the extent to which we have extended the measure is negligible as the measure is only as small as $\epsilon$, which could be arbitrarily small?
4. If the sketch is not complete, could someone please point out how should I finish the proof. What else is left yet to be proved?

Answer 1

1. The proof to be "complete" or not depends on what is assumed the readers already know.
2. The measure is $\sigma$-finite, but the result only applies to sets $A \in \mathcal{A}$ that has finite measure.
3. Yes, this lemma means that the "new" sets that we now can measure with $\mu$ can be approximete arbitrarily close (in the measure sense)by set that were already in $\mathcal{C}$.
4. Well, to make the proof easier to understand it, a couple of sentences and details can be added, but the sketch is essentially complete. Let me know if you what me to detail the sketch.

Per your request, here is a very detailed proof. We will split the result in two: one for $\sigma[ \mathcal{C}]$ and the other for $ \hat{ \mathcal{A}_\mu}$.

The first result:

Let the $\sigma$-finite measure $\mu$ on the field $ \mathcal{C}$ be extended to $ \mathcal{A}=\sigma[ \mathcal{C}]$, and also refer to the extension as $\mu$. Then for each $A \in \mathcal{A}$ such that $\mu(A)<\infty$, and for each $\epsilon>0$, we have $$\mu(A\triangle C)<\epsilon\text{ for some set } C\in \mathcal{C}.$$

Proof: (to help keep all details visible, we are goind to note the extension of $\mu$ to $\sigma[ \mathcal{C}]$ by $\overline{\mu}$).

Since $\mu$ is $\sigma$-finite measure, we know that there is a unique extension $\overline{\mu}$ of $\mu$ to $\sigma[ \mathcal{C}]$. So, as a consequence of Carathéodory's theorem, such extention $\mu$ to $\sigma[ \mathcal{C}]$ coincides with the restriction of the outer measure $\mu^*$ to $\sigma[ \mathcal{C}]$. So, we have, for all $A \in \mathcal{A}$,

$$ \overline{\mu}(A)= \mu^*(A) = \inf \left \{ \sum^\infty_{n=1}\mu(A_n) : \textrm{for all } n , A_n \in \mathcal{C} \textrm{ and } A \subseteq \bigcup^\infty_{n=1}A_n \right \} $$

Now, given any $A \in \mathcal{A}$ and $ \overline{\mu}(A)<\infty$ and given $\epsilon>0$, there is $\{ A_n\}_n$ such that, for all $n$ , $A_n \in \mathcal{C}$, $A \subseteq \bigcup^\infty_{n=1}A_n$ and

$$\overline{\mu}(A) \leqslant \sum^\infty_{n=1}\mu(A_n) < \overline{\mu}(A) + \frac{\epsilon}{2}$$

Take $N_0$ such that $\sum^\infty_{n=N_0+1}\mu(A_n)<\epsilon/2$. Define $C= \bigcup^{N_0}_{n=1}A_n$.

Since $\mathcal{C}$ is a field, it is clear that $C \in \mathcal{C}$ and we have
\begin{equation*} \begin{split} \overline{\mu} (A\triangle C) & = \overline{\mu}(A\setminus C)+\overline{\mu}(C\setminus A) \\ & \leqslant \overline{\mu}\left (\bigcup_n A_n\setminus C \right )+\overline{\mu}\left (\bigcup_n A_n\setminus A \right)\\ & =\mu\left (\bigcup_n A_n\setminus C \right )+\overline{\mu}\left (\bigcup_n A_n\setminus A \right)\\ & =\mu\left (\bigcup_n A_n\setminus C \right )+\overline{\mu}\left (\bigcup_n A_n\right) - \overline{\mu}(A) \\ & \leqslant\sum^\infty_{n=N_0+1}\mu(A_n)+ \sum^\infty_{n=1}\overline{\mu}(A_n) - \overline{\mu}(A) \\ & =\sum^\infty_{n=N_0+1}\mu(A_n)+ \sum^\infty_{n=1}\mu(A_n) - \overline{\mu}(A) \\ & =\sum^\infty_{n=N_0+1}\mu(A_n)+ \left ( \sum^\infty_{n=1}\mu(A_n) - \overline{\mu}(A) \right )\\ & < \frac{\epsilon}{2}+\frac{\epsilon}{2}=\epsilon. \end{split} \end{equation*}

Now the second result:

Let the $\sigma$-finite measure $\mu$ on the field $ \mathcal{C}$ be extended to $ \hat{ \mathcal{A}_\mu}$ where $ \mathcal{A}=\sigma[ \mathcal{C}]$, and also refer to the extension as $\mu$. Then for each $A \in \hat{ \mathcal{A}_\mu}$ such that $\mu(A)<\infty$, and for each $\epsilon>0$, we have $$\mu(A\triangle C)<\epsilon\text{ for some set } C\in \mathcal{C}.$$

Proof: (to help keep all details visible, we are goind to note the extension of $\mu$ to $\hat{ \mathcal{A}_\mu}$. by $\overline{\mu}$).

Since $\mu$ is $\sigma$-finite measure, we know that there is a unique extension of $\mu$ to $ \mathcal{A}=\sigma[ \mathcal{C}]$ and so a unique extension $\overline{\mu}$ of $\mu$ to $\hat{ \mathcal{A}_\mu}$. So, as a consequence of Carathéodory's theorem, such extention $\mu$ to $\hat{ \mathcal{A}_\mu}$ coincides with the restriction of the outer measure $\mu^*$ to $\hat{ \mathcal{A}_\mu}$.

The rest of the proof is identical to the previous proof, just replacing $\mathcal{A}$ by $\hat{ \mathcal{A}_\mu}$.