Why can a probability measure be defined over power set of countable sample space?

Question

Let $\Omega$ be the sample space, $p: \Omega \rightarrow [0,1]$, be any function satisfying $\sum_{w\in\Omega} p(w) = 1$. Then there is a valid probability triple $(\Omega, \mathcal{F},P)$, where $\mathcal{F}$ is the collection of all subsets of $\Omega$ (the power set), and for $A \in \mathcal{F}, P(A) = \sum_{w\in A} p(w)$.

My question is why/how we know that, when $\Omega$ is finite or countable, a probability measure can be defined over ALL possible subsets (vs, say, the fact that a probability measure -- satisfying the standard axioms, of course -- cannot be defined over all subsets of [0,1]). Is there a proof of this somewhere (if so, please let me know where I can look), or could someone provide one?

Answer 1

You can always define certain measures on the whole power set, for example the dirac measure $\delta_x (A)=1$ if $x\in A $ and $\delta_x (A)=0$ otherwise.

Also, countable sums of Diracs work well. On a countable sample space, any measure is of this form (countable linear combination if Diracs).

Only when we consider more interesting measures like the Lebesgue measure, we can't define it on the whole powerset while maintaining desired properties like translation invariance.

Answer 2

A series converges absolutely iff every subseries converges. For countably infinite $\Omega$, it follows that if $p: \Omega \rightarrow [0,1]$ is such that $\sum_{\omega\in \Omega} p(\omega)$ converges, then $\sum_{\omega\in A} p(\omega)$ converges for every $A\subseteq\Omega$.

Answer 3

Sorry for being late to the party, but I felt that the question had still not been answered explicitly.

Claim: Let $\Omega$ be a countable set. Then any measure $\mu$ on any $\sigma$-algebra on $\Omega$ can be extended (not necessarily uniquely) to the whole power set of $\Omega$.

To prove this, fix the given $\sigma$-algebra and $\mu$ and consider the collection $\mathcal{C}$ of all sub-sets $A \subseteq \Omega$ such that $A$ is non-empty and measurable but no proper, non-empty subset of $A$ is measurable. Such $A$'s are disjoint, and they must cover $\Omega$: they are at most countably many (since disjoint), so their union and the complement of their union are also measurable; this complement must be empty, otherwise it would contain an element of $\mathcal{C}$, contradiction! So the $A$'s do cover $\Omega$. Now for each set $A$ in $\mathcal{C}$ pick $x \in A$ and then for each $B \subseteq A$ define $\tilde{\mu}(B) := \mu(A)\delta_x(B)$ (Dirac measure). Now $\tilde{\mu}$ is, in particular, defined on all singletons. Since all subsets of $\Omega$ are disjoint unions of countably many singletons, we can extend $\tilde{\mu}$ to the whole power set of $\Omega$ by additivity. Every measurable set is a disjoint union of countably many $A \in \mathcal{C}$, so this definition is compatible with $\mu$.