I'm reading a text on $p$-adics that argues at follows. We have $(\mathbb Z/p^n\mathbb Z)^\times \cong (\mathbb Z/p\mathbb Z)^\times \times \mathbb Z/p^{n-1}\mathbb Z$. At the very least we get such an isomorphism by noticing both sides are cyclic of the same order. Then we can compute $$ \mathbb Z_p^\times \cong \varprojlim_{n \in \mathbb N} (\mathbb Z/p^n\mathbb Z)^\times \cong (\mathbb Z/p\mathbb Z)^\times \times \varprojlim_{n \in \mathbb N} \mathbb Z/p^{n-1}\mathbb Z \cong (\mathbb Z/p\mathbb Z)^\times \times \mathbb Z_p. $$ To me this argument neglects the maps between the various groups in the inverse system: using the identification $(\mathbb Z/p^n\mathbb Z)^\times \cong (\mathbb Z/p\mathbb Z)^\times \times \mathbb Z/p^{n-1}\mathbb Z$, are the maps still the ones you'd expect (the identity on $(\mathbb Z/p\mathbb Z)^\times$ and on the other factor a quotient map)?
There definitely is a subtle problem with the argument because it isn't quite true for $p = 2$: because $\mathbb Z_2^\times$ contains an element of order 2, namely $-1$, while $\mathbb Z_2$ does not, so $\mathbb Z_2^\times \not\cong \mathbb Z_2$. But I don't see what part of the argument wouldn't work just as well for $p=2$ as for any other prime.
There's another argument for $p \neq 2$ that I trust better: every element of $(\mathbb Z/p\mathbb Z)^\times$ lifts to an unique $(p-1)$th root of unity in $\mathbb Z_p$, which yields $\mathbb Z_p^\times \cong (\mathbb Z/p\mathbb Z)^\times \times (1 + p \mathbb Z_p)$. Next, the second factor is isomorphic to $p\mathbb Z_p$ using the $p$-adic logarithm, and ofcourse $p\mathbb Z_p \cong \mathbb Z_p$.
I know this latter argument doesn't work for $p = 2$, and I believe this is because the $2$-adic logarithm isn't as nicely behaved (the proof of convergence of the exponent on $p\mathbb Z_p$, which should be its inverse, breaks down for $p = 2$). But I'd like to know what the correct argument would be in the $p = 2$ case.