First, I want to mention I solved this for $p > 2$ so there is no need to talk about that case here.
I am trying to understand the squares in $\mathbb{Z}_2$, the 2-adic integers. I managed to show if $a$ is such a square and $0\neq a=2^{v_2(a)}\cdot b \in \mathbb{Z}_2$ then $b\equiv 1\bmod8$ and $2\mid v_2(a)$.
In the other direction, assuming $b\equiv 1\bmod8$ and $2\mid v_2(a)$, I realize I only need to show there is a solution in $\mathbb{Z}_2$ to the equation $x^2=b$. To do that I noticed that $b$ is a square modulo $8$ (more specifically in $\mathbb{Z}_2/8 \mathbb{Z}_2 \cong \mathbb{Z}/8\mathbb{Z}$) and all is left is to use some version of Hensel's Lemma (like I did in the $p>2$ case which worked perfectly).
The problem here is that the version I know of Hensel's Lemma is only applicable when I know some polynomial factors into co-prime factors in $O_v/m_v$ where $m_v$ is the unique maximal ideal in $O_v$ for a discrete valuation $v:K\rightarrow\mathbb{Z}$. But here I only know $x^2-b$ factors in $\mathbb{Z}_2/8 \mathbb{Z}_2$. This is why I need a stronger version where the polynomial factors into co-prime factors only in $O_v/m_{v}^n$.
Also, any simple argument avoiding Hensel's Lemma is welcome! (I figure something like that has a high liklyhood to exist)