We know that for $p \equiv 3 (\text{mod }4)$, there is no solution to $x^2 + 1 \equiv 0 (\text{mod }p^k)$ for $k = 1, 2, \ldots$, by quadratic reciprocity. But can I embed the ring $\mathbb{Z} / p^k \mathbb{Z}$ in some bigger ring so that it does have a solution? What's the simplest way to do that?
As a related question, if I identify $\mathbb{Z} / p \mathbb{Z} \cong F_p$ the finite field of $p$ elments, then it seems there is only one quadratic extension field of $F_p$ that contains square roots of all its elements. Are there good references to this result?
The question was inspired by Example 1.6 of Opera de Cribro.