It is possible to solve this by trying to find which points double to give you 2-torsion points.
Let $d=\sqrt{k} \ne l^2 \space \forall \space l \in \mathbb{F}_p$
The 2-torsion points are $E[2]=\{\infty, (0,0), (d,0), (-d,0)\} \subset E(\mathbb{F}_p)$
A point $P$ of order 4 can be written as $2P=Q$ where $Q \in E[2]-\{\infty \}$. So if we can find the points that double to give us non-trivial points of the 2-torsion then we have our 4-torsion. We can use the doubling equations for the elliptic curve group:
$$
Q_x=m^2-2P_x \quad
Q_y=m(P_x-Q_x)-P_y \quad
m=\frac{3P_x^2-k}{2P_y}
$$
The above equations can be used to get an expression for $P_x$, and then the elliptic curve equation can be used to find $P_y^2$. The following table summarizes the values:
\begin{array}{c|c|c|c}
& Q=(0,0) & Q=(d,0) & Q=(-d,0) \\ \hline
P_x & \pm id & d(1 \pm \sqrt{2}) & d(-1 \pm \sqrt{2}) \\ \hline
P_y^2 \text{ for } +\text{ve} \, P_x & -2id^3 & 2d^3(1-\sqrt{2})^2 & -2d^3(1-\sqrt{2})^2 \\ \hline
P_y^2 \text{ for } -\text{ve} \, P_x & -2id^3 & 2d^3(1+\sqrt{2})^2 & -2d^3(1+\sqrt{2})^2 \\
\end{array}
We now need to check the following:
- Is $P_x \in \mathbb{F}_p$?
- Is $P_y^2 \in \mathbb{F}_p$ and is it a quadratic residue in $\mathbb{F}_p$?
We can use the following lemmas to help us:
- $i \in \mathbb{F}_p \iff p \equiv 1 \pmod{4}$
- proof: use the Legendre symbol formula for $-1$: $\left( \frac{-1}{p} \right) = (-1)^{\frac{p-1}{2}}$
- $\sqrt{i} \in \mathbb{F}_p \iff p \equiv 1 \pmod{8}$
- proof: similar to above but for $i$
- $\sqrt{2} \in \mathbb{F}_p \iff p \equiv 1 \pmod{8}$
- proof: see here for a proof that $\sqrt{2} \in \mathbb{F}_p \iff p \equiv \pm 1 \pmod{8}$, and then use the fact that $p \equiv 1 \pmod{4}$
- if $a,b$ are quadratic non-residues then $ab$ is a quadratic residue
- proof: use Legendre symbol formula again
The table below shows the output of the above lemmas used on the points in the previous table, for $k \ne 1$:
\begin{array}{c|c|c|c}
& Q=(0,0) & Q=(d,0) & Q=(-d,0) \\ \hline
P_x \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y^2)_{+} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y)_{+} \in \mathbb{F}_p & \text{never} & \text{never} & \text{never} \\ \hline
(P_y^2)_{-} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y)_{-} \in \mathbb{F}_p & \text{never} & \text{never} & \text{never} \\
\end{array}
And for $k=1$:
\begin{array}{c|c|c|c}
& Q=(0,0) & Q=(d,0) & Q=(-d,0) \\ \hline
P_x \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y^2)_{+} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y)_{+} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y^2)_{-} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\ \hline
(P_y)_{-} \in \mathbb{F}_p & \text{always} & \text{when } p \equiv 1 \pmod{8} & \text{when } p \equiv 1 \pmod{8} \\
\end{array}
We can see that $E : y^2=x^3-kx$ has a point of order 4 iff $k=1$.