Let's say $p(x)$ is reducible so that $q(x)h(x)=p(x)$ for non-constant polynomials $q(x)$ and $h(x)$.
Now, consider $q(x)h(x)+\langle p(x)\rangle$ in the ring $F[X]/\langle p(x)\rangle$. Normally, $q(x)h(x)=p(x)=p(x)$, so clearly, $q(x)p(x) \in \langle p(x)\rangle$. Therefore:
$$q(x)h(x)+\langle p(x)\rangle=0+\langle p(x)\rangle$$
This means $(q(x)+\langle p(x)\rangle)(h(x)+\langle p(x)\rangle)=0+\langle p(x)\rangle$. Now, we need to prove that $q(x)+\langle p(x)\rangle$ and $h(x)+\langle p(x)\rangle$ are non-zero. Well, what does $q(x)+\langle p(x)\rangle$ being $0$ really mean? It means $q(x) \in \langle p(x)\rangle$, which means $q(x)=p(x)r(x)$ for some polynomial $r(x)$. However, $p(x)=q(x)h(x)$, so we have the following:
$$q(x)=q(x)h(x)r(x)$$
This clearly implies that $h(x)r(x)=1$. However, this can only happen if $h(x)$ is a constant polynomial, which we assumed it wasn't. Therefore, $q(x)+\langle p(x)\rangle$ must be non-zero and $h(x)+\langle p(x)\rangle$ is also non-zero.
Thus, we have found two non-zero elements which, when multiplied, yield zero, which means that the ring $F[X]/\langle p(x)\rangle$ has zero divisors, meaning it is not an integral domain.