Another example: consider $k=\mathbb{Q}$, $F=\mathbb{Q}[\sqrt{2}]$, $E=\mathbb{Q}[\sqrt[4]{2}]$.
The only field $L$ with $k\subseteq L\subseteq E$ and $[L:k]=2$ is $F$. This follows from Galois Theory: the Galois closure of $E$, $\mathbb{Q}[\sqrt[4]{2},i]$ has degree $8$ over $\mathbb{Q}$ and Galois group isomorphic to the dihedral group of order $8$. The field $\mathbb{E}$ corresponds to a subgroup of index $4$ (order $2$) that is not normal, and subfields $L$ would correspond to a subgroup of order $4$ containing the subgroup corresponding to $E$. In $D_8$, the non-normal subgroups of order $2$ are contained each in a unique subgroup of order $4$. Thus, $F$ is the unique intermediate extension of degree $2$ that is contained in $E$.
Now let $a\in E\setminus F$. Then $k(a)$ has degree either $2$ or $4$ over $\mathbb{Q}$, but cannot have degree $2$ since $a\notin F$ and $F$ is the only subextension of degree $2$. So $[k(a):k]=4$, hence $k(a)=E$. In particular, if $f(x)$ is the minimal polynomial of $a$ over $k$, then $\deg(f)=4$.
However, the minimal polynomial of $a$ over $F$ has degree $2$, so $f(x)$ cannot be irreducible over $F$. But it also cannot have a root, since $F$ is Galois over $\mathbb{Q}$ and hence if an irreducible polynomial over $\mathbb{Q}$ has a root in $F$, then it splits in $F$. Thus, $f(x)$ must factor as a product of two irreducible quadratics.
Thus, for all $a\in E-F$, the minimal polynomial of $a$ over $k$ factors nontrivially in $E$; but $E\neq F$.