I think a two-pronged attack using both Sylow $5$s and Sylow $3$s will get the job done.
As you observed, we can assume that $n_5=36$ for otherwise we run into a contradiction with the simplicity of $A_6$. Therefore there are $36(5-1)=144$ elements of order five.
Similarly, $n_3=4$ is out of the reckoning, so we can assume that $n_3=10$. All the Sylow $3$-subgroups have $9=3^2$ elements, so they are abelian. Assume that $x\neq1$ is in the intersection of two Sylow $3$-subgroups $P$ and $P'$. Then $x$ is centralized by both $P$ and $P'$, so $H=C_G(x)$ has more than nine elements, and more than one Sylow $3$-subgroup. Hence $n_3(H)\in\{4,10\}$, so $|H|\ge 36$. Of course, if $H=G$, then $x\in Z(G)$ and we are done. In the other cases $[G:H]\in\{2,5\}$, and again the action of $G$ on the set of cosets of $H$ gives a homomorphism from $G$ with a non-trivial kernel.
The remaining possibility is that any pair of Sylow $3$-subgroups has a trivial intersection. In that case there are $10(9-1)=80$ elements of orders $3$ or $9$. But, together with the elements of order five that's too many.