Here is a partial answer, I'll try to fill in missing parts.
A sylow $5$-subgroup $P$ of $S_5$ has order $5$, so it is cyclic, so generated by some $5$-cycle $\sigma$. So $P = \langle \sigma \rangle$. Let $N := N_{S_5}(P)$. By sylows theorem, we have that $n_5 = 1 + 5k$ for $k \in \mathbb{Z}$ and that $n_5 \;|\; 24$. Since every $5$-cycle generates a sylow $5$-subgroup, we know that there is not $1$, so there are $6$ (the only other choice left).
Let $G$ act on its set $\text{Syl}_{5}(S_5)$ of sylow $5$-subgroups by conjugation. Then by orbit-stabilizer (for arbitrary $P \in \text{Syl}_{5}(S_5)$), we have $\displaystyle 6 = \frac{120}{|N_G(P)|} \iff |N_{G}(P)| = \frac{120}{6} = 20$. We know that $P \leq N_{G}(P)$ so that gives us $5$ elements, and we need to find $15$ more. Note that the centralizer of $\sigma$ is just $\sigma^k$ for $k \in \{1,\ldots,5\}$. Any other element one can see has to be a $4$-cycle, since we need $\tau \sigma \tau^{-1} = \sigma^k$, and $\sigma^k$ will be on the form $(1ijk\ell)$ where $ijk\ell$ have been permuted. Assume that $\tau \sigma \tau^{-1} = \sigma^2$ (I $\underline{\text{claim}}$ one can always find a $4$-cycle in $S_n$ which does this!).
Then we have $$\tau^2\sigma \tau^{-2} = \tau\sigma^2\tau^{-1} = \underbrace{(\tau\sigma)}_{= \sigma^2\tau}\sigma\tau^{-1} = \sigma^2\tau\sigma\tau^{-1} = \sigma^4$$ $$\tau^3\sigma\tau^{-3} = \ldots = \sigma^{2^3} = \sigma^3.$$ Note that $\tau \sigma^2 \tau^{-1} = \tau \sigma^2\tau^3 = \tau(\sigma^2\tau)\tau^2 = \tau (\tau\sigma)\tau^2 = \tau^2\sigma \tau^{-2} = \sigma^4$, $\tau \sigma^3 \tau^{-1} = \sigma$ and $\tau \sigma^4 \tau^{-1} = \sigma^3$. So $\tau P\tau^{-1} = P$. It follows that $\tau^k P \tau^{-k} = P$. So $\langle \tau \rangle$ normalizes $P$.
Recall that if $H,K$ are subgroups of $G$ and $H \leq N_{G}(K)$ then $HK$ is a subgroup with cardinality $|HK| = \frac{|H||K|}{|H \cap K|}$. Since $H = \langle \tau \rangle \leq N$ and $H \cap P = \{1\}$ we see that by order considerations, $N = HP$.
Consider that $P \leq N$ and that $|N/P| = 4$; every group of order $4$ is abelian, and since the $\underline{\text{commutator}}$ subgroup $N'$ is the smallest normal subgroup so that the quotient is abelian, and $N'$ is contained in any normal subgroup of $N$ so that the quotient is abelian, we see that $N' \subseteq P$. Since $\tau\sigma = \sigma^2\tau$ $N$ is not abelian, so $N' \neq 1$ (if the commutator subgroup is trivial, then the group is abelian). This forces $N' = P$.
Recall that $|N/N'| = |N/P| = 4$ are # of linear characters of $N/P$. Each such character gives us a linear character of $N$.
We have to find the conjugacy classes of $N$. I believe one works out that (using representative elements) $$1_{1},\sigma_{4},\tau_{5},\tau^2_{5},\tau^{3}_{5}$$ are the conjugacy classes, where subscript denotes the size of the conjugacy class.
Need to fill in details here. See remark.
So we have $\psi_1,\ldots,\psi_4:N/P \to \mathbb{C}$ that are linear characters, they correspond to linear characters of $N$. But then we see that $\text{Ind}^{N}_{N}\psi_i$ gives us $4$ irreducible degree $1$ characters.
Since there are as many characters as there are conjugacy classes, we are missing one character $\chi$ of $N$.
Recall that $\displaystyle \chi_{\text{reg}} = \sum \chi_i(1)\chi_i$ so that $$\chi_{\text{reg}}(1_{N}) = 20 = 1^2+1^2+1^2+1^2+\chi(1)^2.$$ Hence $\chi(1) = 4.$
Let $\Psi$ be a character of a subgroup $K$ of $N$, then recall that $\text{Ind}^{N}_{K}\Psi(1) = [G:K]\Psi(1)$. Since $[N:P] = 4$, a natural choice is to find a degree $1$ character of $P$. Since $P$ is abelian, all characters are one dimensional. Take $\Psi(\sigma) = \zeta \in \mu_{5}$, where $\mu_{5}$ are the group of $5^{\text{th}}$ roots of unity, and $\zeta$ is primitive.
By definition, we have $$\text{Ind}^{N}_{P}\Psi(g) = \frac{1}{5} \sum_{x \in N}\Psi^{0}(xgx^{-1})$$ where $$\Psi^{0}(g) := \begin{cases} \Psi(g), \ \text{if} \ g \in P\\ 0, \ \text{if} \ g \not \in P\end{cases}.$$
In particular, we have $$\text{Ind}^{N}_{P}\Psi(1) = \frac{1}{5} \sum_{x \in N} \Psi(1) = \frac{20}{5} = 4$$ and $$\text{Ind}^{N}_{P}\Psi(\sigma) = \frac{1}{5}\sum_{x \in X} \Psi^{0}(x\sigma x^{-1}).$$ Notice that $\tau\sigma^k$ for $k \in \{0,\ldots,4\}$ is such that $(\tau \sigma^k)\sigma(\tau\sigma^k)^{-1} = \sigma^2$. Similarly with $\tau^2\sigma^k$ (giving $\sigma^3$) and $\tau^3\sigma^k$ (giving $\sigma^4$). For $\sigma$ we obviously have $1,\sigma,\sigma^2,\sigma^3,\sigma^4$ giving us back $\sigma$. But we have now exhausted all the elements of $N$.
We then get that since $\Psi$ is one-dimensional, it is a homomorphism, and recall that $1 + \zeta+\zeta^2+\zeta^3+\zeta^4 = 0 \iff \zeta+\zeta^2+\zeta^3+\zeta^4 = -1$.
$$\leadsto \text{Ind}^{N}_{P}\Psi(\sigma) = \frac{1}{5}\left(5\Psi(\sigma)+5\Psi(\sigma^2)+5\Psi(\sigma^3)+5\Psi(\sigma^4)\right) = -1.$$ We also know that (see remark below) $\text{Ind}^{N}_{P}\Psi(\tau^k) = 0$ for $k \in \{1,2,3\}$ and hence $ = 0$. The easiest way to see this I believe, is that $P$ is normal in $N$, so if $g \in N\setminus\{P\}$ then $xgx^{-1} \not \in P$ for arbitrary $x \in N$ (if $xgx^{-1} = p \in P \iff g = x^{-1}px$ but this would contradict the normality of $P$).
Finally, one finds that $\left[\text{Ind}^{N}_{P}\Psi,\text{Ind}^{N}_{P}\Psi\right]_{N} = \frac{1}{20}\left(4^2+4 \cdot (-1)^2 \right) = 1 \iff \text{Ind}^{N}_{P}\Psi$ is irreducible.
Remark: To say something more about the conjugacy classes, since $\tau = (abcd)$ is a $4$-cycle, we see that $\tau^2$ is a permutation of cycle type $(2,2)$, so they are not conjugate in $S_n$; since $N \subseteq S_n$ they can not be conjugate in $N$. To see that $\tau^3$ and $\tau$ are not conjugate, note that $\tau^3 = \tau^{-1} = (adcb)$. Since, for general permutation $\theta$, we have $\theta\tau\theta^{-1} = (\theta(a)\theta(d)\theta(c)\theta(b))$, they would have to be conjugate by a $3$-cycle, but there are no $3$-cycles in $N$, since $3 \nmid 20 = |N|$. Furthermore, if $\sigma^k \tau \sigma^{-k} = \sigma^{s} \tau \sigma^{-s} \iff \sigma^{k-s}\tau \sigma^{s-k} = \tau$. I am still not quite sure how to see that the conjugacy classes are:
- $1$.
- $\sigma,\sigma^2,\sigma^3,\sigma^4$.
- $\tau,\tau\sigma,\tau\sigma^2,\tau\sigma^3,\tau\sigma^4$.
- $\tau^2,\tau^2\sigma,\tau^2\sigma^2,\tau^2\sigma^3,\tau^2\sigma^4$.
- $\tau^3,\tau^3\sigma,\tau^3\sigma^2,\tau^3\sigma^3,\tau^3\sigma^4$.
Here is a presentation which gives (essentially) the same solution: http://sporadic.stanford.edu/Math122/lecture14.pdf
Addendum: Inspired by Steve D:s comment. By orbit-stabilizer theorem, we know that (letting $N$ act on itself by conjugation) $|\text{Orb}(\tau)| = \frac{20}{|\text{Cent}_{N}(\tau)|} = 5$, since we know that $\text{Cent}_{S_5}(\tau) = \{1,\tau,\tau^2,\tau^3\}$.
Similarly, we can show this for $\tau^2$ and $\tau^3$. By earlier argument, we saw that they can not be in the same conjugacy class. By elementary arithmetic, we find that $1+4+5+5+5 = 20$, so since the conjugacy classes distributes the elements into disjoint sets, we are done. This however still does not really tell us what the elements in the conjugacy classes are, directly, but I am not sure one needs to know that to solve this problem (my guess is that one does not need it).
Second addendum: "Furthermore, if $\sigma^k \tau \sigma^{-k} = \sigma^{s} \tau \sigma^{-s} \iff \sigma^{k-s}\tau \sigma^{s-k} = \tau$" is a half-sentence. I believe I was trying to show that conjutation by $\sigma^k$ of $\tau$ is unique in the sense that if $k \neq \ell$ then $\tau^{\sigma^{k}} \neq \tau^{\sigma^{\ell}}$, where the superscript denotes conjugation.