This is quite a general question. The case for the previous $S_p$ is trivial and it seems the case with $S_{p^2}$ is different. For all $2\le n\le p-1$, those $p$-Sylows will have $p^n$ order.
So to make that happen, my idea is generally, for each case, I use $n$ different cyclic's generator (like $n$ different $p$-cycles) to generate a new group, which should contain $p^n$ elements.
Am I right?