I want to find a compact Riemannian manifold $M$ such that the group $SU(n)/\mathbb{Z}_n$ is the (orientation-preserving) isometry group of $M$. It would be great if you can provide an explicit construction of $M$, $\forall n$. If the question $\forall n$ is too broad, I am mainly interested in the low-dimensional cases of $n=3,4,5,6$. I am aware that for $n=2$, $SU(2)/\mathbb Z_2\cong SO(3)$, which is the (orientation-preserving) isometry group of $S^2$.
The existence of above space $M$ is guaranteed as follows: $SU(n)$ is a compact topological group, and $\mathbb Z_n$ is the center of $SU(n)$, so $SU(n)/\mathbb Z_n$ is a well-defined quotient group. Quotient map preserves compactness. So $SU(n)/\mathbb Z_n$ is a compact group. Finally, this MathOverflow answer states that "every compact group is the full isometry group of a compact Riemannian manifold."
P.S. For you to gauge the level of details of answer: I am a physicist by training and I am not too well-versed in Riemannian geometry (just the basics).
And it would be great if you can provide relevant references.