Find $M$ with unit columns such that $M^T S M$ is a scaled identity matrix for a diagonal $S$ with unique entries

Question

Suppose $S = {\rm diag}(s_1, s_2, ...)$ is a diagonal matrix with unique positive entries $s_i$. I need to find a square matrix $M$ such that $D = M^T S M$ is a diagonal matrix with identical entries, i.e. $D = d I$. There is an additional requirement that the columns of $M$ must be unit vectors. How do I find such a matrix $M$ and the corresponding value $d$ in the general case?

For $n=2$, I can find the solution as \begin{align} M = \begin{bmatrix} \sqrt{\frac{s_2}{s_1 + s_2}} & \sqrt{\frac{s_2}{s_1 + s_2}} \\ \sqrt{\frac{s_1}{s_1 + s_2}} & -\sqrt{\frac{s_1}{s_1 + s_2}} \end{bmatrix} \end{align} and the corresponding constant $d$ is \begin{align} d = \frac{2 s_1 s_2}{s_1 + s_2}. \end{align} Notice that the columns of $M$ are not perpendicular because $s_1 \ne s_2$.

In the general $n$-dimensional case, there are $n(n+1)/2$ constraints from the condition $D = M^T S M$ (due to symmetry). The number of unknowns is $n(n-1) + 1$ since $n-1$ parameters are needed to specify a unit vector in $n$ dimensions. And "+1" is for the unknown diagonal value $d$.

When $n=2$ (the case above), we have $n(n+1)/2 = 3$ and $n(n-1) + 1 = 3$. So the found solution is unique, except for sign flips. For $n>2$, the number of unknowns is greater than the number of constraints, so I expect a family of solutions will be available. Is it possible to find the form that this family of solutions must follow?

Answer 1

If $M^TSM=dI$, then $S^{1/2}M=d^{1/2}Q$ for some orthogonal matrix $Q$. Therefore $M=d^{1/2}S^{-1/2}Q$. In order that each column of $M$ is a unit vector, $M^TM=dQ^TS^{-1}Q$ must have a diagonal of ones. Hence we must have $d\operatorname{tr}(S^{-1})=n$ where $S$ is $n\times n$. That is, $d=\frac{n}{\operatorname{tr}(S^{-1})}$.

Let $Z$ be the traceless matrix $\frac{nS^{-1}}{\operatorname{tr}(S^{-1})}-I_n$. The problem now boils down to finding an orthogonal matrix $Q$ such that $Q^TZQ$ has a zero diagonal.

If the underlying field is complex and we want to find $M$ and $Q$ such that $Q^\ast ZQ$ has a zero diagonal and $M^\ast SM=dI$, we may simply take $Q$ as the DFT matrix regardless of the value of $Z$. Over the real field, however, $Q$ in general depends on $Z$ and I am not aware of any explicit formula for $Q$ when $n\ge3$. Yet, a feasible $Q$ can always be obtained by iteratively conjugating $Z$ with Givens rotations. This has been discussed in another answer of mine. The iterations will end in at most $n$ steps.