Simultaneous diagonlisation of two quadratic forms, one of which is positive definite

Question

Let $\varphi, \phi$ be quadratic forms on $V$ and suppose $\varphi$ is positive definite.

I want to find a basis for V such that $\varphi$ and $\phi$ are both represented by diagonal matrices.

My idea is to define an inner product $<,>: V\times V \rightarrow F$ where $<v,w> = \varphi(v,w)$.

I know that if I can find a basis that is orthonormal w.r.t. this inner product that diagonalises $\phi$, then I am done, since $\varphi$ will be represented by the identity with respect to this basis.

I know that I can use Gram-Schmidt to get an orthonormal basis for $V$, but I don't understand how to choose a basis that diagonalises $\phi$.

I realised I didn't understand what I was doing when I tried the example where $\phi$ is the symmetric bilinear form associated to $2x^2 + 3y^2 +3z^2 - 2yz = (\sqrt{3}z - \frac{1}{\sqrt{3}}y)^2 + 2x^2 + \frac{8}{3}y^2$ which is positive definite, and wish to simultaneously diagonalise this and $\phi$ which is the symmetric bilinear form associated to the quadratic form $3x^2 + 3y^2 + z^2 +2xy - 3xz + 3yz$.

Help in general, or with relevance to this particular example, gratefully received.

Answer 1

Let $\varphi$ denote the positive definite form. Let $\psi$ denote the second form. I'll assume you're working over $F = \Bbb R$; for $F = \Bbb C$ you'd have to clarify whether we're considering a "sesquilinear" quadratic form.

If $A$ is the matrix of $\psi$ relative to a $\varphi$-orthonormal basis, you question amounts to asking whether there exists an orthogonal matrix $U$ such that $U^TAU$ is diagonal. By the spectral theorem, this is possible whenever $A$ is symmetric. Since $A$ is the matrix associated with a quadratic form, it is always possible; $A$ is necessarily symmetric.

In order to find such a basis, it suffices to find an orthonormal eigenbasis of $A$. That is: from our bilinear form $\psi$, we get a linear transformation $T_\psi$. Once we select mutually orthogonal eigenvectors of $T_\psi$, we have the columns of our desired $U$. To that end, it suffices to find any basis eigenvectors, then apply the Gram-Schmidt process.

Answer 2

Ref: "Foundations of Optimization" by Guler.

Thm [Simultaneous diagonalisability of quadratic forms]
Let ${ q _1 (x) := x ^T A x, }$ ${ q _2 (x) = x ^T B x }$ be quadratic forms on ${ \mathbb{R} ^n }$ with ${ q _1 }$ positive definite.

That is, let ${ A, B \in \mathbb{R} ^{n \times n} }$ be symmetric matrices with ${ A }$ positive definite.

Then there exists a basis ${ P = [P _1. \ldots, P _n] }$ of ${ \mathbb{R} ^n }$ such that

$${ {\begin{align} &\, q _1 \left (\sum _{i = 1} ^n P _i y _i \right) = \sum _{i = 1} ^n y _i ^2 \\ &\, q _2 \left( \sum _{i = 1} ^n P _i y _i \right) = \sum _{i = 1} ^n \lambda _i y _i ^2 . \end{align}} }$$

That is, there exists an invertible matrix ${ P \in \mathbb{R} ^{n \times n} }$ such that ${ P ^T A P = I }$ and ${ P ^T B P = \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

Pf: By spectral theorem for the positive definite matrix ${ A , }$

$${ A U = U \text{diag}(\mu _1, \ldots, \mu _n) }$$

with ${ U \in \mathbb{R} ^{n \times n} }$ orthonormal and ${ \mu _1 \geq \ldots \geq \mu _n > 0 . }$

Defining ${ P _1 := U \text{diag}(\mu _1 ^{- 1/2}, \ldots, \mu _n ^{-1/2}) }$ we have ${ P _1 ^T A P _1 = I . }$

The goal is to modify the invertible matrix ${ P _1 }$ satisfying ${ P _1 ^T A P _1 = I }$ to an invertible matrix ${ P _2 = P _1 \tilde{P} }$ satisfying ${ P _2 ^T A P _2 = I }$ and ${ P _2 ^T B P _2 = \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

It suffices to find an orthonormal matrix ${ \tilde{P} }$ such that ${ (P _1 \tilde{P}) ^T B (P _1 \tilde{P}) = \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

It suffices to find an orthonormal basis ${ \tilde{P} }$ such that ${ (P _1 ^T B P _1 ) \tilde{P} = \tilde{P} \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

By spectral theorem for the symmetric matrix ${ P _1 ^T B P _1 , }$ we have an orthonormal basis of eigenvectors ${ \tilde{P} , }$ as needed. ${ \blacksquare }$

The same approach works for simultaneous diagonalisability of a quadratic form and a symmetric operator wrt the quadratic form.

Thm [Spectral theorem]
Let ${ q (x) := x ^T A x }$ be a positive definite quadratic form on ${ \mathbb{R} ^n }$ and let ${ B \in \mathbb{R} ^{n \times n} }$ be a symmetric operator wrt the quadratic form.

That is, let ${ A , B \in \mathbb{R} ^{n \times n} }$ with ${ A }$ a symmetric positive definite matrix and ${ B }$ such that ${ \langle B x, y \rangle _A = \langle x, B y \rangle _A }$ that is ${ B ^T A = A B . }$

Then there exists a basis ${ P = [P _1, \ldots, P _n] }$ of ${ \mathbb{R} ^n }$ such that

$${ {\begin{align} &\, q \left( \sum _{i = 1} ^n P _i y _i \right) = \sum _{i = 1} ^n y _i ^2 \\ &\, B P = P \text{diag}(\lambda _1, \ldots, \lambda _n) \end{align}} }$$

That is, there exists an invertible matrix ${ P \in \mathbb{R} ^{n \times n} }$ such that ${ P ^T A P = I }$ and ${ B P = P \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

Pf: By Gram-Schmidt orthogonalisation construct an orthonormal basis wrt ${ q . }$ Hence we have an invertible matrix ${ P _1 }$ such that ${ P _1 ^T A P _1 = I . }$

Now the constraint ${ B ^T A = A B }$ becomes ${ B ^T (P _1 ^{-1}) ^T P _1 ^{-1} = (P _1 ^{-1} ) ^T P _1 ^{-1} B }$ that is ${ P _1 ^T B ^T (P _1 ^{-1}) ^T = P _1 ^{-1} B P _1 }$ that is ${ P _1 ^{-1} B P _1 }$ is a symmetric matrix.

The goal is to modify the invertible matrix ${ P _1 }$ satisfying ${ P _1 ^T A P _1 = I }$ to an invertible matrix ${ P _2 = P _1 \tilde{P} }$ satisfying ${ P _2 ^T A P _2 = I }$ and ${ B P _2 = P _2 \text{diag}(\lambda _1, \ldots, \lambda _n) . }$

It suffices to find an orthonormal matrix ${ \tilde{P} }$ such that ${ B P _1 \tilde{P} = P _1 \tilde{P} \text{diag}(\lambda _1, \ldots, \lambda _n). }$

It suffices to find an orthonormal basis ${ \tilde{P} }$ such that ${ (P _1 ^{-1} B P _1) \tilde{P} = \tilde{P} \text{diag}(\lambda _1, \ldots, \lambda _n). }$

By spectral theorem for the symmetric matrix ${ P _1 ^{-1} B P _1, }$ we have an orthonormal basis of eigenvectors ${ \tilde{P} , }$ as needed. ${ \blacksquare }$