Prove that $\text{rank}(X^TX)=\text{rank}(X)$

Question

Prove, for real $X$, that $\text{rank}(X^TX)=\text{rank}(X)$.

Could anyone please help me with this problem? I've tried to use full-rank factorization and rank-related theorems mentioned in my book but still failed to solve this. I am learning linear algebra by myself and my book has no solutions manual so I find it really hard to get used to solve linear algebra problems.

Many thanks in advance for your help!

score 7 · Answer 1 · answered Jun 04 '18 at 16:37

7

Consider the map $T: Im(X)\to Im(X^TX)$ given by $T(w)=X^Tw$. We show that $T$ is an isomorphism.

$T$ is obviously onto, since given any $v\in Im(X^TX)$, we have $v=X^TXv'=X^T(Xv')=T(Xv')$ for some $v'$.

It remains to check that $T$ is one-to-one.

Suppose $T(w)=0$. We have $w=Xv$ for some $v$, so $X^TXv=0$. In particular, $(X^TXv, v)=0$. Thus $(Xv, Xv)=0$, so by positivity o the inner product, we conclude that $Xv=w=0$ as desired.

answered Jun 04 '18 at 16:37

Simon Segert

5,819
11
22

Thank you so much for your answer! Unfortunately, I haven't come to the section that mentioned in your solution (map, isomorphism...) yet. I would come back to your solution later when I learn those materials. – Sophil Jun 04 '18 at 16:57
I don't understand this answer. Are you assuming X is square and has full rank? – ThighCrush Sep 23 '22 at 08:48
@ThighCrush no, there is no assumption about X – Simon Segert Sep 23 '22 at 11:59
My bad, I understand now – ThighCrush Sep 26 '22 at 08:05

score 7 · Accepted Answer · edited Apr 26 '23 at 17:30

Assume $u$ is in the null space of $\mathbf{X}^T\mathbf{X}$.
Then $\mathbf{X}^T\mathbf{X}u=0 \implies u^T\mathbf{X}^T\mathbf{X}u = u^T0=0$.

Then assume, $u$ is not in the null space of $\mathbf{X}$.
Then $\mathbf{X}u=v$, (notice $u^T\mathbf{X}^T = v^T$).

But $u^T\mathbf{X}^T\mathbf{X}u = 0 \implies v^Tv=0$, which implies $v=\mathbf{X}u=0$.

Thus, $null(\mathbf{X}^T\mathbf{X})\subset null(\mathbf{X})$.

Proving the reverse inclusion is trivial.

So, $null(\mathbf{X}^T\mathbf{X}) = null(\mathbf{X})$.

By the rank and nullity theorem (Nullity of A + Rank of A = Total number of columns in A),
rank$(\mathbf{X}^T\mathbf{X})$ = rank$(\mathbf{X})$.

Thank you so much for your help! I can now finally move on to other problems :). — Sophil, Jun 04 '18 at 17:00

score 0 · Answer 3 · answered Apr 26 '23 at 18:29

Lord KK above provided a sufficient answer. I will provide another approach which requires the knowledge of spectral decomposition, singular value decomposition (SVD) and the following lemma.

Lemma: For a full rank matrix $\mathbf{A}_{s\times s}$ and any $\mathbf{B}_{s\times t}$, $rank(\mathbf{A}\mathbf{B}) = rank(\mathbf{B})$.

$$\begin{aligned} rank(\mathbf{X}^T\mathbf{X}) & = \text{number of non-zero eigenvalues of }\mathbf{X}^T\mathbf{X} \\ & = \text{number of non-zero singular values of }\mathbf{X} \\ & \overset{*}{=} rank(\mathbf{X}), \end{aligned}$$

where the final equality follows from the definitions of SVD and the lemma, $$\begin{aligned} rank(\mathbf{X}) & = rank(\mathbf{U} \mathbf{D} \mathbf{V}^T) \\ & = rank( \mathbf{D} \mathbf{V}^T) \ \ \ \ (\text{Lemma})\\ & = rank( \mathbf{D}) \ \ \ \ \ \ \ \ \ \ (\text{Lemma})\\ & = \text{number of non-zero singular values of }\mathbf{X} \\ \end{aligned}$$

Prove that $\text{rank}(X^TX)=\text{rank}(X)$

3 Answers3

Linked

Related