16

My professor always writes on the board:

$A$ is $m \times n$, assuming that the vectors of $A$ form a basis, then $A^TA$ is always invertible.

one thing I know is that $A^TA$ is always symmetric, but I'm not sure about the conditions on a symmetric matrix needed to ensure that it is invertible?

Widawensen
  • 8,517
makansij
  • 1,643
  • 3
    What do you nean by "the vectors for $A$"? – Robert Lewis Jul 09 '17 at 17:38
  • Sorry, should be "vectors of A" - it's my english. – makansij Jul 09 '17 at 17:49
  • 1
    There is a long list of conditions that are equivalent to the condition that a matrix is invertible, all of which are equally valid for symmetric matrices. Are you instead asking for conditions on a matrix $A$ to ensure that $A^TA$ is invertible? – JMoravitz Jul 09 '17 at 17:55
  • 1
    The words you need are "row" and "column." With $m$ rows and $n$ columns, we find $A^T A$ is a square of size $n.$ When $m \geq n$ and the $n$ columns of $A$ are independent, then $A^T A$ is also of rank $n,$ therefore invertible. – Will Jagy Jul 09 '17 at 17:56
  • @JMoravitz yes, sorry perhaps mis-asked the question, thank you. – makansij Jul 10 '17 at 05:27
  • and @WillJagy usually "vector" implies column vector, but i shoulda been more explicitthanks. – makansij Jul 10 '17 at 05:29
  • If $A$ forms a basis, then it is full-rank. A matrix and its transpose have the same rank. Then $A^TA$ is full-rank. –  Jul 11 '17 at 10:15
  • @YvesDaoust "If $A$ forms a basis, then it is full rank...." That is not generally true. [0,1,0,0] and [1,0,0,1] can both be basis vectors. but obviously a matrix composed of those vectors will not be full rank. – makansij Jul 15 '17 at 23:03
  • And, furthermore, if $A$ is composed of those two vectors I just mentioned ([0,1,0,0] and [1,0,0,1]) , then $A^TA$ is symmetric but not invertible. – makansij Jul 15 '17 at 23:19

2 Answers2

11

A sufficient condition for a symmetric $n\times n$ matrix $C$ to be invertible is that the matrix is positive definite, i.e. $$\forall x\in\mathbb{R}^n\backslash\{0\}, x^TCx>0.$$

We can use this observation to prove that $A^TA$ is invertible, because from the fact that the $n$ columns of $A$ are linear independent, we can prove that $A^T A$ is not only symmetric but also positive definite.

In fact, using Gram-Schmidt orthonormalization process, we can build a $n\times n$ invertible matrix $Q$ such that the columns of $AQ$ are a family of $n$ orthonormal vectors, and then: $$I_n=(AQ)^T (AQ)$$ where $I_n$ is the identity matrix of dimension $n$.

Get $x\in\mathbb{R}^n\backslash\{0\}$.

Then, from $Q^{-1}x\neq 0$ it follows that $\|Q^{-1}x\|^2>0$ and so: $$x^T(A^TA)x=x^T(AI_n)^T(AI_n)x=x^T(AQQ^{-1})^T(AQQ^{-1})x \\ = x^T(Q^{-1})^T(AQ)^T(AQ)(Q^{-1}x) = (Q^{-1}x)^T\left((AQ)^T(AQ)\right)(Q^{-1}x) \\ = (Q^{-1}x)^TI_n(Q^{-1}x) = (Q^{-1}x)^T(Q^{-1}x) = \|Q^{-1}x\|^2>0.$$ Being $x$ arbitrary, it follows that: $$\forall x\in\mathbb{R}^n\backslash\{0\}, x^T(A^TA)x>0,$$ i.e. $A^TA$ is positive definite, and then invertible.

Bob
  • 5,995
  • 5
    Why do you need to use Gram-Schmidt? You can argue directly that $x^T A^T Ax = (Ax)^T (Ax) = |Ax|^2$, and the RHS is strictly positive for all nonzero $x$ provided that $A$ has trivial null space (or equivalently, that $A$ has full column rank). –  Aug 27 '18 at 16:04
  • What is the connection between positive definiteness and the determinant being different from zero? – asd11 Jan 20 '19 at 10:47
  • 1
    @asd11 eigen values > 0, which implies $\prod \lambda = det A > 0$ – user1827356 Apr 25 '21 at 15:23
1

@RobertLewis

A Gram matrix is usually defined by giving a set of vectors and then defining the i,j entry as the dot product of the i,j vectors. In doing so, clearly the set of vectors can be thought of as column vectors of A. So saying "the vectors for A" is a completely natural thing to say, and should be unambiguous.

here is an elegant proof Gram matrix invertible iff set of vectors linearly independent

Randy
  • 19
  • you seem to be saying that the ONLY condition needed on $A$ for $A^TA$ to be invertible is if the column vectors of $A$ are linearly independent. Please look at the example I gave to YvesDaoust: if A is composed of those two vectors I just mentioned $([0,1,0,0], [1,0,0,1])$ , then $A^TA$ is symmetric but not invertible. So, I think that your claim is not entirely true. – makansij Jul 17 '17 at 00:41
  • 2
    In your "counterexample", $A^TA = diag(1, 2)$ is most certainly invertible.... – Randy Dec 19 '17 at 21:04
  • 1
    @Randy I think nundo probably meant to put a transpose on those two vectors, so that A is a 4x2 matrix, not 2x4. – Confounded May 10 '18 at 17:49
  • yes that's correct. thanks – makansij Sep 15 '18 at 23:59