13

I'm trying to get some intuition behind the meaning of a positive semidefinite matrix, which I learned a long time ago in undergrad but clearly didn't internalize properly.

As I understand, a symmetric matrix $M \in \textbf{R}^{n~\times~n}$ is positive semidefinite iff $z^TMz \ge 0$, $\forall z \in \textbf{R}^n$. Note that I'd like to use this particular definition, not a more general one that involves complex numbers. As such, $z^TMz \in \textbf{R}$.

This definition makes sense to me, and this question clarified it further, but then I was reading Boyd's textbook and became confused by an unrelated definition explained in $\S$3.1.4, which implies that the Hessian matrix $\textbf{H}$ of function $f$ is positive semidefinite if $\textbf{H} \succcurlyeq 0$, where the $"\succcurlyeq"$ symbol refers to a componentwise inequality between matrices.

Thus, can a positive semidefinite matrix contain negative entries?

EDIT: This question turned out to be silly, but if you have this question and am as rusty with linear algebra as I am, this post might be useful.

  • 4
    A quick and dirty way to see that there can be some negative entries is to think of adding a large positive multiple of the identity to a completely arbitrary symmetric matrix. This shifts all the eigenvalues up as much as you want without breaking the symmetry. – Ian Jul 14 '17 at 18:10
  • 2
    Also notice that your definition of "positive semidefinite" is wrong. In fact, there is no n×n matrix M such that x^T M x > 0 for all vectors x. Indeed, trivially x^T M x = 0 if x is the n-dimensional zero vector. The correct definition is that M is positive semidefinite iff x^T M x >= 0 for all x. If, in addition, equality only happens for the zero vector, we say that M is positive definite. Then x^T M x > 0 for all non-zero vectors x. – Andreas Rejbrand Jul 14 '17 at 19:58
  • 1
    @AndreasRejbrand It was a typo - I fixed it. –  Jul 14 '17 at 20:09

2 Answers2

18

Can a positive semidefinite matrix contain negative elements? If by elements, you mean entries, then yes, $$\pmatrix{2&-1\\-1&2}$$ is positive definite.

Angina Seng
  • 161,540
  • 4
    Any diagonally dominant symmetric matrix is positive definite. So there are positive definite matrices of any size, where every off diagonal term is negative. – alephzero Jul 14 '17 at 19:22
  • 2
    @alephzero: That is not right. You also need the right sign on the diagonal entries (of course). – Andreas Rejbrand Jul 14 '17 at 20:04
13

The symbol $\succeq$ actually does not denote componentwise inequality for matrices in that book. If $A$ and $B$ are symmetric matrices then $A \succeq B$ means $A-B$ is positive semidefinite. Perhaps confusingly, if $x$ and $y$ are vectors rather than matrices, then in that case $x \succeq y$ does mean that each component of $x$ is greater than or equal to the corresponding component of $y$.

(As Lord Shark the Unknown mentioned, a positive semidefinite matrix can have some negative entries.)

Here are a few more details. On p. 43, Boyd and Vandenberghe introduce the notation $x \preceq_K y$ (where $K$ is a proper cone) to mean that $y - x \in K$. If $K$ is the nonnegative orthant then $x \preceq_K y$ means that $y - x$ is in the nonnegative orthant, or in other words that $y_i \geq x_i$ for all $i$. If $K$ is the positive semidefinite cone then $A \preceq_K B$ means that $B - A$ belongs to the positive semidefinite cone, or in other words that $B - A$ is positive semidefinite.

The book goes on to state

The nonstrict and strict partial orderings associated with the nonnegative orthant arise so frequently that we drop the subscript $\mathbb R^n_+$; it is understood when the symbol $\preceq$ or $\prec$ appears between vectors.

Regarding the case where $K$ is the positive semidefinite cone, the book comments

Here, too, the partial ordering arises so frequently that we drop the subscript.

littleO
  • 54,048