12

I was going through Boyd & Vandenberghe's Convex Optimization. On page 38, the authors mentioned that the solution set of a linear matrix inequality (LMI) is convex.

$$ A(x) := x_1 A_1 + \dots + x_n A_n \preceq B $$

where $A_1, \dots, A_n, B \in \mathbb{S}^m$, is called an LMI in $x$. They also gave a brief explanation where they mentioned that this is because

it is the inverse image of the positive semi-definite cone under the affine function.

I could not figure out what would be the affine function that they mentioned.

Rodrigo de Azevedo
  • 23,223
Rajat
  • 2,512
  • 2
    $ x \mapsto B - x_1 A_1 - \cdots - x_n A_n$ – littleO Sep 11 '15 at 07:15
  • Yes, that I understand. Actually, I want to know the function which maps a vector to a Matrix. Like what matrix or vector operation we have to do to find such kind of function. – Rajat Sep 11 '15 at 07:17
  • Hmm, the affine function I mentioned does map a vector to a matrix, so I'm not sure I understand your question. – littleO Sep 12 '15 at 01:27
  • Sorry for the ambiguity in my question. I am looking for an expression like, $T(x)=B-A@x$ where $A$ is some matrix consists of $A_1,\ A_2,\ \dots \ , \ A_n$, $x$ is a vector in $\mathbb{R}^n$ consists of $x_1,\ x_2,\ \dots \ , \ x_n$, and $@$ is some matrix operation. I want to find this $A, \ x \ & \ @$, and also I would like to known whether is it feasible to write the expression like this? – Rajat Sep 12 '15 at 03:33
  • If you think of a matrix as being identified with a big column vector, then the first column of $A $ is $ A_1$, etc, and the operation is just matrix multiplication. – littleO Sep 12 '15 at 05:55
  • If we are doing matrix multiplication, then how to match the dimension of the matrices? – Rajat Sep 12 '15 at 10:38
  • I added some details to try to clarify how the dimensions match up. – littleO Sep 12 '15 at 20:41
  • The assumtion that every matrix is symmetric seems necessary as explained by @Rodrigo, but I can't see why we cannot use the arguments in the book for every set $A_1,\cdots,A_n$ of squared matrices, since $f(x)$ is still an affine function. – Dan Feldman Mar 06 '23 at 15:04

3 Answers3

12

The affine function is $T(x) = B - x_1 A_1 - \cdots - x_n A_n $.

The solution set to your LMI can be described as \begin{equation} \{ x \mid T(x) \succeq 0 \} = T^{-1}(S^m_+), \end{equation} where $S^m_+$ is the positive semidefinite cone in $\mathbb R^{m\times m}$.

Further details:

If we view $A_1,\ldots,A_n$ and $B$ as column vectors in $\mathbb R^{m^2}$, then \begin{equation} T(x) = \underset{\substack{\Bigg \uparrow \\m^2 \times 1}}{B} - \underset{\substack{\Bigg \uparrow \\ m^2 \times n}}{A} \underset{\substack{\uparrow \\ n \times 1}}{x} \end{equation} where \begin{equation} A = \begin{bmatrix} A_1 & A_2 & \cdots & A_n \end{bmatrix}. \end{equation} In this equation, the $A$ is multiplied by $x$ using ordinary matrix multiplication.

littleO
  • 54,048
3

Given a linear matrix polynomial $\mathrm A : \mathbb{R}^{n} \to \mbox{Sym}_k (\mathbb{R})$ defined by

$$\mathrm A (x) = \mathrm A_0 + x_1 \mathrm A_1 + \cdots + x_n \mathrm A_n$$

where $\mbox{Sym}_k (\mathbb{R})$ is the set of $k \times k$ real symmetric matrices, and matrices $\mathrm A_0, \mathrm A_1, \dots, \mathrm A_n$ are symmetric, we form the following linear matrix inequality (LMI)

$$\mathrm A (\mathrm x) \succeq \mathrm O_k$$

whose solution set is the spectrahedron

$$\mathcal S := \{ \mathrm x \in \mathbb{R}^n \mid \mathrm A (\mathrm x) \succeq \mathrm O_k\}$$

Let $\mathrm x, \mathrm y \in \mathcal S$ and $\lambda \in [0,1]$. Hence,

$$\mathrm A (\lambda \mathrm x + (1-\lambda) \mathrm y) = \cdots = \lambda \, \underbrace{\mathrm A (\mathrm x)}_{\succeq \mathrm O_k} + (1-\lambda) \, \underbrace{\mathrm A (\mathrm y)}_{\succeq \mathrm O_k} \succeq \mathrm O_k$$

Since a convex combination of two positive semidefinite matrices is also positive semidefinite, we conclude that set $\mathcal S$ is convex.

Rodrigo de Azevedo
  • 23,223
0

While the book defines a real-valued affine function $f:R^d\to R$, in this example there is a hidden assumption/generalization to a function $g:R^d\to R^{d\times d}$. It also assumed that all the properties of real-valued affine functions still hold.

This question, as similar questions, bothered me when reading the book. They are related to the fact that, in many places, this wonderful book defines a property of a real-valued function and then, at some point, apply it for functions that map to other spaces, like matrices, or cones. This is without a disclaimer (Say, "easy generalization of.. that we leave to the reader".) In this case, it is affine functions, common other examples are with respect to convexity .

The other answers here also use a definition of affine functions for matrix-valued function which I could not find before the relevant LMI definition.

Dan Feldman
  • 454
  • 2
  • 9