1

i have a convex optimization problem to find $M$ as a $n\times n$ matrix and $x,y,z,s$ are data vectors and all are $n\times 1$:

$$\min_M f(M,x,y,s,z) $$ $$s.t: x^TMy+z^TMz-s^TMs <0$$ $$M>0$$

The problem is that the above constraint is not feasible at all for some data samples, and those instances of data should be assumed as outlayers and not to be considered in the optimization problem.

Then i will face another problem which is checking the feasibility of the constraint for each data sample batch {x,y,z,s} to see regardless of the objective function $f(.)$ the quadratic constraint can ever be negative or not.

So is there any way to find out that sort of outlayers in a simple way?

Bob
  • 700
  • Does "M > 0" mean each entry is positive, or that it's a positive-definite matrix, (i.e., all eigenvalues are positive reals)? – John Hughes Sep 29 '16 at 14:10
  • Actually $M$ is originally as $M=L^TL$ and i'm interested in $L$ at the end. so i think $M$ should be p.s.d ideally. However i'm still interested to see how to find the feasibility of the constraint even assuming $M>0$ as just positive entries. – Bob Sep 29 '16 at 14:19

1 Answers1

0

The constraint is feasible if $\frac{1}{2}(yx^T+xy^T) + zz^T - ss^T$ is negative semidefinite.

You are interested in $\inf_M\{ x^TMy + z^TMz - s^T M s : M > 0 \} < 0$. This is optimizing a linear function over the cone of positive definite matrices. There is much software out there that can solve this problem for you (sedumi, sdpa, sdpt3 just to name a few), but since this problem is unconstrained, the solution is trivial. Rewrite the objective to $tr((yx^T + zz^T - ss^T)M)$. Now if the matrix $\frac{1}{2}(yx^T+xy^T) + zz^T - ss^T$ is positive semidefinite, the $\inf$ is $0$, otherwise it's $-\infty$.

LinAlg
  • 20,093
  • i calculated the $yx^T + zz^T - ss^T$ , but the resulting matrix has complex eigen-values, so i calculated the eigenvalues of its Hermitian matrix. Then for infeasible examples there was some positive and some negative eigen-values, although the value of negative ones sometimes were around 0.04 of positive ones. And for infeasible cases the eigenvalues of the Hermitian were all positive with few nearly zero ones. So is it reliable to use this result to distinguish between the infeas/feasable cases? – Bob Sep 29 '16 at 19:26
  • Thanks for pointing this out, I have updated my answer. – LinAlg Sep 29 '16 at 19:34
  • yes now the result is perfectly matching. I really liked this mathematical way of handling the problem. By the way is there any linear algebra documentation so i can read to understand the reason behind this relationship between the inf and the above matrix? – Bob Sep 29 '16 at 19:52
  • I used that $x^TMy = tr(x^TMy) + tr(yx^TM)$. The final step follows from the fact that the cone of positive semidefinite matrices is self-dual. – LinAlg Sep 29 '16 at 20:38
  • Unfortunately the 2nd part is not so clear to me, could you please explain a bit more or link a reference about it? – Bob Sep 29 '16 at 21:54
  • Pages 1 and 2 of http://www.math.ucsb.edu/~atzberg/finance/notesDuality.pdf give an intro. Essentially, ${ A : tr(AM) \geq 0 }$ is the set of psd matrices. Hence, if A is psd, $\inf_M tr(AM)$ is 0, and if A is not psd, it is $-\infty$. – LinAlg Sep 29 '16 at 22:20
  • Based on some try with different sets of vector, i noticed that in comparison between two sets of vectors ${x,y,z,s}i$ and ${x,y,z,s}_j$, we can consider $r=\frac{\left|\lambda{-}\right|}{\left| \lambda_{+} \right|}$ which is the absolute value of most negative eigenvalue comparing to the most positive one. Then if $r_{i}>r_{j}$ then the solution area for set_i is bigger than set_j. So i'm curious to know if there is a way to mathematically show the above conclusion? – Bob Oct 11 '16 at 16:16
  • i meant between ${x,y,z,s}_i$ and ${x,y,z,s}_j$. – Bob Oct 11 '16 at 16:20
  • Please ask a new question where you properly define the sets and the relation "bigger than". – LinAlg Oct 11 '16 at 16:32
  • i did: here – Bob Oct 11 '16 at 23:32