3

For a software project I'm involved on, I have a situation where I have a large vector that is the sum of some smaller vectors. I know all the possible small vectors, and I know that no two of them start at the same place. Therefore, I think I can solve the problem (determine which small vectors are in the large one) if I put each small vector into each possible position as columns in a large matrix $A$. If $b$ is the signal (a particular sum of parts), I would need to solve this: $$ \min \|x\|_1 \text{ where } Ax = b \text{ and } x_i\in\{0,1\} \text{ for all } i. $$ From what I've read from Stanford, I should be able to get the solution that minimizes the sum of x using $A^T (AA^T)^{−1}b$. Indeed that solution seems to have, um, hills where I should have a 1. However, it is not what I would call a sparse solution. And there is no way for me to threshold the data to get the right answer as it varies too much. Any ideas on how to solve this more effectively?

Here's an example:

Suppose I have two small vectors: $[3, 4, 5]$ and $[2, -1, -2]$. Suppose that the first starts at position 0 and 3 and the second starts at position 2 in the final signal. You would then have this sum/signal/$b$: $[3, 4, 7, 2, 2, 5]$. The $A$ in this scenario would be drawn up like this:

$$ \left[\begin{array}{rrrrrr} 3 & 0 & 0 & 0 & 2 & 0 & 0 & 0 \\ 4 & 3 & 0 & 0 &-1 & 2 & 0 & 0 \\ 5 & 4 & 3 & 0 &-2 &-1 & 2 & 0 \\ 0 & 5 & 4 & 3 & 0 &-2 &-1 & 2 \\ 0 & 0 & 5 & 4 & 0 & 0 &-2 &-1 \\ 0 & 0 & 0 & 5 & 0 & 0 & 0 &-2 \end{array}\right] $$

I want to perform some kind of solve operation to return me this $x$: $[1, 0, 0, 1, 0, 0, 1, 0]$.

Rodrigo de Azevedo
  • 23,223
Brannon
  • 187
  • 1
    What do you mean by "no two of them start at the same place"? If you mean each vector looks like $(0,\dots,0,1,\dots)$, and no two have the same number of zeros before that first 1, then you can just order your vectors lexicographically, and match them up to your target vector, one component at a time. But maybe you mean something else. – Gerry Myerson Oct 06 '13 at 04:13
  • What do you mean by a "large" vector and "smaller" vectors? And what do you mean when you say that no two of the smaller vectors start at the same place? – littleO Oct 06 '13 at 04:14
  • 2
    There's no closed form solution for the problem of minimizing $|x|_1$ subject to $Ax = b, x_i \in {0,1} , \forall i$. You could try relaxing the second constraint and minimizing $|x|_1$ subject to $Ax = b, 0 \leq x_i \leq 1 , \forall i$. This is a convex problem and could be solved with CVX in Matlab or with some other software. – littleO Oct 06 '13 at 04:23
  • 1
    The question title and question body don't match. What you want to minimise is not $|Ax-b|$ --- $|Ax-b|$ is required to be zero in your constraint! You are minimising the 1-norm of $x$ subject to the constraint $Ax=b$. – user1551 Oct 06 '13 at 07:09
  • 1
    @GerryMyerson, I have added an example that should help answer your question. – Brannon Oct 06 '13 at 18:27
  • @littleO, I have read a good portion of Boyd's book and seen the CVX library that goes with it. However, I don't have access to Matlab, and the library does not work with Octave, unfortunately. I have used the IPOPT C++ framework some. I was hoping to solve it without some kind of iterative approach. – Brannon Oct 06 '13 at 18:29
  • @user1551, in Boyd's Convex Optimization book, he regularly changes notation from minimizing |x| s.t. Ax = b to minimizing |Ax - b| s.t. x being sparse. It seems that they are commonly declared equivalent. – Brannon Oct 06 '13 at 18:31
  • Is it the case that no two of the small vectors have the same first component? In your example, one of the small vectors has a 3 as its first component, the other has a 2; is it always like that, that the first components are different? – Gerry Myerson Oct 06 '13 at 22:13
  • 1
    Looking at your example, it's almost (but not quite) as if $Ax = A_1 x_1 + A_2 x_2$, where $x = \begin{bmatrix} x_1 \ x_2 \end{bmatrix}$ (using block notation), and $A_1$ convolves with the kernel $\begin{bmatrix} 3 & 4 & 5\end{bmatrix}^T$ and $A_2$ convolves with the kernel $\begin{bmatrix} 2 & -1 & -2 \end{bmatrix}^T$. I wonder if it might be helpful to look at it that way. – littleO Oct 06 '13 at 23:49

2 Answers2

3

This is an instance of the multi-objective knapsack problem, which is NP-complete. You have a little bit of extra structure, since many of your "items" have the same form, but I have doubts that this will be enough to find an efficient (polynomial-time) solution.

user7530
  • 50,625
0

This is a 0-1 integer linear programming problem listed in 21 Karp's NP-complete problems. Polynomial-time algorithms to solve this problem are not known, but there are many practical efficient methods (Branch and [Bound, Cut, Price], Cutting Planes, Benders Decomposition, Column Generation, and its variants).

Check your objective function:

$$||x||_1 = \overbrace{\sum_{i=1}^{n} |x_i| = \sum_{i=1}^{n} x_i}^{x_i\in \{0,1\}\implies |x_i|=x_i }$$

your model is only

$$ \begin{align} \min & \sum_{i=1}^{n} x_i&\\ s.t.& ~&\\ & \sum_{i=1}^{n} a_{ji}x_i = b_j & \forall j\in I_m \end{align} $$

this 0-1 linear integer programming seems the multidimensional knapsack problem having strict equalities. Probably, you need to modify something in a pseudo-polynomial time algorithm introduced in this book Knapsack Problems.

Alexandre Frias
  • 462