7

For a bunch of items with values $v_i$ and weights $w_i$, and with a total weight $W$ that our bag can carry, how do we achieve maximum total value without breaking the bag? Dantzig proposed that we consider the ratio $v_i/w_j$, and add items for which this ratio is largest until the bag fills up. But this is regarded as an approximate solution.

I have also been thinking about this problem. I came up with the same solution as Dantzig and even proved it for the $0-1$ problem. Of course, I was suspicious of my proof even before I knew the method had been proposed before (since it would imply that $P= NP$). I am even more suspicious now.

So put me out of my misery. Why doesn't it work?

MJD
  • 67,568
  • 43
  • 308
  • 617
Matthew Matic
  • 457
  • Well, how would you prove that this method is the best in a general setting? – naslundx Mar 20 '14 at 16:18
  • 1
    This method is no help if $v_i=w_i$ for all $i$ $-$ it essentially says pick items at random. This objection applies to the $0-1$ problem as well. So it can't be right! – TonyK Jan 08 '19 at 23:44

2 Answers2

20

It doesn't work because it doesn't work.

Say you have $W=10$ and there are three kinds of objects:

  1. Type $A$ has weight 9 and value 90.
  2. Type $B$ has weight 2 and value 19.
  3. Type $C$ has weight 1 and value 1.

If you look at $v/w$, type $A$ wins, so you take one type $A$ object and fill up the remaining space with one type $C$, for total value $91$.

But the optimal solution takes five type $B$ objects for total value $95$.

We might characterize the general difficulty as follows: A choice that seemed good early on ran into trouble later, because our early choice of $A$, optimal in the short term, forces us to take the very bad $C$ afterwards. The optimal solution takes many choices that seem suboptimal in the short run, but which work well together.

This is quite typical of NP-complete problems. They have many parts that interact in complicated ways, so that the effects of a particular choice in one part of the problem can't be localized, but have far-reaching implications for choices made elsewhere. Compare this with the prototypical NP-complete problem, CNF-SAT, where the parts are logical clauses containing several components each, but each component may appear in multiple clauses. Each choice about the value of one component of one clause may have far-reaching effects on the other clauses.

Or similarly, consider bin packing. Items that fit optimally into the first bin may turn out to be just what was needed to fill out the space in later bins, so that an optimal packing of the first bin may get a short-term win at the expense of the overall packing; an overall optimal solution may pack the first bin suboptimally in order to save certain special items for later on.

MJD
  • 67,568
  • 43
  • 308
  • 617
  • For the $0-1$ problem it seems to me that choosing $A$ is optimal. I guess I should try to find a counter example that does work in the $0-1$ case, though. – Matthew Matic Mar 20 '14 at 16:58
  • 3
    @Matthew Is the $0-1$ case the case where you can take at most one of each item? If so, split type $B$ into types $B_1\ldots B_5$ each with weight 2 and value 19, or if you want them to be different, give $B_i$ weight $2-\epsilon_i$ and value $19+\epsilon_i$. – MJD Mar 20 '14 at 16:58
  • That's great. Thanks. I think I identified my faulty step. I think it gives the optimal solution if it exactly obtains the weight bound, and not otherwise. – Matthew Matic Mar 20 '14 at 17:35
  • 1
    @MatthewMatic: If you're interested in implementing a knapsack solver, check out this post: http://codereview.stackexchange.com/questions/44421/improving-knapsack-branch-and-bound-how-to-forward-filter – rookie Mar 20 '14 at 18:00
  • @Matthew I don't think so. It's easy to perturb my $A, B_i, C$ example to one where the optimal solution exactly obtains the weight bound. Just make sure that $\sum\epsilon_i = 0$. – MJD Mar 20 '14 at 18:23
  • @MJD I mean that if the method obtains the weight bound, it is optimal. Not that if the optimal solution obtains the weight bound, this method will find it. – Matthew Matic Mar 20 '14 at 19:33
  • 2
    I don't think that's true either. In 0-1 case with W = 10 consider A with (weight 6, value 60), B and C both with (weight 5, value 49), and D,E,F,G all with (weight 1, value 1)

    your method chooses A (with v/w = 10) first, then B and C don't fit, and it then it chooses D, E, F, and G to meet the weight bound exactly, with total value 64; while the optimal solution is B+C with total value 98.

     – Sumudu Fernando Mar 20 '14 at 21:07
2

Just to be clear: If you fill up items according to their value/weight ratio until you cannot fit any more item in this order, the result is optimal, if the accumulated weight by then matches exactly the available space. In other words, it is ALWAYS optimal for the total weight it accumulated. I guess this is the result you had in mind. It is not hard to prove and can be found in any introduction to knapsack problems, for example in "Martello, S. & Toth, P. Knapsack problems: algorithms and computer implementations. John Wiley & Sons, 1990"

However, as discussed in the first answer and the comments there, your method obviously isn't always optimal if it ends without filling the bag completely. Think of $W=10^n$ for large $n$, and items $A$(weight $1$, value $1$) $B$(weight $10^n$, value $10^n-1$). Then your algorithm ends up with $A$ and a value of $1$, when it could instead have had $B$ with a value of $10^n-1$. Nevertheless, $A$ is trivially optimal for the weight it achieves, i.e. optimal for a knapsack with total weight $1$.

Bananach
  • 8,332
  • 3
  • 32
  • 53
  • This answer may be misleading? Consider the case mentioned in @MJD post, If you greedily fill the items according to their v/w for W=12 you end up with [ A, B, C] for a value of 110 while 6xB gives a value of 114. – Jeroen Vuurens Dec 17 '18 at 18:23
  • @JeroenVuurens this doesn't contradict what I'm saying in the answer. Note that I say "until you cannot fit any more item in this order". Hence, assuming there are multiple items of type B since you talk about a scenario where multiple B items are used, my answer does not cover the situation where you start inserting C. If you have an idea how to express the statement more clearly, l'll happily edit my answer, but the mathematical content is correct as it is now – Bananach Dec 17 '18 at 19:59
  • @JeroenVuurens if you agree with me, you may upvote my answer to give others confidence in the statement behind it. I found it very troubling that the existing answer only gives a negative result, when the reality is that greedy packing works just fine in most cases (and my answer shows why) – Bananach Dec 17 '18 at 20:02
  • still do not agree. In the given example for W=12, ordering the item types by v/w results in [ A, B, C ] (or [A, B, B, B, B, B, B, C] if you insist on 0/1 assignment). So if I understand you correctly you claim the algorithm will be optimal if we fill up items greedily if we start out with item A because that has the highest v/w before we move to the next. In this example that will fill the knapsack with ABC while BBBBBB is better. – Jeroen Vuurens Dec 19 '18 at 13:19
  • @JeroenVuurens my post says "fill up items according to their value/weight ratio until you cannot fit any more item in this order". Hence, you would start by including A because it has the best v/w and it fits. Then you would include B because it has the best v/w of the remaining items and it also fits. After this, the next B has the best v/w but since it doesn't fit you stop. You end up with [A,B] which has weight 11 and value 109. No other combination with weight $\leq 11$ has better value – Bananach Dec 19 '18 at 13:29