8

Update: Problem and solution found here (p. 17, 61), although my prof's solution (formulation) is different.

Convert

$$\min z = f(x)$$

where

$$f(x) = \left\{\begin{matrix} 1-x, & 0 \le x < 1\\ x-1, & 1 \le x < 2\\ \frac{x}{2}, & 2 \le x \le 3 \end{matrix}\right.$$

s.t.

$$x \ge 0$$

into a linear integer programming problem.

What I tried:

It seems that according to this,

$$\min \ f(x) = \max(a_1x + b_1, a_2x + b_2, ..., a_mx + b_m), x \ge 0$$

is equivalent to

$$\min \ t \ \text{s.t.}$$

$$a_1x + b_1 \le t$$

$$a_2x + b_2 \le t$$

$$\vdots$$

$$a_mx + b_m \le t$$

$$x \ge 0$$

Following that, I tried

$$g(x) = \max(1-x, x-1, x/2) = \left\{\begin{matrix} 1-x, & 0 \le x \le 2/3\\ x-1, & x \ge 2\\ \frac{x}{2}, & 2/3 \le x \le 2 \end{matrix}\right.$$

If we allow only integer $x$, then we have

$$g(x) = \max(1-x, x-1, x/2) = \left\{\begin{matrix} 1-x, & x=0\\ x-1, & x=2\\ \frac{x}{2}, & x=1 or 2 \end{matrix}\right.$$

If we allow only integer $x$ for $f$, then we have

$$f(x) = \left\{\begin{matrix} 1-x, & x=0\\ x-1, & x=1\\ \frac{x}{2}, & x=2 or 3 \end{matrix}\right.$$

It doesn't look like $f(x) = g(x)$, w/ or w/o integer constraint. How can I approach this?

(The following is copied from an answer I deleted and comments on it)

Prof's answer (assuming I remembered question right):

Let Xi = X for ith constraint.

Minimise

$$z = (1-x_1)+(x_2-1)+(1/2)(x_3)$$

s.t.

$$0 \le x_1 \le y_1$$ $$y_2 \le x_2 \le 2y_2$$ $$2y_3 \le x_3 \le 3y_3$$ $$y_1, y_2, y_3 \in \{0,1\}$$ $$x_1,x_2,x_3 \ge 0$$

Comments below it:

(You should probably annotate this to indicate the source.) Anyway, if you suspected something was off about this solution, then you're right: it's incorrect. This formulation is similar to Kuifje's Option 3, but it incorrectly encodes the objective function. The minimum value occurs at $(y_1,y_2,y_3)=(1,0,0)$, presumably as expected, but here $(x_1,x_2,x_3)=(1,0,0)$ giving $z=−1$. The value $−1$ is never taken by the original function! The $(x_2−1)$ term should have been chosen to minimize at $0$ in the case that $y_2=0$, but it wasn't. – Erick Wong 1 hour ago

@ErickWong I was lacking one thing: 'let xi=X for the ith constraint.' what about now? Thanks for the feedback XD honestly I haven't yet bothered to analyse any of these. We didn't discuss boolean logic. I'm about to ask my prof about this. I could have remembered the question wrong. I do remember that function and something about an integer linear programming problem – BCLC 11 mins ago

That extra line shouldn't make a difference: it just declares the Intent of the variable $x_i$ (as an aid to the reader) but it doesn't change its value. Good luck, I do believe if the function is exactly as you remember then this answer is flawed. I haven't carefully analyzed Kuifje's answers and there may be minor typos there too :) – Erick Wong 5 mins ago

@ErickWong Edit: Thanks for the feedback XD honestly I haven't yet bothered to analyse any of these. I mean I guess I could understand if I analysed, but the point is that I don't think average student in my class can come up with this without boolean logic because we didn't discuss boolean logic. I'm about to ask my prof about this. I could have remembered the question wrong. I do remember that function and something about an integer linear programming problem. – BCLC 2 mins ago edit

@ErickWong THANK YOU XD – BCLC 2 mins ago edit

BCLC
  • 14,197
  • 1
    It was a long time ago since I did this, but I will give it a go. One dimension per interval with function $f_k$ (vars $x_k$) and then use slack variables $s_k$ as imitating booleans. $$s_1\cdot f_1(x_1) +s_2\cdot f_2(x_2) + s_3\cdot f_3(x_3)$$ Optimality should be in a "corner" if it is a linear problem (two of $s_k$ being 0). But wait, this will destroy linearity won't it? – mathreadler May 11 '16 at 17:23
  • @mathreadler thanks ^-^ 1 So your answer is the same as Kuifje's Option 1? 2 So the problem might have no solution since such approach destroys linearity? What about Kuifje's Option 1? 3 Do you think this problem is beyond reasonable expectation of difficulty presented to a beginning operations student with practically no background on Boolean algebra? – BCLC May 11 '16 at 17:54
  • 1
    I don't think it is identical. Kuifjes answer nr 1 is sure to be linear as it only takes the function value at the specific end-points (this works because of piecewise linearity combined with the corner solution property). I think it is a good question in any way. Maybe not suitable for an exam, but very interesting and practically useful question anyway. – mathreadler May 11 '16 at 18:10
  • @mathreadler Ah, thanks ^-^ Do you think average operations research can answer this during an exam without knowing boolean algebra? – BCLC May 11 '16 at 18:12
  • I don't really know what other courses are usually included in operations research. – mathreadler May 11 '16 at 18:23
  • @mathreadler I mean, for you, do you think you can answer in 20 minutes without knowing boolean algebra? – BCLC May 11 '16 at 18:43

2 Answers2

4

There are many ways to do this. Here are three:

Option 1

Let $y_i$ be a binary that equals $1$ if and only if $x$ is in the $i^{th}$ interval ($i\in \{[0,1],[1,2],[2,3]\}$). The idea is then to express $x$ as a convex combination of the extreme points of the intervals. Therefore, we introduce variables $\lambda_0, \lambda_1,\lambda_2,\lambda_3 \in \mathbb{R}^+$ with which we will achieve this.

The objective function is $$ f(0)\lambda_0+f(1)\lambda_1+f(2)\lambda_2+f(3)\lambda_3=\lambda_0+\lambda_2+1.5\lambda_3 $$

and the constraints are \begin{align*} &y_1+y_2+y_3 = 1\\ &x = 0\lambda_0+\lambda_1+2\lambda_2+3\lambda_3\\ &\lambda_0+\lambda_1+\lambda_2+\lambda_3=1\\ &y_1\le \lambda_0+\lambda_1\\ &y_2\le \lambda_1+\lambda_2\\ &y_3\le \lambda_2+\lambda_3\\ &y\in \{0,1\}\\ &\lambda\ge 0 \end{align*}

Option 2

Alternatively, you can write the objective function as $$ f=(1-x_1)+(x_2)+(\frac{x_3}{2}) $$ with constraints $$ x=x_1+x_2+x_3\\ x\le 3\\ y_1\le x_1 \le 1\\ y_2\le x_2 \le y_1\\ 0 \le x_3 \le y_2\\ y_1,y_2\in \{0,1\} $$

$y_1$ and $y_2$ are binaries that equal $1$ if and only if variables $x_1$ and $x_2$ reach their upper bound. This ensures that, for example, if $x=1.5$, the solver has to fix $x_1=1$ and $x_2=0.5$.

Option 3

Write the function as follows: $$ f=(y_1-x_1)+(x_2)+(\frac{x_3}{2}+y_3) $$ subject to $$ x=(x_1)+(x_2+y_2)+(x_3+2y_2)\\ y_1+y_2+y_3=1\\ 0\le x_1\le y_1\\ 0\le x_2\le y_2\\ 0\le x_3\le y_3\\ y_1,y_2,y_3\in \{0,1\} $$

This time, binaries $y_i$ equal $1$ if and only if $x$ is in the $i^{th}$ interval ($i\in \{[0,1],[1,2],[2,3]\}$).

Kuifje
  • 9,802
  • Wow. Thanks Kuifje. Congrats on passing 2000. I hope you pass my rep soon. In your opinion, can someone arrive at your answer or answer the original question using only techniques taught in Chapters 2 and 3 here? This was given to me in a make-up exam, and it seems to be beyond the techniques we were taught. – BCLC Apr 19 '16 at 17:05
  • 2
    Well, I think so. If you are familiar with boolean logic, than it should not be a problem. It is however hard to come up with this on your own if you are not used to mathematical modeling. As I said in my answer there are other options involving different types of binary variables, but they are not (in my opinion) easier than the answer I proposed. If you had never seen any question like this one in class before the exam, than it is a difficult question. – Kuifje Apr 19 '16 at 17:54
  • Kuifje, right, we have never encountered anything like converting a piecewise function into an objective function and a bunch of inequalities. All our LP and IP formulations have so far been based on descriptions of real-life situations. You said, 'If you had never seen any question like this one in class before the exam, than it is a difficult question.' After what I said, would you say that is it is too difficult in the sense that no student is expected to, at least in your opinion, answer such? – BCLC Apr 19 '16 at 17:59
  • 1
    Lets just say that I would not be surprised if everyone failed the question. – Kuifje Apr 19 '16 at 18:24
  • 1
    I started bounty. If there are no takers, I'm giving it to you. – BCLC Apr 23 '16 at 13:44
  • I have added an additional option to complete my answer. – Kuifje Apr 23 '16 at 14:09
  • oh thanks. wait i edited my question. sorry forgot min. does it change much? are your objective functions min/max depending on whether i meant min/max? so in the first option, we want to min $$z = \lambda_0+\lambda_2+1.5\lambda_3$$ subject to the constraints you said? wait in the constraints (for both first and second option), is $x$ some parameter? If so, is it also the case in the Vandenberghe link I gave in OP? – BCLC Apr 23 '16 at 14:19
  • Saw edit. Did you mean $y_1 + 1 - x_1$? So like fixed costs or something like here? – BCLC Apr 23 '16 at 14:26
  • Kuifje, in options 2 and 3, can we define $y_i$ that way? I think those should be implied by the values of $y_i$. something like here, we can't just say that $y_3 = 0$ if $x_2 \ge 4$. i think we have to say that $x_2 - 3 \le M(1-y_3)$. So if $y_3 = 1$, then we have $x_2 \le 3$. Something like that – BCLC Apr 23 '16 at 14:29
  • sorry forgot z. edited question. is your $f$ in options 2 and 3 equivalent to my $z$ ? – BCLC Apr 23 '16 at 14:32
  • 1
    @BCLC I suspect the question was written a bit hastily given the circumstances. Notice that $f$ isn't even well-defined: the domain is supposed to be $x\ge 0$, but $f(4)$ does not exist. – Erick Wong Apr 23 '16 at 14:45
  • @ErickWong Well yeah, it was in a make-up exam. I don't quite remember it exactly, but I do remember the $x \ge 0$. – BCLC Apr 23 '16 at 14:54
  • @ErickWong Is there anything else that makes you think that it was written a bit hastily? – BCLC May 07 '16 at 10:28
  • 1
    @BCLC Well, to be honest I also read the question hastily. The title said "linear optimization" and the function wasn't convex, which seemed unreasonable. I originally thought that this was also caused by hastiness – but later I saw that the question body says "integer-linear optimization". This makes it able to handle arbitrary computation, so non-convex objective functions are fair game. – Erick Wong May 07 '16 at 15:41
  • @ErickWong Interesting. You mean linear wouldn't work, but integer linear would work? – BCLC May 07 '16 at 16:42
  • 1
    @BCLC If the piecewise-linear function were convex, then it would be exactly equal to the max of its constituent pieces (so the very first function $g(x)$ that you tried would have worked fine). I believe it isn't possible to represent a non-convex function in this way with a linear program. But as Kuifje suggests, once you have integer variables you can have $0/1$-variables and arbitrary Boolean logic, which lets you encode if statements. – Erick Wong May 08 '16 at 05:34
  • @ErickWong Which part? – BCLC May 09 '16 at 18:44
  • 1
    @BCLC Which part what? – Erick Wong May 09 '16 at 18:48
  • @ErickWong Which part of what Kuifje suggested made you say that last part of your comment? Like the non-convexity is not problematic because this is an integer programming problem? – BCLC May 09 '16 at 18:51
  • 1
    @BCLC Oh, Kuifje said "If you are familiar with Boolean logic then it should not be a problem". Previous exposure to Boolean logic would tell you that if you can have 0/1 variables then you just need mechanisms for AND/OR/NOT to be able to construct very complex logic. And each of those is very easy to implement with linear inequalities and integer constraints. The trick is in perceiving their utility. – Erick Wong May 09 '16 at 18:56
  • @ErickWong Never encountered Boolean logic in undergrad or grad. Do you have a reference for this type of formulation problem please? – BCLC May 09 '16 at 19:01
  • 1
    @BCLC I can't think of a favorite reference but the Wikipedia article on Boolean algebra seems a decent place to start. The important thing is the concept of transcribing the logic of English sentences (such as "if $x \ge 2$ and $x \le 3$ then...") into a formal algebraic description. It's a simple concept with a lot of power. Once you perceive a statement as something that can be expressed in terms of binary variables, it becomes much easier to see that it can be encoded into an integer program. – Erick Wong May 09 '16 at 21:32
  • @ErickWong errrrrr okay thanks – BCLC May 10 '16 at 16:16
  • @ErickWong I posted my prof's answer. What do you think? Is it less simpler compared to any of Kuifje's answers? – BCLC May 11 '16 at 14:16
  • Kuifje, I posted my prof's answer. What do you think? – BCLC May 11 '16 at 14:16
  • 1
    It is similar to option 2, but there is something i don't understand: how do you link variables $x_1,x_2,x_3$ with variable $x$? In other words, if $x=1.5$, how do you express this in terms of $x_1,x_2,x_3$? It looks like it should be $x=x_1$ or $x=x_2$ or $x=x_3$ but in this case the model is incomplete. – Kuifje May 11 '16 at 15:42
  • Kuifje Might you please have a look at this or this? – BCLC May 11 '16 at 16:54
  • Kuifje I was lacking one thing: 'let xi=X for the ith constraint.' what about now? Thanks for the feedback XD honestly I haven't yet bothered to analyse any of these. I mean I guess I could understand if I analysed, but the point is that I don't think average student in my class can come up with this without boolean logic because we didn't discuss boolean logic. I'm about to ask my prof about this. I could have remembered the question wrong. I do remember that function and something about an integer linear programming problem. What do you think about what ErickWong said? – BCLC May 11 '16 at 16:54
  • @ErickWong Might you please have a look at this or this? – BCLC May 11 '16 at 16:55
  • @ErickWong Deleted answer. Moved to OP. Thanks to you and Kuifje XD – BCLC May 11 '16 at 16:58
  • @ErickWong What if the objective function in my prof's answer was instead $$(1-x_1) + (x_2 - 1) + (1/2)x_3 \color{red}{+ 1}$$? – BCLC May 11 '16 at 18:15
  • Kuifje, I notice your objective functions in optons 2 and 3 seem to have an extra $\pm 1$. Why is this? For option 2, it seems you have $$(1-x_1) + (x_2 - 1) + (1/2)x_3 \color{red}{+ 1}$$? For option 3, it seems you have $$(1-x_1) + (x_2 - 1) + (1/2)x_3 +y_1 + y_2 + y_3 \color{red}{+ (- 1)}$$? What's up with those? – BCLC May 11 '16 at 18:30
  • Kuifje, prof corrected. How is answer now? – BCLC May 13 '16 at 16:21
  • 1
    Your professor's answer is very similar to Option 3, but you need to add the constraint which ensures that "$x_i=x$ for the ith constraint". If you do not have such a constraint, the answer is wrong. Are you 100% sure this is exactly your professor's answer? – Kuifje May 13 '16 at 18:46
  • Kuifje, my professor is a graduate of mathematical computational science and thus had boolean algebra or something. I asked him if he had that in undergrad and that if someone knew that e would find this question easy. He said he didn't use boolean algebra. He explained to me that this formulation has to do with fixed costs. Does that ring any bells? I kind of get his explanation. Man that question was tough but technically the handouts (linked) include discussions on how to handle fixed costs in a piecewise way. Does the problem seem fixed cost-like to you? – BCLC May 13 '16 at 18:56
  • Kuifje I am 99.9% sure. He wrote down answer in my make up test paper. Then I asked him about it and was about to show him how his formulation gave an answer of -1 on lips but then he realised his mistake, gave me the formulation and explained it in a fixed cost way – BCLC May 13 '16 at 18:59
  • 1
    I understand the fixed cost approach, which does include boolean logic by the way ($y_1+y_2+y_3=1$). But in my opinion the model is incomplete. What does Eric Wong think about it? It is a very interesting question and you should ask other people's opinion. I know Michael Grant and Erwin Kalvelagen are very good at modeling. – Kuifje May 13 '16 at 19:12
  • 1
    The problem with the model as it is, is that it will never consider the third interval. For example, if $x=3$, the solver has absolutely no reason to set $y_3$ to $1$. – Kuifje May 13 '16 at 19:20
  • Kuifje, if I plug into a solver, it won't give me $(x1,x2,x3)=(0,1,0)$? Or $(x1,x2,x3)=(1,0,0)$. I haven't tried inputting in Lips yet. What solver do you use? – BCLC May 15 '16 at 10:25
  • 1
    Personnaly I use PuLp (with Python) and call whichever solver is available (cplex, gurobi, cbc,..). – Kuifje May 15 '16 at 13:45
  • Kuifje, used a solver. I got $(1,0,0)$. Not sure how to get alternative solution in Lips. Soooo model is right? – BCLC May 15 '16 at 19:50
0

Prof's answer:

Let $X_i = X$ for ith constraint.

Minimise

$$z = (y_1-x_1)+(x_2-y_2)+(1/2)(x_3)$$

s.t.

$$0 \le x_1 \le y_1$$ $$y_2 \le x_2 \le 2y_2$$ $$2y_3 \le x_3 \le 3y_3$$ $$y_1+y_2+y_3=1$$ $$y_1, y_2, y_3 \in \{0,1\}$$ $$x_1,x_2,x_3 \ge 0$$

BCLC
  • 14,197
  • 1
    (You should probably annotate this to indicate the source.) Anyway, if you suspected something was off about this solution, then you're right: it's incorrect. This formulation is similar to Kuifje's Option 3, but it incorrectly encodes the objective function. The minimum value occurs at $(y_1,y_2,y_3) = (1,0,0)$, presumably as expected, but here $(x_1,x_2,x_3) = (1,0,0)$ giving $z=-1$. The value $-1$ is never taken by the original function! The $(x_2-1)$ term should have been chosen to minimize at $0$ in the case that $y_2=0$, but it wasn't. – Erick Wong May 11 '16 at 15:40
  • @ErickWong I was lacking one thing: 'let xi=X for the ith constraint.' what about now? Thanks for the feedback XD honestly I haven't yet bothered to analyse any of these. We didn't discuss boolean logic. I'm about to ask my prof about this. I could have remembered the question wrong. I do remember that function and something about an integer linear programming problem – BCLC May 11 '16 at 16:44
  • 1
    That extra line shouldn't make a difference: it just declares the Intent of the variable $x_i$ (as an aid to the reader) but it doesn't change its value. Good luck, I do believe if the function is exactly as you remember then this answer is flawed. I haven't carefully analyzed Kuifje's answers and there may be minor typos there too :) – Erick Wong May 11 '16 at 16:50
  • @ErickWong Edit: Thanks for the feedback XD honestly I haven't yet bothered to analyse any of these. I mean I guess I could understand if I analysed, but the point is that I don't think average student in my class can come up with this without boolean logic because we didn't discuss boolean logic. I'm about to ask my prof about this. I could have remembered the question wrong. I do remember that function and something about an integer linear programming problem. – BCLC May 11 '16 at 16:52
  • @ErickWong THANK YOU XD – BCLC May 11 '16 at 16:53
  • @ErickWong Well, prof corrected. How is answer now? – BCLC May 13 '16 at 16:20
  • 1
    The problem with the model as it is, is that it will never consider the third interval. For example, if $x=3$, the solver has absolutely no reason to set $y_3$ to $1$. – Kuifje May 13 '16 at 19:20
  • 1
    @BCLC Looks pretty reasonable now. Each of the terms $(y_1-x_1), (x_2-y_2), (x_3/2)$ has been carefully chosen so that when $y_i = 1$ it is equivalent to the appropriate function among $1-x, x-1, x/2$, while when $x_i=y_i=0$ it is equivalent to $0$. To achieve both aims simultaneously the trick is to replace any constant terms $c$ with $cy_i$ for the appropriate $i$. I do note that the nonnegativity conditions on $x_i$ are redundant. – Erick Wong May 14 '16 at 03:46
  • @Kuifje Thanks. What do you mean? The minimum of the piecewise function is indeed $(0,1)$ right? PS I found the problem. Linked in OP. – BCLC May 15 '16 at 10:22
  • @ErickWong Thanks ^-^ So there is or isn't a problem? Kuifje seems to think there is – BCLC May 15 '16 at 10:22
  • 1
    I don't actually see why that's a problem. @Kuifje Can you elaborate? – Erick Wong May 15 '16 at 12:44
  • 1
    Ok, after giving it some thought, the model looks correct. However, if you were to add constraints to the problem that depend on variable $x$, it would be incomplete, as variables $x,x_1,x_2,x_3$ are not linked together here. But since the problem has no constraints, it is fine. – Kuifje May 15 '16 at 13:42
  • 1
    Note that in the link you posted, the solution proposed is in the spirit of Option 1, but is different! Fascinating that there are at least 5 correct ways to answer the question. – Kuifje May 15 '16 at 13:59
  • Thanks @Kuifje ^-^ What's an example of such constraints please? – BCLC May 15 '16 at 19:49
  • Imagine you have the extra constraint $x\ge 2.5$. – Kuifje May 15 '16 at 19:58
  • @Kuifje Yeah actually, what's up with that? Your formulations have $x$ in some constraint while my prof's formulation doesn't...? – BCLC May 15 '16 at 20:01
  • yes, precisely . – Kuifje May 15 '16 at 20:12
  • @Kuifje I mean, why are constraints that depend on $x$ okay in yours but not okay in my prof's? – BCLC May 15 '16 at 20:24
  • because, if you need to add for instance $x\ge 2.5$, your professor's model will not take this constraint into account, it will keep returning the solution $x=1.$ – Kuifje May 15 '16 at 20:32
  • @Kuifje So how come a constraint that depends on yours is okay? – BCLC May 18 '16 at 20:07
  • @Kuifje you said 'there are other options involving different types of binary variables, but they are not (in my opinion) easier than the answer I proposed' Do you stand by that? – BCLC May 26 '16 at 13:01