Steepest descent for a function defined over a specific domain

Question

I am trying to minimize a convex function using the steepest descent method. The function is defined over the domain $D = \{(x, y) \in R^2 : 2x^2+y^2 < 10\}$.

The gradient descent iterations:

$$\mathbf{x}_{n+1}=\mathbf{x}_n-\gamma \nabla F(\mathbf{x}_n),\ n \ge 0$$ where $\gamma$ is my constant stepsize (I tried diminishing stepsize and I get the same problem)

I easily get a solution $\mathbf{x}_{n+1} \notin D$ and then diverge. How should I deal with the problem of going outside the domain?

@Rajada: I am actually looking for another solution to compare to the Armijo rule. — amaatouq, Nov 26 '15 at 07:55
@littleO: the function I am trying to minimize is $f(x,y) = -log(10-2x^2 -y^2)$ so $10-2x^2 -y^2 > 0$ is an implicit constraint — amaatouq, Nov 26 '15 at 10:08
In that case, I think the typical thing to do is to use a line search procedure, which will keep you in bounds. — littleO, Nov 26 '15 at 10:15
Right. There is no need for projection for that specific function. — Michael Grant, Nov 27 '15 at 16:12
@littleO, we are off our game here. The optimum is obviously $(x,y)=(0,0)$. No need for a line search, gradient search, anything. — Michael Grant, Nov 27 '15 at 20:06

score 1 · Answer 1 · answered Nov 26 '15 at 08:51

1

what about considering using gradient projection method, after each step, project your current result onto your constraint set.

answered Nov 26 '15 at 08:51

Xia

121

score 0 · Answer 2 · edited Apr 13 '17 at 12:21

0

You are minizing the sum of a smooth function $F$, and a non-smooth function $i_D$ (the indicator function of the domain $D$) for you can compute the proximal operator, which in fact equals the projection operator onto $D$ ($D$ is an ellipsoid, so this is an easy business).

Projected gradient methods like ISTA, FISTA, etc. are your friends here.

You may want to checkout my detailed answer to a similar question.

edited Apr 13 '17 at 12:21

Community

1

answered Nov 26 '15 at 10:31

dohmatob

9,753

I was thinking this too, but the constraints have strict inequality, which I think will be a problem for projected gradient methods. The objective function isn't finite on the boundary of the ellipsoid. – littleO Nov 26 '15 at 10:42
Now that the OP has offered the specific function, it is clear that no projection is necessary. – Michael Grant Nov 27 '15 at 16:12

Steepest descent for a function defined over a specific domain

2 Answers2