3

The following problem comes from information theory. Specifically, for an impossibility bound I need to supply the decoder with side information and the answer will say what policy gives the tightest bound. But the optimization problem is self-contained.

We are given two points on the plane: an origin $(x_0,y_0)$ and destination $(x_1,y_1)$. It is known that $|y_0,y_1| \leq 1$ and that $0 \leq x_0 \leq x_1 \leq 1$. I need to minimize the integral of a cost function over the trajectories. The trajectory is constrained to be increasing in x, thus I can write the problem as an integral over x: minimize $$ \int_{x_0}^{x_1} \frac{[1-y'(x)]^2}{1-y^2(x)} x^4 dx $$ where $y'(x) = dy(x) / dx$, $|y(x)| \leq 1$ always, and of course $y(x_0)=y_0,y(x_1)=y_1$. In case it helps, the interesting case seems to be $x_1=y_1=1$.

  • 1
    Are you aware of variational calculus? If so, what was the result of your attempt to apply it? I get $\frac{y''}{1-y'}=\frac4x+\frac y{1-y^2}$, which I don’t see how to solve (but I didn’t check my calculations carefully). – joriki Mar 14 '24 at 09:45
  • Thanks @joriki ! If I understand correctly, the Euler-Lagrange equation states that there is a local minimum for $$ \frac{\partial L}{\partial y} = \frac{d}{dx} \frac{\partial L}{\partial y'},$$ for integrand $L$. It doesn't take into account the constraints, but lt's forget that for a moment. I calculated it and I'm getting what you got plus an extra term $2 y y' / (1-y^2)$ that comes from the derivative of the denominator w.r.t. x in the RHS, am I right? In any case it does not look tractable. But maybe just in the limit where $x$ and $y$ both approach $1$, we can find the behavior? – info_theorist Mar 14 '24 at 11:14
  • to be clear, the equation I get is: $$ \frac{y''}{1-y'} = \frac{4}{x} + \frac{(1+y') y}{1-y^2}. $$ – info_theorist Mar 14 '24 at 12:42
  • Yes – I did differentiate the denominator, but I missed a factor $2$ in doing so – now I get the same result as you. – joriki Mar 14 '24 at 14:44

1 Answers1

2

As discussed in the comments, this sort of problem can be addressed using variational calculus, and the Euler–Lagrange equation in this case is

$$ \frac{y''}{1-y'}=\frac4x+\frac{(1+y')y}{1-y^2}\;. $$

I don’t see how to solve this differential equation analytically. If it weren’t for the term $\frac4x$, we could multiply by $\frac{y'}{1+y'}$ and then integrate both sides with respect to $x$, but that doesn’t work with the additional term.

However, we can analyze the behaviour near the point of interest, $(x,y)=(1,1)$. The second term on the right diverges at this point, and this divergence must somehow be compensated to satisfy the equation. Unless $y'$ or $y''$ diverges, the numerator on the left must also go to $0$, that is, $y'(1)=1$.

We could now try to use the known values of $y$ and $y'$ at $x=1$ to determine $y''$ from the differential equation. However, using L’Hôpital’s rule merely yields

\begin{eqnarray*} y''(1) &=& \lim_{x\to1}\left(\left(1-y'\right)\left(\frac4x+\frac{(1+y')y}{1-y^2}\right)\right) \\ &=& \lim_{x\to1}\frac{1-y'^2}{1-y^2} \\ &=& \lim_{x\to1}\frac{\left(1-y'^2\right)'}{\left(1-y^2\right)'} \\ &=& \lim_{x\to1}\frac{-2y'y''}{-2yy'} \\[5pt] &=& y''(1)\;. \end{eqnarray*}

Thus, the differential equation doesn’t constrain $y''(1)$, and we can freely chose it as an initial value instead of the missing initial value $y'(1)$ that we’re not free to choose. (For another instance of this phenomenon, see this answer.)

Here are some numerical results obtained using Runge-Kutta integration from $x=1$ to $x=0$ for various values of $y''(1)$ equally spaced between $-2$ and $2$:


Runge-Kutta results


Here’s the Java code I used.

Note that $y=x$ is a solution (which is clear when you multiply the differential equation by $1-y'$).

If you want the curve that starts at a given point $(x_0,y_0)$, you can integrate from $(1,1)$ with some value of $y''(1)$ and then adjust this value until you hit the desired origin.

joriki
  • 242,601
  • 1
    Thank you, that was most helpful! I should say, that I did run numerics for a discrete version of this problem (that is, I have a cost function between points on a grid, and I perform a search to minimize the sum), and I get trajectories that look (at least qualitatively) very similar to your numerical integration. – info_theorist Mar 14 '24 at 14:54