2

When considering the expression $|e^x - c|$, there are several ways to choose our constant $c$ that minimizes this for different norms. There are standard methods for doing so in the $\infty$-norm and the 2-norm, but there doesn't appear to be an immediately obvious way to find $c$ that minimizes

$$\int_0^1 |e^x - c \ | \ dx$$

The question originally asks for an equation to find $c$, rather than an exact derivation. The derivation is below, but I would be curious to see what other methods there might be to go about solving a problem like this.

cnolte
  • 424

2 Answers2

7

You can rephrase your question as finding $c$ to minimize $E[|e^X-c|]$ where $X$ is a uniform random variable. In general,

The $c$ that minimizes $E[|Y-c|]$ is the median of the distribution of $Y$.

Some intuition can be found in this answer.

Here, we see the median of $X$ is $1/2$, and since $x \mapsto e^x$ is increasing, the median of $e^X$ is $e^{1/2}$, as you computed.

A similar result:

The $c$ that minimizes $E[(Y-c)^2]$ is the mean of the distribution of $Y$.]

The result for the mean is quite easy. Setting the derivative equal to zero gives $$\frac{d}{dc} E[(Y-c)^2] = -2E[Y] +2c = 0 \implies c =E[Y].$$ An alternate proof that does not use calculus is the following: $$E[(Y-c)^2] = E[(Y-E[Y]+E[Y]-c)^2] = E[(Y-E[Y])^2] + (E[Y]-c)^2.$$ (The cross term is zero since $E[Y-E[Y]]=0$.) From this decomposition it is clear choosing $c=E[Y]$ minimizes the expectation.

For the median, you have to separate into cases as you have done. \begin{align} \frac{d}{dc}E[|Y-c|] &= \frac{d}{dc}\Big(\int_c^\infty (y-c) f_Y(y) \mathop{dy} - \int_{-\infty}^c (y-c) f_Y(y) \mathop{dy}\Big)\\ &= \frac{d}{dc} \Big(\int_c^\infty yf_Y(y) \mathop{dy} - \int_{-\infty}^c y f_Y(y) \mathop{dy} - c \int_c^\infty f_Y(y) \mathop{dy} + c \int_{-\infty}^c f_Y(y) \mathop{dy}\Big)\\ &= -2 c f_Y(c) + (P(Y>c)- P(Y<c)) + 2cf_Y(c)\\ &= P(Y>c)-P(Y<c). \end{align} Setting this equal to zero amounts to choosing $c$ to be the median of $Y$.

Community
  • 1
angryavian
  • 93,534
5

The question specifically asks to find an equation for the constant $c$ that minimizes this expression. The typical method here is to let this expression be a function of $c$ and find the minimum of this function with respect to $c$. Thus, we have

$$ f(c) = \int_0^1 |e^x - c| \ dx$$

We have to be careful here, as we are not sure if the expression inside of the absolute value is always positive, always negative, or changes sign somewhere in $[0,1]$. Since $e^x$ is monotonically increasing, we know that there will only be at most one sign change on this interval. If we assume that $c \leq e^x$ on $[0,1]$, that is, $c \leq 1$, then we have that this is always positive, and

$$f(c) = \int_0^1 e^x - c \ dx = e - c -1$$ $$f'(c) = -1$$

On the other hand, if $e^x \leq c$, or $c \geq e$ on $[0,1]$, we have

$$f(c) = \int_0^1 c - e^x \ dx = c - e + 1$$ $$f'(c) = 1$$

For both of these cases, $f(c)$ does not obtain a minimum value, and by observation, in each case respectively, we have that our choices for $c$ are $c = 1$ and $c = e$. The former integrates $e - 2 \approx .719$, and the latter to $1$. We now consider the case that $c \in (1, e)$, and see if our minimizing values for $c$ offer any improvement.

As we had previously stated, for the expression $e^x - c$, there will be exactly on e sign change for $c \in (1,e)$. Suppose that this sign change occurs at $x = x^*$. Then, we can rewrite our integral as

\begin{align*} f(c) = \int_0^1 |e^x - c| \ dx &= \int_0^{x^*} c - e^x + \int_{x^*}^1 e^x - c \\[1em] &= - 2e^{x^*} + 2cx^* + 1 + e - c \end{align*} From our definition of $x^*$, we have that

$$e^{x^*} - c = 0 \quad \Longrightarrow \quad e^{x^*} = c \quad \Longrightarrow \quad \log c = x^*$$

Making these substitutions into the above expression, we have

$$f(c) = -3c + 2c \log c + 1 + e$$

The minimum is obtained by setting $f'(c) = 0$ and checking the endpoints (details omitted), giving us a value of $c = e^{1/2}$. Then $f(e^{1/2}) = (\sqrt{e} - 1)^2 \approx .421$, making $e^{1/2}$ our desired value for our constant $c$.

cnolte
  • 424
  • 1
    I would say not "The former integrates numerically to about .71828...." but "The former integrates to $e - 2 \approx .719$. I'd also change "The minimum of this can be obtained..." to "We can find the minimum of this by setting a derivative to zero and checking the endpoints --- I leave the details to you -- and get the value $c = e^{\frac{1}{2}}$," but that's mostly because I hate the passive voice. – John Hughes Jul 24 '16 at 19:56
  • This is a constructive comment. Part of the reason I've been writing things out like this is to work on communicating ideas that I come across, so thank you. Also, good advice on the $e - 2 \approx .719$. That was laziness on my part – cnolte Jul 24 '16 at 20:02
  • Glad you appreciated it. Your answer was generally good, and I wanted to point out just a few small ways it could be better. In general, I aim to make clear my expectations of the reader (as in "I leave the details to you") so that they can be corrected. Sometimes I even start a reply with "I'm assuming here that dot-products and linear independence are familiar to you, because this sounds like a question from a linear algebra text" or "I'm going to fill in lots of detail; I apologize if you already know most of it; I'm hoping others with similar questions in the future will find it helpful." – John Hughes Jul 25 '16 at 00:55