Is the solution to Mean Squared Error also the one to Mean Absolute Error?

Question

When I was dealing with optimization problem and deciding whether to minimize MSE or MAE, I had the following question: Is the solution to Mean Squared Error also the one to Mean Absolute Error? If yes, why would we choose one model over the other?

Answer 1

Consider $x_1,\ldots,x_n\in \mathbb R$.

The solution of the MSE problem is $\text{argmin}_x \sum_{k=1}^n (x_i-x)^2 = \{\frac 1n \sum_{k=1}^n x_i\}$ which you might know as the sample mean.

The solution of the MAE problem is $\text{argmin}_x \sum_{k=1}^n |x_i-x| = \{\text{medians of }(x_1,\ldots,x_n) \}$.

In general, the sample mean is not a median of $(x_1,\ldots,x_n)$, so the solutions of both problems are different.

---

Using this result with the distribution that puts mass $\frac 1n$ on each sample, the difference between the two solutions must be less than $\displaystyle \sqrt{\frac 1n\sum_{k=1}^n (x_k-\overline x)^2}$ where $\overline x = \frac 1n \sum_{k=1}^n x_k$.

Answer 2

The answer is no. The MAE and MSE problems are not generally equal.

It generally depends on the type of your problem and what you exactly pursue. For example let the following optimization problems ($x,x_0\in\Bbb R^2$)$${P1: MAE\\\min ||x-x_0||_1\\s.t.\\ ||x||_2=1}$$$${P2: MSE\\\min ||x-x_0||_2\\s.t.\\ ||x||_2=1}$$Both the problems $P_1$ and $P_2$ stand for minimizing the distance between our parameter to a desired value in some sense. As an example, let $x_0=(1,2)$. Then the optimal solutions of the two optimization problems are $$x_1^*\approx (0.7071,0.7071)\\x_2^*\approx (0.4472,0.8944)$$which are not equal. Also a plot of the error functions is as follows