What is the idea behind the $^2$ in the mean squared error?

Question

Mean Squared Error: $\operatorname{MSE}=\frac{1}{n}\sum_{i=1}^n(\hat{Y_i} - Y_i)^2.$ <-- what is the purpose of the '$^2$' in here?

Mean Absolute Error: $\mathrm{MAE} = \frac{1}{n}\sum_{i=1}^n \left| f_i-y_i\right| =\frac{1}{n}\sum_{i=1}^n \left| e_i \right|$

Better behaved than the absolute value in terms of differentiability maybe? — Eckhard, Dec 14 '13 at 21:02
@K.Rmth If you take the absolute value as with the 'Mean Absolute Error' you always a positive as well. — Golan, Dec 14 '13 at 21:54
@Eckhard In which case can't you differentiate the the 'Mean Absolute Error'? — Golan, Dec 14 '13 at 21:54
@Golan: it depends on you specific application but in general the absolute value function $x\mapsto|\cdot|$ is not differentiable at $x=0$. — Eckhard, Dec 14 '13 at 21:59
@Eckhard Why would I differentiate it at x = 0, instead of letting the value being undefined at that point? — Golan, Dec 14 '13 at 22:00
Probably the same idea as in Motivation behind standard deviation? — , Dec 14 '13 at 22:23

score 1 · Answer 1 · answered Dec 14 '13 at 21:53

1

In addition to the reasons pointed out to you by K.Rmth and Eckhard in their comments, consider that since $x^2 > |x|$ for $|x| > 1$ while $x^2 < |x|$ for $|x| < 1$ and thus the mean-square error penalizes large errors more that the mean absolute error does, but is more forgiving of small errors. Some people think that this emphasis on penalizing large errors more than small errors is a good thing; others don't.

answered Dec 14 '13 at 21:53

Dilip Sarwate

26,411

Thanks! Could you give examples when 'penalizing large errors more than small errors' is a good thing and when it is a bad thing. – Golan Dec 14 '13 at 21:59
Nice explanation $;)$ – K. Rmth Dec 14 '13 at 23:41

score 0 · Answer 2 · answered Dec 14 '13 at 21:14

0

It is the same square as in the pythagorean theorem -- you are trying to find the Euclidean distance between the expected and the actual results.

answered Dec 14 '13 at 21:14

Igor Rivin

26,372

I don't see the relation between this and the pythagorean theorem. Could you explain that, please? – Golan Dec 14 '13 at 21:53

What is the idea behind the $^2$ in the mean squared error?

2 Answers2