14

I am not a mathematics student but somehow have to know about L1 and L2 norms. I am looking for some appropriate sources to learn these things and know they work and what are their differences. I am asking this question since there are lots of stuff on this matter in the internet and I am looking for a simple applicable one to understand this.

To add more, I want to understand why/how/when these norms are used in optimization problems? And maybe, how these cost functions are solved.

user3482383
  • 391

1 Answers1

15

Just the norm? The norm acts on a vector and any norm has to satisfy three properties. Given a vector space V over a field F, the norm must

  1. $\|x\|\ge0$ for all x and $x=0\Leftrightarrow\|x\|=0$
  2. $\|ax\|=|a|\|x\|$ for all $a\in\mathbb F, x\in V$
  3. $\|x+y\|\le \|x\|+\|y\|$ (triangle inequality)

You ask about the L1 and L2 norms. The L1 norm is the sum of the absolute value of the entries in the vector. The L2 norm is the square root of the sum of the squares of entries of the vector. In general, the Lp norm is the pth root of the sum of the entries of the vector raised to the pth power.

$$\begin{split}\|x\|_1&=\sum |x_i|\\ \|x\|_2&=\sqrt{\sum x_i^2}\\ \|x\|_p&=(\sum |x_i|^p)^{1/p}\end{split}$$

It is fairly common to square the l2 norm to get $\|x\|^2_2=\sum x_i^2$

For more use Wikipedia and go to the examples section.

weirdsmiley
  • 103
Vons
  • 11,285
  • 1
    Actually I was looking for some information useful for optimization problems where you have a cost function which includes these two norms – user3482383 Feb 07 '21 at 05:56
  • 3
    It would be useful to mention L1 norm is also called the Manhattan distance and the L2 norm is known as the Euclidean distance. – misiu_mp Dec 20 '23 at 15:29
  • 1
    @misiu_mp I think the norms are not called so... They're the metrics induced by them which have such names. – Nothing special May 29 '24 at 13:28