19

I know that we can define two vectors to be orthogonal only if they are elements of a vector space with an inner product.

So, if $\vec x$ and $\vec y$ are elements of $\mathbb{R}^n$ (as a real vector space), we can say that they are orthogonal iff $\langle \vec x,\vec y\rangle=0$, where $\langle \vec x,\vec y\rangle $ is an inner product.

Usually the inner product is defined with respect to the standard basis $E=\{\hat e_1,\hat e_2 \}$ (for $n=2$ to simplify notations), the standard definition is: $$ \langle \vec x,\vec y\rangle_E=x_1y_1+x_2y_2 $$ Where $$ \begin{bmatrix} x_1\\x_2 \end{bmatrix} =[\vec x]_E \qquad \begin{bmatrix} y_1\\y_2 \end{bmatrix} =[\vec y]_E $$ are the components of the two vectors in the standard basis and, by definition of the inner product, $\hat e_1$ and $\hat e_2$ are orho-normal.

Now, if $\vec v_1$ and $\vec v_2$ are linearly independent the set $V=\{\vec v_1,\vec v_2\}$ is a basis and we can express any vector in this basis with a couple of components: $$ \begin{bmatrix} x'_1\\x'_2 \end{bmatrix} =[\vec x]_V \qquad \begin{bmatrix} y'_1\\y'_2 \end{bmatrix} =[\vec y]_V $$ from which we can define an inner product: $$ \langle \vec x,\vec y\rangle_V=x'_1y'_1+x'_2y'_2 $$

Obviously we have: $$ [\vec v_1]_V= \begin{bmatrix} 1\\0 \end{bmatrix} \qquad [\vec v_2]_V= \begin{bmatrix} 0\\1 \end{bmatrix} $$ and $\{\vec v_1,\vec v_2\}$ are orthogonal (and normal) for the inner product $\langle \cdot,\cdot\rangle_V$.

This means that any two linearly independent vectors are orthogonal with respect to a suitable inner product defined by a suitable basis. So orthogonality seems a ''coordinate dependent'' concept.

The question is: is my reasoning correct? And, if yes, what make the usual standard basis so special that we chose such basis for the usual definition of orthogonality?

I add something to better illustrate my question.

If my reasoning is correct than, for any basis in a vector space there is an inner product such that the vectors of the basis are orthogonal. If we think at vectors as oriented segments (in pure geometrical sense) this seems contradicts our intuition of what ''orthogonal'' means and also a geometric definition of orthogonality. So, why what we call a ''standard basis'' seems to be in accord with intuition and other basis are not?

Emilio Novati
  • 64,377

4 Answers4

7

To expand a bit on Daniel Fischer’s comment, coming at this from a different direction might be fruitful. There are, as you’ve seen, many possible inner products. Each one determines a different notion of length and angle—and so orthogonality—via the formulas with which you’re familiar. There’s nothing inherently coordinate-dependent here. Indeed, it’s often possible to define inner products in a coordinate-free way. For example, for vector spaces of functions on the reals, $\int_0^1 f(t)g(t)\,dt$ and $\int_{-1}^1 f(t)g(t)\,dt$ are commonly-used inner products. The fact that there are many different inner products is quite useful. There is, for instance, a method of solving a large class of interesting problems that involves orthogonal projection relative to one of these “non-standard” inner products.

Now, when you try to express an inner product in terms of vector coordinates the resulting formula is clearly going to depend on the choice of basis. It turns out that for any inner product one can find a basis for which the formula looks just like the familiar dot product.

You might also want to ask yourself what makes the standard basis so “standard?” If your vector space consists of ordered tuples of reals, then there’s a natural choice of basis, but what about other vector spaces? Even in the Euclidean plane, there’s no particular choice of basis that stands out a priori. Indeed, one often chooses an origin and coordinate axes so that a problem takes on a particularly simple form. Once you’ve made that choice, then you can speak of a “standard” basis for that space.

amd
  • 55,082
  • So there is not a purely geometrical definition of orthogonality that is captured by a particular inner product? – Emilio Novati Mar 06 '16 at 18:00
  • 3
    @EmilioNovati I’d put it this way: You can often find an inner product that reflects whatever additional structure exists. E.g., if you start with the Euclidean plane, then the usual dot product paired with an orthonormal basis captures the definitions of length and angle in Euclidean geometry. – amd Mar 06 '16 at 19:38
  • 2
    @EmilioNovati Hmm... reading that comment now I’m a bit unsatisfied because it’s starting to sound a bit circular. Let me try again: if the vector space comes with some additional structure that includes a “natural” notion of length, angle and/or orthogonality, you can define an inner product on that space. These ideas are intimately connected and can be defined in terms of each other. – amd Mar 07 '16 at 04:12
  • yes, I agree. The main problem here is to avoid circularity. And the way seems to have a definition of angles that is independent from the inner product and from which we can find a definition of orthogonality that ''selects'' a particular basis as the only basis that is compatible with that definition of angle. This circularity is subtle and difficult to avoid and, it seems to me, it's present also in other answers, in the form of ''angle'' ( @Theodor) or ''Cartesian coordinates system'' ( @Quarky) or ''natural choice of basis'' (@amd). – Emilio Novati Mar 07 '16 at 13:08
  • 1
    A geometrical definition of orthogonality can be seen in Hilbert's Foundations of geometry. Here the concept is introduced in the chapter 7 as a consequance of the axioms of congrence. So it seems that congrence is the kay concept, but I don't see how to introduce a concept of congrence in an abstract vector space without an inner product, and, fon any inner product we have a different ''congruence'', so we return to the starting point. – Emilio Novati Mar 07 '16 at 13:15
  • 2
    @EmilioNovati These circularities reflect the fact that these things are at some level equivalent. Since they’re all structure that’s added to a “bare” vector space, which one you start with is ultimately going to depend the problem you’re trying to solve. – amd Mar 07 '16 at 20:18
5

Your reasoning is good, and, as Daniel Fisher said, the assertion "be orthogonal to" depends only on your inner product.

What makes the standard basis $$\mathcal{C}=(e_1,\dots,e_n) \hspace{0.5cm}\text{ with }\hspace{0.5cm} e_i=(0,\dots,0,\underset{i \text{ rank}}{\underbrace{1}},0,\dots,0)$$ special when you are working in $\mathbb{R}^n$ with an inner product $\langle\cdot,\cdot\rangle$ is that you have the two following points:

  • $\forall x\in\mathbb{R}^n,$ $[x]_\mathcal{C}={}^tx,$

  • If $\mathcal{F}=(f_1,\dots,f_n)$ is an orthonormal basis of $(\mathbb{R}^n,\langle\cdot,\cdot\rangle)$ (which always exists according to Gram-Schmidt's process), then you have : $$\forall u,v\in\mathbb{R}^n, \langle u,v\rangle={}^t[u]_\mathcal{F}[v]_\mathcal{F}=\langle {}^t[u]_\mathcal{F},{}^t[v]_\mathcal{F}\rangle_\mathcal{C}=\langle U, V\rangle_\mathcal{C}.$$

In other words: you can easily compute the coordinates of your vectors in this basis, and, after you have chosen a "good" basis, your inner product can always be written as the inner product given by your method in the standard basis.

Edit : For the geometrical part, I will try to detail my comment : when you represent a vector in the plane, what you really do is to choose a base and draw the (vector of) coordinates in the plane (you well-understand this process when you consider vectorial spaces which are not $\mathbb{R}^n$, the difference being that you can not immediately see the vectors as $n-$tuples and that you have to consider a basis to see the things vectorially).

Then, your question is :

Which basis should I consider to see orthogonality as I am used to see, i.e. that my vectors do a "right angle" ?

(The good definition of angle comes from euclidian geometry, here I suppose that we understand what we want to see on the drawing). And the answer is the second point I already noted : we are in the usual case of $\mathbb{R}^n$ and its usual inner product when we considerate a orthonormal basis for your inner product.

For example, if you consider $(v_1,v_2)=\big((1,0),(1,1)\big)$ and the inner product $\langle x,y\rangle_V={}^t[x]_V[y]_V,$ then as $v_1$ and $v_2$ are not orthogonal in $\mathbb{R}^2$ for the usual inner product, then they won't appear as orthogonal vectors represented in the standard basis (the "angle" between the two being $45^°$), but if you represent them in the base $(v_1,v_2),$ which is orthonormal for your inner product, they will appear as orthogonal vectors.

Balloon
  • 8,624
  • 1
    The problem is that for any basis $V={ \vec v_i}$ the $j-$component of the basis vector $v_i$ is $[v_i]j=\delta{i,j}$ so the set of vectors ${ \vec v_i}$ are the same as ${ \vec e_i}$ when expressed in the basis $V$. – Emilio Novati Mar 06 '16 at 16:29
  • 1
    Forget my past comment sorry, which that I wanted to write is : why do you call that a problem ? – Balloon Mar 06 '16 at 16:48
  • @ChristianBlatter: I've added something that, I hope, can better illustrate my problem. But maybe that it's a ''void'' problem? – Emilio Novati Mar 06 '16 at 17:17
  • 2
    I understand your problem : when we usually represent vectors in $\mathbb{R}^2,$ we implicitely chose the standard basis before doing our drawing. If we would chose an other basis (i.e. that the vectors we draw are in reality the vectors of coordinates of the vectors in our other basis), orthogonality would not appear as we are used to see : it will depend of the basis. In reality it will be visible on the drawing (as we are used to see) only if the base that you chose is orthonormal for your inner product. – Balloon Mar 06 '16 at 17:44
3

The unit base vectors $e_1$,$e_2$: form an orthonormal basis.

Let us take a look what this requires:

  • all elements in $M:=\{e_1,e_2,...,e_n \}$ are linearly independent
  • the vectors $e_1$,$e_2$,...,$e_n$ are pairwise orthogonal, so their dot product $\langle e_i,e_k \rangle=0 $, $\forall i,k \leq n$ and $i\neq k$
  • they are normalized, so $||e_i||=1$

You are right, there are more vectors that fulfill this criteria. In fact, there is an uncountably infinite amount of them. But what makes the standart basis so appealing?

  1. They are easy to calculate

For any vector $e_i$ you set all but one position to zero. The only nonzero entry has the only requirement, that its position is unique compared to the other unit vectors.

If you tried to create a basis starting from a non-unit vector $v$, you would have to ensure that they are linearly independent, pairwise orthogonal and normalized.

  1. They have good numerical properties You will not run into rounding errors while trying to calculate $||e_i||$ This is a mighty advantage.

Furthermore you ask, if the dot product is a coordinate dependent concept. Remember that a vector $v=\left( \begin{array}{c} a\\ b\\ \end{array} \right)$ has a magnitude and a direction. So the entries $a,b$ have a direct relationship on the direction the vector $v$ will be facing. So in a sense, yes, the dot product is as coordinate dependent as the overall appearance of the vector is.

But what happens if you convert your vector $v$ to polar coordinates, such that $v=(angle, length)$. What would you mean by coordinate dependent now?

So instead of saying the dot product is coordinate dependent, say it is magnitude and angle dependent. That is even how the dot product is defined, because

$\langle v,u \rangle=||u||||v||cos(alpha)$

Theodor Johnson
  • 342
0

Let me try to add my personal intuition to this conversation and see if that helps you a bit. If you take the standard basis vectors, and draw them in the Cartesian coordinate system, then you will note that they satisfy your intuition of what we think of perpendicular vectors. What we further note is that any two vectors that are orthogonal as per the inner product in this basis do in fact satisfy the same intuition of being perpendicular as per our intuition. If you pick a different basis, lets say ${v_1, v_2}$ then as you noted these two vectors will be orthogonal as per the inner product defined in their basis. In the standard basis, the angle between them is $\theta = arccos(v_1 . v_2) / |v_1||v_2|$. Any set of vectors that are orthogonal in this new basis, will in fact have the same angle $\theta$ between them in the standard basis.

To summarize, if you pick any arbitrary basis, the angle between the vectors in that basis, when drawn in the standard basis, is the same as the angle between any two vectors, again when drawn in the standard basis, that are orthogonal in this new basis. So, it makes sense to pick the standard basis as the standard because the basis vectors there are in fact 90 degrees (fitting well with our intuition). And consequently all the vectors that are orthogonal in that basis will also have 90 degree angle between them.

Quarky Quanta
  • 101