2

enter image description here

When finding the orthogonal projection for this problem, why were those vectors added? Aren't the vectors normally subtracted for Gram-Schmidt and finding projections?

Also, how do you carry out the Gram Schmidt process for doing part (a)?

user3832862
  • 73
  • 1
  • 3
  • 9
  • Gram Schmidt is used to orthonormalise an ordered list of vectors. One point in having an orthonormal list of vectors is that the projections are trivial to compute (as above). – copper.hat Dec 12 '14 at 03:29
  • The Gram-Schmidt process was already done to $u_1$ and $u_2$. The part below that (where the vectors are being added) is a projection of $\mathbf b$ onto the subspace spanned by the columns of $A$. –  Dec 12 '14 at 03:31
  • And you told me in this question that you knew how to do Gram-Schmidt orthogonalization. Have you forgotten since then? –  Dec 12 '14 at 03:32
  • Yes, unfortunately...I'm not sure how to apply it to this problem. What do I set as v1 and v2? When I set b as v1 and the first column vector as v2, it doesn't work out... – user3832862 Dec 12 '14 at 03:34
  • For some basic information about writing math at this site see e.g. here, here, here and here. –  Dec 12 '14 at 03:35
  • I understand how to do this conceptually, but what two vectors do I even use for Gram Schmidt orthogonalization? If I wanted to do it for the beginning and use matrix A and vector b, then how would one go about setting the vectors for Gram Schmidt? – user3832862 Dec 12 '14 at 03:39
  • For example, if I let v1 = b and v2 = first column of A, then I do not get the correct values for the Gram Schmidt computation. – user3832862 Dec 12 '14 at 03:43

2 Answers2

1

The column space of $A$ is $\operatorname{span}\left(\begin{pmatrix} 1 \\ -1 \\ 1 \end{pmatrix}, \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}\right)$.

Those two vectors are a basis for $\operatorname{col}(A)$, but they are not normalized.

NOTE: In this case, the columns of $A$ are already orthogonal so you don't need to use the Gram-Schmidt process, but since in general they won't be, I'll just explain it anyway.

To make them orthogonal, we use the Gram-Schmidt process:

$w_1 = \begin{pmatrix} 1 \\ -1 \\ 1 \end{pmatrix}$ and $w_2 = \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix} - \operatorname{proj}_{w_1} \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}$, where $\operatorname{proj}_{w_1} \begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}$ is the orthogonal projection of $\begin{pmatrix} 2 \\ 4 \\ 2 \end{pmatrix}$ onto the subspace $\operatorname{span}(w_1)$.

In general, $\operatorname{proj}_vu = \dfrac {u \cdot v}{v\cdot v}v$.

Then to normalize a vector, you divide it by its norm:

$u_1 = \dfrac {w_1}{\|w_1\|}$ and $u_2 = \dfrac{w_2}{\|w_2\|}$.

The norm of a vector $v$, denoted $\|v\|$, is given by $\|v\|=\sqrt{v\cdot v}$.

This is how $u_1$ and $u_2$ were obtained from the columns of $A$.

Then the orthogonal projection of $b$ onto the subspace $\operatorname{col}(A)$ is given by $\operatorname{proj}_{\operatorname{col}(A)}b = \operatorname{proj}_{u_1}b + \operatorname{proj}_{u_2}b$.

  • Yeah, it's a tricky thing! Thanks for your explanation though, I get everything else, I was just getting stuck on the first step for some odd reason. The explanation as to the projections and how they are summed is helpful and illuminating, so thank you. – user3832862 Dec 12 '14 at 03:52
0

Questions (a) and (b) turn out to be the same. By definition, the least squares solution is the $\DeclareMathOperator{\argmin}{\arg\!\min} \argmin_x \Vert Ax-b \Vert_2$. It's easy to prove that the minimum is attained for the orthogonal projection, i.e. for $x:(Ax-b)\perp col(A)$, or in matrix notation, $$ A^t(Ax-b)=A^tAx-A^tb=0 $$ If the columns of $A$ are linearly independent, the solution is $$x=(A^t A)^{-1}A^tb$$ It is both (b) the least squares solution and (a) the coordinates of the orthogonal projection in the basis of the columns-vectors of $A$, $Ax$ being the same vector given in the standard basis of the ambient space.

rych
  • 4,445