24

I am learning linear algebra, and I am a bit confused by the dot product and how the answer to the process turns out to be a scalar rather than a matrix.

For $2$ vectors with $2$ components, I learned that dot product is equivalent to a $1 \times 2$ row vector left multiplied by a $2 \times 1$ column vector. The result of such a multiplication should result in a $ 1 \times 1 $ vector. But I am learning that the dot product somehow transforms the result into a scalar, rather than a $ 1 \times 1 $ vector.

Maybe I am missing something but the difference between a $ 1 \times 1 $ vector and a scalar seems important, because you can multiply a scalar by a matrix of any size, but you can only left-multiply a $ 1 \times 1 $ vector by another matrix with $1$ row.

Thanks for any help in understanding this.

Widawensen
  • 8,517
treeorriffic
  • 249
  • 2
    I guess what I am confused about is that I learned that matrix multiplication is only defined when the number of columns on the left matrix = the number of rows in the right matrix. In which case, a 1x1 matrix could only be left multiplied by other 1 row matrices, whereas a scalar can multiply by a matrix of any size. So simply converting a 1x1 matrix to a scalar seems to removes some of the restrictions that matrix multiplication has. – treeorriffic Mar 01 '17 at 17:06
  • 4
    You're not wrong, of course. But sometimes mathematicians take shortcuts. See my answer below for more. – Bobbie D Mar 01 '17 at 17:11
  • 1
    You might find my answer here useful – Ben Grossmann Mar 01 '17 at 17:13
  • 1
    If you want to replace scalars with $1 \times 1$ matrices, you need to replace the scalar product with the tensor product (i.e. kronecker product). i.e. $aM = [a] \otimes M = M \otimes [a]$ for any scalar $a$ and matrix $M$. Of course, if $M$ is $1 \times n$ then the matrix product works too: $[a]M = [a] \otimes M$. Similarly, $M[a] = M \otimes [a]$ if $M$ is $n \times 1$. –  Mar 02 '17 at 01:23
  • A related issue is: is a vector of length $n$ the same as a $n\times 1$ matrix? – Federico Poloni Mar 02 '17 at 09:39
  • This had not really occurred to me. So we do not have $\langle x,y\rangle=x^{\rm t} y$, but $(\langle x,y\rangle)=x^{\rm t} y$ – Carsten S Mar 02 '17 at 13:39

5 Answers5

18

Sometimes, we just say that a $1\times 1$ matrix is the same as a scalar. Afterall, when it comes to addition and multiplication of $1\times 1$ matrices vs addition and multiplication of scalars, the only difference between something like $\begin{bmatrix}3\end{bmatrix}$ and $3$ is some brackets. Consider $$(3+5)\cdot 4 = 32 \\ (\begin{bmatrix} 3\end{bmatrix} + \begin{bmatrix} 5\end{bmatrix})\begin{bmatrix} 4\end{bmatrix} = \begin{bmatrix} 32\end{bmatrix}$$ The algebra works out exactly the same. So sometimes it's not ridiculous to think of $1\times 1$ matrices as just another way of writing scalars.

But if you do want to distinguish the two, then just think of the formula $a\cdot b = a^Tb$ as a way of finding out which scalar you get from the dot product of $a$ and $b$ and not literally the dot product value itself (which should be scalar). That is, we calculate the dot product of $\begin{bmatrix} 1 \\ 2\end{bmatrix}$ and $\begin{bmatrix} 3 \\ 4\end{bmatrix}$ by using the formula $$\begin{bmatrix} 1 \\ 2\end{bmatrix}^T\begin{bmatrix} 3 \\ 4\end{bmatrix} = \begin{bmatrix} 1 & 2\end{bmatrix}\begin{bmatrix} 3 \\ 4\end{bmatrix} = \begin{bmatrix} 11\end{bmatrix}$$ and then say that this tells us that the dot product is really $11$. So the formula $a^Tb$ is just an algorithm we use to find the correct scalar.

You can view it either way. It doesn't really make a difference.

Bobbie D
  • 2,081
14

You are not wrong and it's always good to examine statements very carefully. We usually do not distinguish 1$\times$1 matrices from scalars, and in some sense you can think of scalar multiplication as a special rule for when one of the matrices is 1$\times$1. But the truth is that it is a convenient abuse of notation.

Here's one other thing to think about. A $1 \times 1$ real matrix is supposed to represent a linear mapping from a one-dimensional real vector space into a one-dimensional real vector space--essentially just $\mathbb R$ into $\mathbb R$. The only such mappings are those that take $x \mapsto ax$ for some fixed real number $a$. But this mapping is entirely determined by that real number $a$, so they are essentially equivalent.

Philip Hoskins
  • 2,095
6

It is true that you can only multiply a $m \times n$ matrix by a $n \times p$ matrix, i.e., the column size of the left matrix has to match the row size of the right matrix. With this, we can conclude that a product of a $1 \times 1$ matrix by a $n \times p$ matrix makes no sense for $n > 1$.

That being said, the space of matrices is a vector space, so it has the multiplication between scalars and matrices. So it makes sense to multiply a scalar by a matrix. Once you realize the space of scalars and $1 \times 1$ matrices can be identified, the confusion goes away.

Integral
  • 6,674
5

There are two ways to look at dot products:

  • The scalar product of two vectors $(v_1,...,v_n)$ and $(w_1,...,w_n)$ can be simply defined as the sum $v_1w_1+\cdots+v_nw_n$, and so its a scalar by definition, or as you learned it
  • as a special kind of matrix multiplication.

In the latter case, the result is indeed a $(1\times 1)$-matrix, and as the comments already stated, its just a convention to call this a scalar, as "the only difference are the brackets". Of course, when you view your result as a matrix, you can only multiply it by $(1\times m)$-matricies. But here is the clue: the space of $(1\times 1)$-matricies is (for all practical purpose) equivalent to the space of scalars. This means, there is a bijective linear map from $K$ to $K^{1\times1}$, even an algebra isomorphism. So, to use the result in more general fashion, we allow us to interpret it as a scalar. Further, you often use the result of a dot product only to multiply it with vectors. And vectors are not so far from the $(1\times m)$-matricies where you are allowed to use your $(1\times1)$-matricies anyway.

M. Winter
  • 30,828
-2

A dot product is not really a scalar, but it behaves just like one. In math we call that an ISOMORPHISM. For every dot product result, there is a corresponding real number that you get by simply removing the brackets. All of the operations you do with the 1x1 matrix correspond to the same operations done with a real number. Multipliction of a matrix by a 1x1 matrix is defined as multiplcation by the corresponding scalar.

Recommend you look up "isomorphism" in a math dictionary. To do this, you may need to brush up on your German language skills.

richard1941
  • 1,051