Linear Algebra Versus Functional Analysis

Question

As it is mentioned in the answer by Sheldon Axler in this post, we usually restrict linear algebra to finite dimensional linear spaces and study the infinite dimensional ones in functional analysis.

I am wondering that what are those parts of the theory in linear algebra that restrict it to finite dimensions. To clarify myself, here is the main question

Question.
What are the main theorems in linear algebra that are just valid for finite dimensional linear spaces and are propagated in the sequel of theory, used to prove the following theorems?

Please note that I want to know the main theorems that are just valid for finite dimensions not all of them. By main, I mean the minimum number of theorems of this kind such that all other such theorems can be concluded from these.

Answer 1

In finite-dimensional spaces, the main theorem is the one that leads to the definition of dimension itself: that any two bases have the same number of vectors. All the others (e.g., reducing a quadratic form to a sum of squares) rest on this one.

In infinite-dimensional spaces, (1) the linearity of an operator generally does not imply continuity (boundedness), and, for normed spaces, (2) "closed and bounded" does not imply "compact" and (3) a vector space need not be isomorphic to its dual space via canonical isomorphism.

Furthermore, in infinite-dimensional vector spaces there is no natural definition of a volume form.

That's why Halmos's Finite-Dimensional Vector Spaces is probably the best book on the subject: he was a functional analyst and taught finite-dimensional while thinking infinite-dimensional.

Answer 2

To add to avs's answer, in finite dimensions you have the result that a linear operator $V\to V$ is injective iff surjective. This fails in infinite dimensions.

Answer 3

For important results of linear algebra that need the space to be finite dimensional, besides the already mentioned:

An endomorphism is injective if and only if it is surjective.

I would nominate:

For every endomorphism $f: V \to V$ there is a polynomial $p$ such that $p(f)= 0$.

While not all presentations focus on this, this fact is not hard to prove yet pretty powerful. (For example over an algebraically closed field it implies immediately that there is an eigenvalue.)

Another, but that's almost cheating:

Every ascending chain of subspaces becomes stationary.

That is, finite dimensional vector spaces are the notherian modules over fields.

In addition that one can represent linear maps conveniently via matrices and has the determinant function helps (also already mentioned), too.

Especially the former is more a practical consideration, but I do think it is part of the reason why many courses restrict completely to finite dimensional spaces.

Answer 4

Working in finite-dimensional linear spaces, and knowing that dimension is independent of basis leads to many interesting properties that do not hold for infinite-dimensional spaces.

For example, every square matrix $A$ must have a minimal polynomial $m$ for which $m(A)=0$, which follows because the linear space of $n\times n$ matrices has dimension $n^2$, which means that $\{ I,A,A^2,\cdots A^{n^2} \}$ must be a linearly-dependent set of matrices. So there is a unique monic polynomial of lowest order for which $m(A)=0$. If $m(\lambda)=\lambda^k+a_{k-1}\lambda^{k-1}+\cdots +a_{1}\lambda+a_0$, then it can be seen that $A$ is invertible iff $m(0)=a_0\ne 0$. Indeed, if $a_0\ne 0$, $$ I=-\left[\frac{1}{a_0}(A^{k-1}+a_{k-1}A^{k-2}+\cdots+a_1 I)\right]A \\ = -A \left[\frac{1}{a_0}(A^{k-1}+a_{k-1}A^{k-2}+\cdots+a_1 I)\right]. $$ So $A$ has a left inverse iff it has a right inverse, and, in that case, the left and right inverses are the same polynomial in $A$. That's most definitely not true in infinite-dimensional spaces. Most generally, $m(\lambda) \ne 0$ iff $A-\lambda I$ is invertible; and $m(\lambda)=0$ iff $A-\lambda I$ has a non-trivial kernel. So $A-\lambda I$ is non-invertible iff it has a non-trivial kernel, which consists of all eigenvectors of $A$ with eigenvalues $\lambda$. The rank and the nullity of any $n\times n$ matrix are nicely related, which also does not happen in infinite dimensional spaces, even if the kernel and complement of the range are finite-dimensional.

Then if you're working over $\mathbb{C}$, the minimal polynomial factors as $$ (A-\lambda_1 I)^{r_1}(A-\lambda_2 I)^{r_2}\cdots(A-\lambda_k I)^{r_k}=0. $$ Such a factoring leads to the Jordan Canonical Form, which is also something you don't generally get in infinite-dimensional spaces. And $A$ can be diagonalized iff the minimal polynomial has no repeated factors, which basically comes down to a trick with Lagrange polynomials $p_k$, which are the unique $n-1$ order polynomials defined so that $p_k(\lambda_j)=\delta_{j,k}$. Then $$ 1 \equiv \sum_{k=1}^{n}p_k(\lambda) \implies I = \sum_{k=1}^{n}p_k(A), $$ and $(A-\lambda_k I)p_k(A)=0$. That's how you get a full basis of eigenvectors for $A$ when the minimal polynomial has no repeated fators. The matrices $p_k(A)$ are projections onto the eigenspace with eigenvalue $\lambda_k$, and every vector can be written in terms of the ranges of these projections, which are eigenvectors. Normal (and selfadjoint) matrices $N$ are special cases where the minimal polynomial has no repeated factors because $\mathcal{N}((N-\lambda I)^2)=\mathcal{N}(N-\lambda I)$ for all $\lambda$. This algebraic formalism is not generally available for infinite-dimensional spaces.

Though a determinant is not essential for finite-dimensional analysis, it is nice, and there is no determinant for the general infinite-dimensional space.

Answer 5

I think several: all the norms are equivalent, all subspace is closed and others. But what struck me most, when I learned it, is the existence of linear discontinuous functions that are not continuous at any point despite its linearity (and this dont need both spaces were infinite dimensional, for example there exist discontinuous linear functionals (i.e, to values in the field $\Bbb R$ or $\Bbb C$) which are those whose kernel is a dense hyperplan). This last fact is not possible in finite dimension where all the linear functions are continuous.

Answer 6

This is mainly a comment for the answer given by avs, but perhaps too long, so I post it as an answer too.

I would add that when $V$ is infinite-dimensional and carrying a topology there are two different notions of the dual vector space for it:

1. algebraic dual of $V$ consists of all linear functionals on $V$.
2. topological dual of $V$ consists of the bounded (continuous) functionals on $V$.

Both of these duals are usually denoted by $V^*$ and you need to distinguish them from the context.

The algebraic dual is never isomorphic to $V$ when $V$ is infinite dimensional. Thus we have the following classification:

$V$ is finite-dimensional if and only if it is isomorphic to its algebraic dual $V^*$.

Answer 7

To add to the previous answers, which both raise good points, I want to add a few other things which I feel are important.

It is true, as avs has stated, that linear maps are in general not continuous anymore. However, we do have the following theorem which I personally have found very useful:

Let $F:X \rightarrow Y$ be a linear map, $X,Y$ normed vector spaces. Then the following are equivalent:

1.) $F$ is continuous

2.) $F$ is continuous at a point

3.) There is a $c>0$ such that $|F(x)|_Y \leq c|x|_X$ for every $x \in X$.

This holds in vector spaces of any dimension and can simplify the process of proving continuity considerably by simply considering any of the above equivalent statements which may be easier to prove in certain circumstances.

Another thing is in infinite dimensional normed spaces, the (closed) unit ball is never compact. This is quite inconvenient.

Answer 8

If you don't assume the Axiom of Choice, the theorem that every vector space has a basis is valid in finite dimensional spaces but not in infinite dimensional spaces (in general). Even if you do assume the Axiom of Choice, the fact that it is required when dealing with infinite dimensional spaces illustrates that even in the case of results which hold in any vector space, results which are constructive in the finite dimensional case are sometimes nonconstructive in the infinite dimensional case.

Answer 9

No analog of Determinant. Functional analysis also includes non linear analysis, Differential Calculus in Banach spaces and surprisingly there is not much difference from finite dimension case in the treatment of Calculus.