I have the intuition that PCA (Principal Component Analysis) can be formulated as the following constrained optimization problem.
Let $A$ be a $n\times n$ symmetric and positive semi-definite real matrix. Let
$$L = \sum_{i=1}^m \vec v_i^TA\vec v_i$$
where $\vec v_i$ are $m$ real $n$-dimensional vectors. Now maximize $L$ with respect to the vectors $\{v_i\}$, subject to the constrain that these vectors be of unit norm and orthogonal to each other. Assume that $m \le n$.
For the sake of educating my intuition, I need to prove or disprove the following statement:
The maximization of $L$ defined above (with the constrains given) results in the $\vec v_i$ being the eigenvectors of $A$ associated to its $m$ largest eigenvalues.
I tried to do this by Lagrange multipliers, but it gets very messy and I think I am overlooking some simple way of thinking about this.
