Geometric interpretation of eigenvector centrality/PageRank

Question

I am trying to achieve a better understanding of the relationship between different uses of eigenvectors, in particular how network applications (eigenvector centrality, PageRank) relate to dimension reduction applications (like principal components analysis). A geometric interpretation would be helpful. I have (I think) a reasonable geometric understanding of eigenvectors as a rotation of data points (or a covariance matrix) in PCA. I also understand the general idea of eigenvector centrality and its approximation as the result of a recursive multiplication of an adjacency matrix. And I understand that there is an interpretation in terms of random walks through the network. But I have a hard time connecting these different facts back to the more concrete geometrical interpretation of PCA. Is there a way to visualize eigenvector centrality in terms of rotation, analogous to PCA?

My question is related to this older question but narrower in scope: I am not asking about the general principle underlying many applications of eigenvectors, but trying to 'translate' what I understand about PCA to eigenvector centrality in networks, to gain a better understanding of the latter, ideally with a geometric interpretation I can visualize (while recognizing that this involves more than three dimensions for nontrivial networks, so that will be tricky).

Answer 1

I'm going to try to answer my own question but still appreciate more insights.

Bonacich (1972) actually states something related in his paper proposing eigenvector centrality. For an adjacency matrix W representing friendship:

"$S_i$ would be individual i's propensity to form friendships...[the adjacency matrix] $W$ should be close to $S_i S_j$. In matrix notation, we want to calculate a column vector S such that the sum of squared differences between $SS'$ and $W$ is minimized. $W$ is like a peculiar correlation matrix with zero communalities and the criterion for $S$ is identical to the criterion for the first principal components factor of a correlation matrix (i.e. minimizing squared differences)."

That is, the first eigenvector. So the trick is to see the adjacency matrix as analogous to the covariance (or correlation) matrix in PCA. PCA rotates the covariance matrix, and the operation on the adjacency matrix is equivalent.

Reference: Bonacich, P. (1972) ‘Factoring and weighting approaches to status scores and clique identification’, The Journal of Mathematical Sociology, 2(1), pp. 113–120. Available at: https://doi.org/10.1080/0022250X.1972.9989806.

Answer 2

Mmmh, let me try.

In PCA, your do not have a graph with vertices and edges, you have data points which are numerical values in several dimensions. Now, humans have some difficulty in looking at data in more than 2 or 3 dimensions, so, you are trying to simplify the depiction of data by reducing to fewer dimensions. 2 or 3, so that you can see (!) how the data is clustered or correlating or ...
Turns out, that one of the best ways of doing that is to start with the dimensions where the data points 'differs most'; these 'dimensions' can be linear combinations of the data point's dimensions and linear algebra, eigenvectors and eigenvalues are a tool to identify these 'best' dimensions or directions.

In page rank, you have a real graph with vertices and edges that is based on the mutual linking of web pages (or sites). You are looking for the web pages that are linked to by many other sites as some sort of relevant reference. Turns out, that one way of ranking these pages is just looking at the eigenvector of the largest eigenvalue to a standard matrix that you can build from the graph as defined with vertices and edges.

I would say that the tools may be similar but the use cases are very much unrelated.