4

It is known that for matrices, under the correct conditions, the eigenvalues of the matrix are continuous when considered as functions of some perturbation. See for example, here . Similarly, under somewhat more restrictive conditions, one can also say that the eigenvectors can be chosen continuous. See for example, here.

I am interested in whether anything can similar can be said for tensor rank decompositions. For example, suppose an $M$-dimensional tensor, $\mathcal A$, can be written in the form

$\mathcal A \equiv \sum_{r=1}^R \lambda_r~ \mathbf{a}_1^r\otimes \mathbf{a}_2^r\otimes \mathbf{a}_3^r ...\otimes \mathbf{a}_M^r $

where the $\lambda_r$ are scalars (akin to eigenvalues), the $\mathbf{a}_i^r$ are vectors (i.e. dimension 1 tensors, akin to eigenvectors) and the smallest possible value of $R$ is the rank of $\mathcal A$. Finding the minimal such expansion efficiently is an open problem. However, I am interested in the much easier case where one already has this expansion of $\mathcal A$ (and that such expansion may or may not be minimal). This could for example be gotten by using standard software like PARAFAC to approximately solve the tensor decomposition problem.

Given this, suppose we allow $\mathcal A$ to change. For example, suppose $\tilde{\mathcal A} (t) = \mathcal A +t ~\mathcal B$ for some variable $t$ and an appropriately chosen tensor $\mathcal B$. Under what conditions can we define

$\tilde{\mathcal A}(t) \equiv \sum_{r=1}^R \tilde{\lambda}_r(t)~ \tilde{\mathbf{a}}_1^r(t)\otimes \tilde{\mathbf{a}}_2^r(t)\otimes \tilde{\mathbf{a}}_3^r(t) ...\otimes \tilde{\mathbf{a}}_M^r(t) $

such that $\tilde{\lambda}_r(t)$ and $\tilde{\mathbf{a}}_i^r(t)$ are continuous functions of $t$? For example, what if the tensors were real and totally symmetric (akin to a real symmetric matrix)? Or what if the tensor was positive definite (i.e. a symmetric tensor such that the $M$-fold trace with any vector is positive)? I am aware that the tensor decomposition is much less unique than the eigenvalue decomposition - even the "eigenvalues", $\lambda_r$, are ambiguous unless we choose the $\mathbf{a}_i^r$ to be of fixed norm. That is both a blessing (because it suggests there might be one of the many possibilities that is continuous) and also a curse (because it makes it hard to make definitive progress).

I suspect this may be a hard problem because if these parameters were smooth functions, it seems like we could solve the tensor decomposition of one (easy to approximate) tensor and then slowly deform it into another (hard to approximate) tensor. But I am interested what is known about the problem and my internet searches have turned up nothing.

Troy Van Voorhis
  • 180
  • Already tensor rank is not (semi)continuous. There are tensors (even in the 2x2x2 case) such that almost all pertubations have higher or lower tensor rank (pick a simple tensor for the first phenomenon or the symmetric tensor corresponding to $x^2y$ for the second). Moreover, linearly interpolating between two low-rank tensors usually increases their rank, so this behaviour would be quite special, at least if $R$ is close to the tensor rank of $\mathcal A$. – Leobeth Jun 10 '25 at 14:21
  • @Leobeth - yes, there are pathological cases where the rank changes. But in a measure theory sense those are rare (at least for large balanced tensors). So if you choose a tensor with the generic rank, almost every tensor has the same rank as your starting point (which is partway to what we want). I am interested in knowing what other conditions might be needed to ensure continuity. – Troy Van Voorhis Jun 10 '25 at 14:46
  • How do you define an eigenvector of $\mathcal A$? If you have the definition then you can try to determine how the tensor decomposition varies with $\mathcal A$. – Ted Black Jun 15 '25 at 19:32
  • @TedBlack I am unsure if there is another formula besides the canonical decomposition here. For diagonalizable matrices, there is the canonical (diagonal) form as well as the eigenvalue equation for the eigenvectors. It is the eigenvalue equation that one uses to prove smoothness and I am not aware of any similar relationship here. – Troy Van Voorhis Jun 16 '25 at 00:42
  • I was not aware that the canonical decomposition is equivalent to an eigenequation for higher order tensors. If you treat $\mathcal{U}^r=\mathbf{a}_1^r\otimes \mathbf{a}_2^r\otimes \mathbf{a}_3^r ...\otimes \mathbf{a}_M^r$ as an eigenvector in tensor product space and $\lambda^r$ as its eigenvalue then since $\mathcal{A} \mathcal{U}^r = \lambda^r \mathcal{U}^r$ you can generate expressions for $\partial\mathcal{U}^r/\partial\mathcal{A}$. – Ted Black Jun 16 '25 at 12:18

0 Answers0