1

Let $O(n)$ be the orthogonal (matrix) group. We consider the action of $O(n)$ on the space of symmetric $3$-tensor $S^{0,3}$, where the action is $$ A\mapsto O^* A , \ \ \ (O^*A)_{ijk}= A_{abc} O_{ai} O_{bj} O_{ck}.$$

If I understand correctly, the above representation is irreducible. I want to know if there is any nontrivial subset of $S^{0,3}$: A set $I \subset S^{0,3}$ is called invariant if $A \in I \Rightarrow O^*A \in I$ for all $O \in O(n)$.

Examples are $S^{0,3}$, $\{ 0\}$ and

$$ \mathbb S(r) := \{ A \in S^{0,3} : \sum_{i, j, k} A_{ijk}^2 = r^2\}.$$

I was wandering if there exists other invariant set besides the above.

As a motivation, in the orthogonal representation of symmetric two tensors $A_{ij} \mapsto A_{ab}O_{ai} O_{bj}$, the positive cone

$$ I = \{ A_{ij} : A_{ij}v^i v^j >0\}$$

is an invariant subsets.

Arctic Char
  • 16,972

1 Answers1

2

I'll write $S^kV$ instead of $S^{0,k}.$

First of all: the $O(n)$-representation $S^3V$ is reducible; it has an orthogonal decomposition as $H^3V\oplus r^2V$ where $H^3V$ is the space of traceless symmetric tensors i.e. contracting any two indices gives the zero vectors, and $r^2$ takes $v_i$ to the symmetrization of $v_i\delta_{jk}.$ This is a special case of the decomposition of polynomials into spherical harmonic polynomials. See for example Goodman-Wallach "Symmetry, Representations, and Invariants" Corollary 5.6.12. A simpler case is how a symmetric matrix $M$ decomposes into $\frac{\operatorname{tr} M}n\delta_{ij}$ + traceless part.

There are as many invariant sets as there are subsets of the continuum: for each $R\subset \mathbb R_{>0}$ the set $\{A\in S^3V\mid \sum_{i,j,k}A_{ijk}^2\in R\}$ is invariant, and these sets are all distinct. Any orbit $\{gx\mid g\in O(n)\}$ is an invariant set, and any invariant set is a union of orbits.

Nice enough invariant sets can be classified in a sense using algebra. Any expression written using as a linear combination of these kinds of "words" in $A$ and using summation convention necessarily gives an $O(n)$-invariant polynomial. Your example $A_{ijk}A_{ijk}-1=0$ is of this form. The traceless tensors are given by $A_{iij}A_{jkk}=0.$

A converse holds. Start with a $O(n)$-invariant zero set $W$ and assume it is the zero set of a finite number of real polynomials $f_1,\dots,f_m$ defined on the space of $3$-tensors. (For example, $O(n)$ orbits happen to be Zariski closed.) This implies it is the zero set of the single non-negative polynomial $p=f_1^2+\dots+f_m^2.$ The averaging operator $\rho$

$$\rho(p)(x)=\int_{g\in O(n)}p(gx)d\mu(g),$$

where $d\mu$ is Haar measure, is a linear map acting on polynomials. So $W$ is the zero set of a $O(n)$-invariant polynomial.

We can think of the degree-$k$ part of $p$ as being a $3k$-tensor; when $p$ is applied to $A,$ the degree-$k$ part is applied to $k$ copies of $A.$ Given an even integer $k$ and a permutation $s$ of $\{1,\dots,3k\}$ define $$p_{k,s}=\delta_{s(1)s(2)}\dots\delta_{s(3k-1)s(3k)}.$$ By Goodman-Wallach Theorem 5.3.3, any $O(n)$-invariant $p$ is a finite linear combination of polynomials $p_{k,s}.$ If you sum out the Kronecker deltas in the expression $p_{k,s}(A)$ you get a word in $A,$ for example $\delta_{il}\delta_{jm}\delta_{kn}A_{ijk}A_{lmn}=A_{ijk}A_{ijk}.$ We have shown that $I$ is the zero set of a linear combination of words of this form.

The characteristic polynomial of a symmetric matrix (i.e. 2-tensor) "classifies" the $O(n)$-orbits. A similar but less explicit result holds for $3$-tensors: see https://en.wikipedia.org/wiki/Invariant_theory or Goodman-Wallach Theorem 5.1.1 (and a little extra work to apply this to the real case).

Dap
  • 25,701
  • 20
  • 52