If $\sum\limits_i\sum\limits_j\alpha_{ij}x_iy_j$ converges for every square integrable $(x_n)$ and $(y_n)$, then the order of the sums commutes

Question

Let $(\alpha_{ij})$ be a square infinite matrix such that for all $x=(\xi_{n}),y=(\eta_{n}) \in \ell ^{2}$ we have that ${\displaystyle \sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\alpha_{ij}\xi_{i}\eta_{j}}$ converges.

The question: Is it correct to say that $$\sum_{i=0}^{\infty}\sum_{j=0}^{\infty}\alpha_{ij}\xi_{i}\eta_{j}=\sum_{j=0}^{\infty}\sum_{i=0}^{\infty}\alpha_{ij}\xi_{i}\eta_{j} \:\:?. \tag{$\bigstar$}$$

Remark: We know that if the serie converges absolutely, then every rearrangement converges. I do not know if in this case it is necessary to show that the series converges absolutely to be able to demonstrate $(\bigstar)$, however, I have not been able to show this absolute convergence.

Answer 1

Interesting question! I feel like one could construct a counter-example to your conjecture using a square infinite matrix where summation does not interchange (e.g. $B=(\beta_{ij})_{i,j=0}^\infty$ with $$ \beta_{ij}=\frac{\operatorname{sgn}(i-j)}{2^{|i-j|}}, $$

see Problem 9.16 in the book "Counterexamples in Analysis" by Gelbaum & Olmsted) and the fact that, very handwavingly speaking, $\ell_2$ sequences have to "converge better than $1/\sqrt n$". So something like $B$ or maybe $\hat\beta_{ij}=\beta_{ij}\sqrt{ij}$ could work; although I didn't play around with this idea too much for now.

However, one can give a sufficient criterion such that the sums in your problem interchange.

Theorem. Let a square infinite matrix $A=(\alpha_{ij})_{i,j=0}^\infty$ be given such that $A$ is square-summable (c.f. Hilbert-Schmidt operator), so $$ \sum_{i,j}|\alpha_{ij}|^2<\infty. $$ Then $\sum_{i,j}\alpha_{ij}\xi_i\eta_j=\sum_{j,i}\alpha_{ij}\xi_i\eta_j$ for any $x=(\xi_n)_{n=0}^\infty$, $y=(\eta_n)_{n=0}^\infty\in\ell_2$.

Proof. We know that if $\sum_{i,j}|b_{ij}|<\infty$ for some doubly indexed infinite sequence $(b_{ij})_{i,j}$ then we can interchange the sums (e.g. Fubini for sums). Now this is a simple consequence of Cauchy-Schwarz / Hölder's inequality on $\ell_2$ since

$$ \sum_{i,j}|\alpha_{ij}\xi_i\eta_j|=\sum_{i}|\xi_i|\sum_j|\alpha_{ij}\eta_j|\leq \sum_{i}|\xi_i|\left( \sum_j|\alpha_{ij}|^2 \right)^{1/2}\underbrace{\left(\sum_j|\eta_{j}|^2 \right)^{1/2}}_{=\|y\|_2} $$

and doing that trick twice yields

$$ \sum_{i,j}|\alpha_{ij}\xi_i\eta_j|\leq\ldots\leq\left( \sum_{i,j}|\alpha_{ij}|^2 \right)^{1/2}\|x\|_2\|y\|_2<\infty.\tag*{$\blacksquare$} $$

Note however, that this criterion is not a necessary one. The identity matrix $I=(\delta_{ij})_{i,j=0}^\infty$ is obviously not square-summable but the double sum in question is interchangable (since it simply becomes the scalar product on the space of real $\ell_2$ sequences, which converges absolutely using the same trick as above just once). I hope my answer somehow helps you out regardless!