Quantum probability vs. Kolmogorov's probability

Question

It has been argued that quantum probability requires a different treatment from other random events, and a different formalization is needed than that provided by Kolmogorov's axioms. And as a physicist by training I have read such statements dozens of times.

However, I have never seen a simple and concrete example showing that some result of quantum mechanics cannot be reduced to the usal treatment random events over a measure space $(\Omega, \mathcal{F},\mathbb{P})$.

Could someone provide a reasonable example of the probability used in quantum mechanics not being reducible to Kolmogorov's axioms?

---

Disclaimer: I am asking the question here and not in a physics space, because I would like to see a rigorous demonstration of the fact, with the formality and rigor proper to mathematics, so as not to leave room for the ambiguity typical of discussions in physics.

Answer 1

A simple counterexample is given in a 9-page talk note [1]. It shows that for a certain $8$-dimensional Hilbert space, there is no embedding to any classical probability space.

More concretely, assuming the converse, i.e. there is a measurable space $M = (\Omega, \mathcal{F})$, an embedding (a map $(A \mapsto f_A)$ that sends each quantum observable to a classical random variable, and a map ($\psi \mapsto \rho_\psi$) that sends each quantum (pure) state to a probability measure on $M$), such that

1. The induced probability distribution is compatible (condition KS1 in [1]).
2. The associated random variables are assigned in a compatible way that respects any measurable re-scaling (condition KS2 in [1]).

Note: The proof only use a weaker version KS2' of KS2, which says if $A$ and $B$ are commuting quantum observables (i.e. $AB=BA$), then $f_{AB} = f_A f_B$ (i.e. $\forall \omega \in \Omega, f_{AB}(\omega) = f_A(\omega) f_B(\omega)$).

Then cleverly construct a state $\Psi$ [1, (18)] and an observable $O = A_1 A_2 A_3$ [1, (15)(17)]. Its expected value is $\langle O\rangle_\Psi = -1$ [1, (24)]. However, you can show that if such an embedding exists, mapping $O$ to $a_1 a_2 a_3$, then $a_1 a_2 a_3$ takes value $1$ almost everywhere [1, (22)], a contradiction.

Discussions

1. Locality assumption of any sort is not required here. Locality is about the Bell's theorem, not Kochen-Specker theorem.

2. Note that [1] claims to prove this for all Hilbert spaces with dimension at least $3$, but as far as I can tell, the proof only show for a $8$ dimensional one.

3. More details from the paper: If you dig hard into it, the contradiction still comes from noncommutativity. In the clever setup, three observables $Q_1=A_1B_2B_3, Q_2=B_1A_2B_3, Q_3=B_1B_2A_3$ are given. While they are commuting, their sub-components (the $A$'s and $B$'s) are not. And while $$1 = \langle Q_1Q_2Q_3 \rangle_\Psi = \langle (A_1 B_2B_3)(B_1A_2B_3)(B_1B_2A_3) \rangle_\Psi$$$$ = \langle A_1 (B_2A_2B_2) A_3 \rangle_\Psi = \langle A_1 (-A_2) A_3 \rangle_\Psi = \langle (-1) A_1 A_2 A_3 \rangle_\Psi,$$ with the nonexistent classical map $Q_1Q_2Q_3 = A_1 (B_2A_2B_2) A_3$ is mapped to $$q_1q_2q_3 = (b_1^2)(b_2^2)(b_3^2) a_1 a_2 a_3 = (+1) a_1a_2a_3.$$ A contradiction thus arises from the sign difference.

4. Even though KS2' is a much weaker assumption than KS2, and even though KS2' seems natural from the mathematical point of view, until this day I kept questioning myself whether KS2' is a reasonable assumption. The paper [1] provided an artificial solution to the hidden variable problem posed as above, provided that KS2 is dropped. However, the artificial solution has issue that each $f_A$ is independent, which should not be the case. So to introduce dependency, [1] proposed KS2. However, in my opinion KS2 (or even its weaker version KS2') is probably too strong for that. Instead, we should try to come up with KS2B that respects the Born rule, and solve the problem under KS1 and KS2B. A proposal of KS2B may be: Let $A$ be a quantum observable, $\lambda$ an eigenvalue of $A$ with a normalized eigenvector $\psi$, $\phi$ a test function, then

$$\int_\Omega \phi d\,\rho_{\psi} = \frac{\int_{f_A^{-1}( \lambda)} \phi d\rho_{\psi}}{\int_{f_A^{-1}( \lambda)} d\rho_{\psi}}.$$

We may also need to require another natural condition $$\rho_\psi(f^{-1}_A(\lambda)) = \rho_{Q\psi}(f^{-1}_{QAQ^*}(\lambda)).$$

However, note that KS2 has had some history, especially in the math community of topos [2].

Reference

• [1] A Simple Proof of the Kochen-Specker Theorem on the Problem of Hidden Variables
• [2] A topos perspective on the Kochen-Specker theorem: I. Quantum States as Generalized Valuations

Answer 2

While I am not positive about this, I believe that quantum physics can in fact be formulated in ways that are compatible with Kolmogorov's axioms - it's just that any attempt to do so provably must involve maps that physicists find "distasteful", based on physics principles like the principle of locality or something called "non-contextuality" (which is a bit difficult to explain without getting into the weeds of how QM works).

For example, I believe that Bohmian mechanics is a formulation of QM that satisfies all of the Kolmogorov axioms, albeit at the cost of violating locality.

In Mermin's excellent paper at https://arxiv.org/abs/1802.10119, he says

John Bell did not believe that either of his no-hidden-variables theorems excluded the possibility of a deeper level of description than quantum mechanics... He viewed them all as identifying conditions that such a description would have to meet. ...

To those for whom non-locality is anathema, Bell’s Theorem finally spells the death of the hidden-variables program. But not for Bell. None of the no-hidden-variables theorems persuaded him that hidden variables [satisfying Kolmogorov's axioms] were impossible. What Bell’s Theorem did suggest to Bell was the need to reexamine our understanding of Lorentz invariance.

So unfortunately, I don't think that it's possible to fully understand what makes quantum mechanics so weird purely from a mathematical perspective. You need to be familiar with physics concepts like locality in order to appreciate how weird it is that QM breaks them (or at least satisfies them in subtle and nonobvious ways.)