Relationship between minmax theorems and strong duality

Question

I've seen it described in at least a couple places that swapping of min and max in a min-max expression is equivalent to, or at least implied by strong duality.

I'm only really familiar with strong duality in the context of Convex Optimization via Boyd Vandenberghe. But in that context the dual problem would contain terms related to the feasible set so I don't really see how it's simply the swap of min and max.

For example this answer on MathSE states that asking when we can swap min and max is equivalent to asking when strong duality holds.

Furthermore, in the supplemental material of the paper Robust Classification Under Sample Selection Bias NEURIPS 2014 the author states that

\begin{equation} \min_{\hat{P}(Y|X)\in\Delta} \max_{\check{P}(Y|X)\in\Delta\cap\Xi} \mathbb{E}_{P_{trg}(x)\check{P}(y|x)}\left[-\log{\check{P}(Y|X)}\right] \end{equation}

is equivalent to

\begin{equation} \max_{\check{P}(Y|X)\in\Delta\cap\Xi} \min_{\hat{P}(Y|X)\in\Delta} \mathbb{E}_{P_{trg}(x)\check{P}(y|x)}\left[-\log{\check{P}(Y|X)}\right] \end{equation}

when $\Xi$ is convex and a solution exists on the relative interior of the set. They state that this is because strong duality holds, which allows them to switch the order of the min and max.

This justification sounds like Slater's condition, but I don't see how the second expression is the dual of the first.

My guess is that I'm missing something obvious, could someone shed some light on how switching min and max in a min-max expression is related to the concept of strong duality?

Answer 1

Perhaps it is easiest to understand the relation of the Minimax Theorem and linear programming in the classical context of finite zero-sum two-player games. Suppose that we want to determine if player I in a two-person zero-sum game with $m\times n$ payoff matrix $A=(a_{ij})$ can guarantee an expected gain of at least $V$. % no matter how player II plays. It suffices for her to find a mixed strategy $\bf x$ which guarantees her an expected gain of at least $V$ for each possible pure strategy $j$ player II might play. These conditions are captured by the following system of inequalities: $$ x_1 a_{1j} + x_2 a_{2j} + \cdots + x_m a_{mj} \ge V \text{ for $1\le j\le n$.}$$

In matrix-vector notation, this system of inequalities becomes: $${\bf x}^T A \ge V \bf 1^T,$$ where $\bf 1$ is an all-1's vector. (Its length will be clear from context.)

Thus, to maximize her guaranteed expected gain, player I should find an ${\bf x}\in {\bf R}^m$ and a $V\in {\bf R}$ that solve \begin{eqnarray} &\text{maximize } V \notag\\ \text{subject to}\; &x^TA \ge V {\bf 1}^T,\\ &\sum_{1\le i \le m} x_i = 1,\notag\\ &x_i \ge 0 \text{ for all $1\le i \le m$}.\notag \end{eqnarray}

In this linear program, the variables are $V$ and $x_1, \ldots, x_m$.

The problem of finding the optimal strategy for player II can similarly be formulated as a linear program, and it is the dual to the program above: \begin{eqnarray} &\text{minimize } V\notag\\ \text{subject to } \; &A{\bf y} \le V {\bf 1}\\ &\sum_{1\le j \le n} y_j = 1,\notag\\ &y_j \ge 0 \text{ for all $1\le j \le n$}.\notag \end{eqnarray}

See, e.g., the discussion of the minimax theorem and the appendix on linear programming in [1]. The equivalence was first demonstrated in [2], see [3] for a modern account which includes the converse.

Addendum: Going beyond the linear case, for minimax to hold the function $f(x,y)$ should be convex (or quasi convex) in the variable being minimized and concave (or quasi-concave) in the variable being maximized. See https://projecteuclid.org/journals/kodai-mathematical-journal/volume-11/issue-1/Elementary-proof-for-Sions-minimax-theorem/10.2996/kmj/1138038812.full and especially https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.444.1418&rep=rep1&type=pdf which discusses the connection with duality

[1] https://www.yuval-peres-books.com/game-theory-alive/ https://homes.cs.washington.edu/~karlin/GameTheoryBook.pdf

[2] George B Dantzig. A proof of the equivalence of the programming problem and the game problem: In activity analysis of production and allocation, Cowles commission monograph 13. Koopmans. New York: John Wiley, 1951.

[3] Ilan Adler. The equivalence of linear programs and zero-sum games. International Journal of Game Theory, 42(1):165–177, 2013.