I am trying to verify the the dual of the dual is the primal? using a simple convex QP: \begin{align} \min_x& \frac{1}{2} x^\top H x + h^\top x\\ \text{s.t.} &~Ax\leq b \\ &~ A_e x = b_e \end{align} where \begin{equation} H=\begin{bmatrix}1 & -1 \\ -1 & 2\end{bmatrix},\quad h=\begin{bmatrix}-2\\-6\end{bmatrix},\quad A=-\begin{bmatrix}1 & 1 \\ -1 & 2 \\ 2 & 1\end{bmatrix}, \quad b=\begin{bmatrix}2\\2\\3\end{bmatrix},\quad A_e=\begin{bmatrix}0.7 & 0.5\end{bmatrix},\quad b_e = -1. \end{equation} Solving the problem, we have \begin{equation} x^* = \begin{bmatrix}-1.6667 & 0.3333\end{bmatrix}^\top, \quad f_p(x^*) = 3.889. \end{equation} The dual problem of convex QP is \begin{align} \min_{\lambda,\eta}& \frac{1}{2} \begin{bmatrix} \lambda^\top & \eta^\top \end{bmatrix} \tilde{H} \begin{bmatrix}\lambda \\ \eta \end{bmatrix} + \tilde{h}^\top \begin{bmatrix}\lambda \\ \eta \end{bmatrix} + \frac{1}{2}h^\top H^{-1} h\\ \text{s.t.} &~ \lambda\geq 0 \end{align} where $\lambda$ and $\eta$ are Lagrange multipliers of inequality and equality constraints of the primal QP, respectively, \begin{equation} \tilde{H} = \begin{bmatrix} A \\ A_e \end{bmatrix} H^{-1} \begin{bmatrix} A^\top & A_e^\top \end{bmatrix}, \quad \tilde{h}^\top = h^\top H^{-1} \begin{bmatrix} A^\top & A_e^\top \end{bmatrix} + \begin{bmatrix} b^\top & b_e^\top \end{bmatrix}. \end{equation} Solving the dual problem, as expected, we have \begin{equation} \lambda^*=\begin{bmatrix}0 & 0 & 1.88889\end{bmatrix}^\top,~ \eta^*= 11.1111,~f_d(\lambda^*, \eta^*) = - f_p(x^*) = 3.8889, \end{equation} and \begin{equation} x^* = -H^{-1}\left(\begin{bmatrix} A^\top & A_e^\top \end{bmatrix}\begin{bmatrix}\lambda^* \\ \eta^*\end{bmatrix} + h\right) \end{equation} Now when I try to construct the dual problem of dual problem above: \begin{align} \min_{\gamma}& \frac{1}{2} \gamma^\top \hat{H} \gamma + \hat{h}^\top \gamma + \frac{1}{2}\tilde{h}^\top \tilde{H}^{-1} \tilde{h}\\ \text{s.t.} &~ \gamma\geq 0 \end{align} where $\gamma$ is the Lagrange multiplier of inequality constraints of the dual problem, \begin{equation} \hat{H} = \tilde{A} \tilde{H}^{-1} \tilde{A}^\top, \quad \hat{h}^\top = \tilde{h}^\top \tilde{H}^{-1} \tilde{A}^\top + \tilde{b}^\top, \quad \tilde{A} = \begin{bmatrix} -I & \boldsymbol{0}\end{bmatrix}, \quad \tilde{b} = \boldsymbol{0}. \end{equation} I realized that matrix $\tilde{H}$ is only postive semi-definite, which is not fully rank/inverible, this due to the size of matrix $A$.
How can I construct the dual of dual problem of my convex QP problem? Does the dual of dual problem even exist?
P.S. I tried to make $\tilde{H}$ invertible by letting $A=-\begin{bmatrix}2 & 1\end{bmatrix}$ and $b=3$, then we have $f^*_{dd} = -f_d^* = f_p^* = 3.3889$.
