Problem with Symmetric Positive Definite Matrix Gradient w.r.t. to its Lower Triangular Cholesky Factor

Question

I apologize if my question is duplicate, but I really need to know this. I tried to verify equation $\frac{\partial vech(X)}{\partial vech(C)}$ suggested by Lynn in this post. As a simple case, I consider a 2x2 symmetric p.s.d matrix $X=CC^T$. To this end, I computed every element of $\frac{\partial vech(X)}{\partial vech(C)}$ one by one as follows (equation 1). Assuming matrix $C$ to be: \begin{equation} C=\begin{bmatrix} c_{11} & 0 \\ c_{21} & c_{22} \end{bmatrix} \end{equation} then matrix X will be: \begin{equation} X=\begin{bmatrix} x_{11} & x_{12} \\ x_{21} & x_{22} \end{bmatrix} = \begin{bmatrix} c_{11}^2 & c_{11}c_{12} \\ c_{11}c_{21} & c_{21}^2+c_{22}^2 \end{bmatrix} \end{equation} $=>$

\begin{equation} \frac{\partial vech(X)}{\partial vech(C)}= \begin{bmatrix} \frac{\partial c_{11}^2}{\partial c_{11}} & \frac{\partial c_{11}^2}{\partial c_{21}} & \frac{\partial c_{11}^2}{\partial c_{22}} \\ \frac{\partial c_{11}c_{21}}{\partial c_{11}} & \frac{\partial c_{11}c_{21}}{\partial c_{21}} & \frac{\partial c_{11}c_{21}}{\partial c_{22}} \\ \frac{\partial (c_{21}^2+c_{22}^2)}{\partial c_{11}} & \frac{\partial (c_{21}^2+c_{22}^2)}{\partial c_{21}} & \frac{\partial (c_{21}^2+c_{22}^2)}{\partial c_{22}} \\ \end{bmatrix} =\begin{bmatrix} 2c_{11} & 0 & 0 \\ c_{21} & c_{11} & 0 \\ 0 & 2c_{21} & 2c_{22} \end{bmatrix} \quad\quad\quad\quad (1) \end{equation}

Now using equation $\frac{\partial vech(X)}{\partial vech(C)}=L[(C \otimes I)+(I \otimes C)K]D$ from this post, the following result is obtained:

\begin{equation} \frac{\partial vech(X)}{\partial vech(C)}= \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix} \left\{ \begin{bmatrix} c_{11} & 0 & 0 & 0 \\ 0 & c_{11} & 0 & 0 \\ c_{21} & 0 & c_{22} & 0 \\ 0 & c_{21} & 0 & c_{22} \end{bmatrix} + \\ \begin{bmatrix} c_{11} & 0 & 0 & 0 \\ c_{21} & c_{22} & 0 & 0 \\ 0 & 0 & c_{11} & 0 \\ 0 & 0 & c_{21} & c_{22} \end{bmatrix} \begin{bmatrix} 1 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 \\ 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 \end{bmatrix} \right\} \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ \end{bmatrix} => \end{equation}

\begin{equation} \frac{\partial vech(X)}{\partial vech(C)}= \begin{bmatrix} 2c_{11} & 0 & 0 \\ c_{21} & c_{11}+c_{22} & 0 \\ 0 & 2c_{21} & 2c_{22} \end{bmatrix} \quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad\quad (2) \end{equation} As can be seen, value at cell (2,2) of the Jacobian matrix in equation 2 is different from the value at the same cell in equation 1. Please correct me if I am making a mistake here. But if I'm right, then does it mean that $\frac{\partial vech(X)}{\partial vech(C)}$ equation suggested by Lynn is wrong? If that's the case, can anyone give the correct answer, please? thank you.

Answer 1

I've found the solution myself. The problem with the approach suggested by Lynn is that we can't use duplication matrix to convert $vec(dC)$ to $D_k vech(dC)$ because matrix $dC$ is lower triangular and not symmetric. Recall that, for any $n \times n$ matrix $A$, $vec(A)=D_n vech(A)$ only when $A$ is symmetric [see here].

The correct answer for $\frac{\partial vech(X)}{\partial vech(C)}$ is available in section 10.5.4 of book Handbook of Matrices: \begin{equation} \frac{\partial vech(CC^T)}{\partial vech(C)^T} = 2D_k^+(C \otimes I_k)L^T_k \quad (1) \end{equation}

In the equation above $D_k^+ = (D_k^T D_k)^{-1} D_k^T$ is Moore-Penrose pseudoinverse of duplication matrix [see here].

In the mentioned book, it is said that proof for equation (1) is available in book Linear Structures. Unfortunately, I don't have access to that book and I have failed to write the proof for equation (1) myself. So I appreciate if anyone can come up with the proof for equation (1).