If you are looking for a blockwise closed-form formula in terms of $A,B$ and $C$, I am very skeptical about its usefulness. Yet that doesn't mean there isn't one: since $X$ is invertible,
$$
X^{-1} = (X^TX)^{-1}X^T
={\underbrace{\begin{bmatrix}A^TA+C^TC & A^TB\\ B^TA & B^TB\end{bmatrix}}_M}^{-1}
\begin{bmatrix}A^T & C^T\\ B^T & 0\end{bmatrix}.
$$
As $X$ is invertible, $M=X^TX$ is positive definite. Hence the diagonal sub-block $B^TB$ is invertible (and in fact, positive definite). In turn, its Schur complement in $M$ is also invertible. By applying the matrix inversion formula in terms of Schur complement, we get
$$
X^{-1}=
\begin{bmatrix}
S^{-1} &
-S^{-1} A^TB (B^TB)^{-1} \\
-(B^TB)^{-1} B^TA S^{-1} &
(B^TB)^{-1} + (B^TB)^{-1} B^TA S^{-1} A^TB (B^TB)^{-1}
\end{bmatrix}
\begin{bmatrix}A^T & C^T\\ B^T & 0\end{bmatrix},
$$
where $S=A^TA+C^TC-A^TB(B^TB)^{-1}B^TA$ is the Schur complement of $B^TB$ in $M$.
Edit. The above idea can be generalised to any invertible complex matrix
$X=\begin{bmatrix}A&B\\ C&D\end{bmatrix}$ with square diagonal sub-blocks
$A$ and
$D$ of possibly different sizes. We have
\begin{aligned}
X^{-1} = (X^\ast X)^{-1}X^\ast
&={\begin{bmatrix}A^\ast A+C^\ast C & A^\ast B+C^\ast D\\ B^\ast A+D^\ast C & B^\ast B+D^\ast D\end{bmatrix}}^{-1}
\begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix}\\
&=:{\underbrace{\begin{bmatrix}P&Q\\ Q^\ast&R\end{bmatrix}}_M}^{-1}
\begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix}
\end{aligned}
where
$P= A^\ast A+C^\ast C,\ Q=A^\ast B+C^\ast D$ and
$R=B^\ast B+D^\ast D$.
Since $X$ is invertible, $M=X^\ast X$ is positive definite. Thus the principal submatrices $R$ and its Schur complement $S=P-QR^{-1}Q^\ast$ in $M$ are also positive definite and we may apply the usual block matrix inverse formula to $M$ to obtain
$$
X^{-1}=
\begin{bmatrix}
S^{-1} & -S^{-1} QR^{-1} \\
-R^{-1}Q^\ast S^{-1} & R^{-1} + R^{-1}Q^\ast S^{-1}QR^{-1}
\end{bmatrix}
\begin{bmatrix}A^\ast & C^\ast\\ B^\ast & D^\ast\end{bmatrix}.
$$