I will work with the integral found after a substitution of $t = \sin^2 x$ has been made so that
$$I = \int_0^{\pi/2} \frac{\ln^2 (\sin x) \ln^2 (\cos x)}{\sin x \cos x} \, dx = \frac{1}{32} \int_0^1 \frac{\ln^2 t \ln^2 (1 - t)}{t (1 - t)} \, dt.$$
The first method that comes to mind is to evaluate the integral appearing to the right as the derivative of a beta function. Since
$$\text{B} (x,y) = \int_0^1 t^{x - 1} (1 - t)^{y - 1} \, dt,$$
we see that
$$\lim_{x \to 0^+} \lim_{y \to 0^+} \partial^2_x \partial^2_y \text{B}(x,y) = \int_0^1 \frac{\ln^2 t \ln^2 (1 - t)}{t (1 - t)} \, dt.$$
Thus
$$I = \frac{1}{32} \lim_{x \to 0^+} \lim_{y \to 0^+} \partial^2_x \partial^2_y \text{B}(x,y).$$
While the above calculation is doable, it is far from trivial (at least "by hand") and will accordingly not be given here.
As a second method, I will make use of the following Maclaurin series expansion for the term $\ln^2 (1 - t)$ which is given by
$$\ln^2 (1 - t) = 2 \sum_{n = 2}^\infty \frac{H_{n - 1}}{n} t^n, \qquad |t| < 1, \tag1$$
where $H_n$ is the $n$th Harmonic number. A proof of this result can be found here.
As
$$\frac{1}{t(1 - t)} = \frac{1}{t} + \frac{1}{1 - t},$$
we can write the integral $I$ as
$$I = \frac{1}{32} \int_0^1 \frac{\ln^2 t \ln^2 (1 - t)}{t} \, dt + \frac{1}{32} \frac{\ln^2 t \ln^2 (1 - t)}{1 - t} \, dt.$$
If in the second of these integrals we enforce a substitution of $t \mapsto 1 - t$ we have
$$I = \frac{1}{16} \int_0^1 \frac{\ln^2 t \ln^2 (1 - t)}{t} \, dt.$$
Replacing the term $\ln^2(1 - t)$ with its Maclaurin series expansion, after interchanging the summation with the integral sign we have
$$I = \frac{1}{8} \sum_{n = 2}^\infty \frac{H_{n - 1}}{n} \int_0^1 t^{n - 1} \ln^2 t \, dt.$$
The integral that results can be readily found using integration by parts twice. The result is
$$\int_0^1 t^{n - 1} \ln^2 t \, dt = \frac{2}{n^3}.$$
Thus
$$I = \frac{1}{4} \sum_{n = 2}^\infty \frac{H_{n - 1}}{n^4}.$$
Since
$$H_n = H_{n - 1} + \frac{1}{n},$$
we have
$$I = \frac{1}{4} \left [\sum_{n = 2}^\infty \frac{H_n}{n^4} - \sum_{n = 2}^\infty \frac{1}{n^5} \right ] = \frac{1}{4} \left [\sum_{n = 1}^\infty \frac{H_n}{n^4} - \sum_{n = 1}^\infty \frac{1}{n^5} \right ].$$
The value for the Euler sum is well-known (see here for example). It is
$$\sum_{n = 1}^\infty \frac{H_n}{n^4} = 3 \zeta (5) - \zeta (2) \zeta (3),$$
while the second sum is just the zeta function $\zeta (5)$. Thus
$$I = \frac{1}{4} \big{[}3 \zeta (5) - \zeta (2) \zeta (3) - \zeta (5) \big{]},$$
or
$$\int_0^{\pi/2} \frac{\ln^2 (\sin x) \ln^2 (\cos x)}{\sin x \cos x} \, dx = \frac{1}{4} \big{[} 2 \zeta (5) - \zeta (2) \zeta (3) \Big{]}.$$