Somehow, it appears that
$$\int_1^{\sqrt{2}} \frac{\coth^{-1}(x)}{(1+x) \sqrt{2-x^2}} \, dx=\frac{\pi ^2}{24}.$$
Essentially, I guessed the result of the integral above based on this one: Evaluating $\int_1^{\sqrt2} \frac{\tanh^{-1}(\sqrt{2-x^2})}{1+x} dx$ (I just divided the numerical result by $\pi^2$ and then guessed the rational multiplyer of it)
However, I was unable to prove it formally. Any ideas?
\begin{aligned} &\begin{gathered} \Omega=\int_1^{\sqrt{2}} \frac{\operatorname{coth}^{-1}(x)}{(1+x) \sqrt{2-x^2}} d x \quad(\text { Let }: x=\sqrt{2} \cos \theta) \\ \Omega=\int_0^{\frac{\pi}{4}} \frac{\operatorname{coth}^{-1}(\sqrt{2} \cos \theta)}{1+\sqrt{2} \cos \theta} d \theta\left(\text { Let }: \theta=\frac{\pi}{4}-\theta\right) \\ \Omega=\int_0^{\frac{\pi}{4}} \frac{\operatorname{coth}^{-1}(\cos \theta+\sin \theta)}{1+\cos \theta+\sin \theta} d \theta \\ \operatorname{coth}^{-1}(\cos \theta+\sin \theta)=-\frac{1}{2} \ln \left(\frac{\cos \theta+\sin \theta-1}{\cos \theta+\sin \theta+1}\right)=-\frac{1}{2} \ln \tan \left(\frac{\theta}{2}\right)-\frac{1}{2} \ln \left(\frac{1-\tan \left(\frac{\theta}{2}\right)}{1+\tan \left(\frac{\theta}{2}\right)}\right) \\ \Omega=-\frac{1}{2} \int_0^{\frac{\pi}{4}} \frac{\ln \tan \left(\frac{\theta}{2}\right)}{1+\cos \theta+\sin \theta} d \theta-\frac{1}{2} \int_0^{\frac{\pi}{4}} \frac{1}{1+\cos \theta+\sin \theta} \ln \left(\frac{1-\tan \left(\frac{\theta}{2}\right)}{1+\tan \left(\frac{\theta}{2}\right)}\right) d \theta \quad\left(\text { Let }: t=\tan \left(\frac{\theta}{2}\right)\right) \\ \Omega=-\frac{1}{2} \int_0^{\sqrt{2}-1} \frac{\ln t}{1+t} d t-\frac{1}{2} \int_0^{\sqrt{2}-1} \frac{\ln \left(\frac{1-t}{1+t}\right)}{1+t} d t \end{gathered}\\ &\text { In the second integral Let : } t=\frac{1-t}{1+t}\\ &\begin{gathered} \Omega=-\frac{1}{2} \int_0^{\sqrt{2}-1} \frac{\ln t}{1+t} d t-\frac{1}{2} \int_{\sqrt{2}-1}^1 \frac{\ln t}{1+t} d t=-\frac{1}{2} \int_0^1 \frac{\ln t}{1+t} d t \\ \Omega=-\frac{1}{2}\left[\ln (1+t) \ln t+\operatorname{Li}_2(-t)\right]_0^1=-\frac{1}{2} \operatorname{Li}_2(-1)=\frac{\pi^2}{24} \\ \int_1^{\sqrt{2}} \frac{\operatorname{coth}^{-1}(x)}{(1+x) \sqrt{2-x^2}} d x=\frac{\pi^2}{24} \end{gathered} \end{aligned}
Very much parallel to the evaluation in Evaluating $\int_1^{\sqrt2} \frac{\tanh^{-1}(\sqrt{2-x^2})}{1+x} dx$ $$ x=\sqrt{2}\sin y,$$ $$dx = \sqrt{2}\cos y dy,$$ $$ \int_{1}^{\surd 2} \frac{\coth^{-1}(x)}{(1+x)\sqrt{2-x^2}}dx $$ $$ =\surd 2 \int_{\pi/4}^{\pi/2} \frac{\coth^{-1}(\sqrt{2}\sin y)}{(1+\sqrt{2}\sin y)\sqrt{2-2\sin^2 y}}\cos y dy $$ $$ =\int_{\pi/4}^{\pi/2} \frac{\coth^{-1}(\sqrt{2}\sin y)}{(1+\sqrt{2}\sin y)\sqrt{1-\sin^2 y}}\cos y dy $$ $$ =\int_{\pi/4}^{\pi/2} \frac{\coth^{-1}(\sqrt{2}\sin y)}{1+\sqrt{2}\sin y} dy $$ $$ =\int_0^{\pi/4} \frac{\coth^{-1}(\sqrt{2}\sin (y+\pi/4))}{1+\sqrt{2}\sin (y+\pi/4)} dy $$ $$ =\int_0^{\pi/4} \frac{\coth^{-1}(\sin y+\cos y)}{1+\sin y+\cos y} dy $$ $$ =\frac12 \int_0^{\pi/2} \frac{\coth^{-1}(\sin y+\cos y)}{1+\sin y+\cos y} dy $$ $$ = \frac14 \int_0^{\pi/2} \frac{\ln\frac{1+\sin y+\cos y}{\sin y+\cos y-1)}}{1+\sin y+\cos y} dy $$ $$y=2\arctan z, dy = 2/(1+z^2)dz,$$ $$1+\sin y+\cos y = 2(z+1)/(1+z^2),$$ $$\sin y+\cos y -1= -2z(z-1)/(1+z^2).$$ $$ = \frac14 \int_0^1 \frac{\ln\frac{z+1}{z(1-z)}}{2(z+1)/(1+z^2)} \frac{2}{1+z^2}dz $$ $$ = \frac14 \int_0^1 \frac{\ln\frac{z+1}{z(1-z)}}{z+1} dz = \frac14 \int_0^1 \frac{\ln\frac{z+1}{z}}{z+1} dz + \frac14 \int_0^1 \frac{\ln\frac{1}{1-z}}{z+1} dz $$ where $$ \int_0^1 \frac{\ln\frac{z+1}{z}}{z+1} dz =\frac{\pi^2}{12}+\frac12\ln^2 2; $$ $$ \int_0^1 \frac{\ln\frac{1}{1-z}}{z+1} dz =\frac{\pi^2}{12}-\frac12\ln^2 2. $$
