Let $$I=\int_0^{\sqrt2/4}\frac{1}{\sqrt{x-x^2}}\arcsin\sqrt{\frac{(x-1)(x-1+x\sqrt{9-16x})}{1-2x}}dx$$
Prove that $I=\dfrac{\pi^2}{8}$.
Wolfram suggests that it's true but does not find the antiderivative.
Here is the graph of the function being integrated.
Equivalent forms
Using $\space \arcsin \theta=\arccos\sqrt{1-\theta^2}$, we have $$I=\int_0^{\sqrt2/4}\frac1{\sqrt{x-x^2}}\arccos\sqrt{\frac{x^2+(x^2-x)\sqrt{9-16x}}{2x-1}}dx$$
Using $\space \arcsin \theta=\frac12\arccos\left(1-2\theta^2\right)$ (from @Srini's comment), we have $$I=\int_0^{\sqrt2/4}\frac1{2\sqrt{x-x^2}}\arccos\left(\frac{1+(2x^2-2x)\left(1+\sqrt{9-16x}\right)}{2x-1}\right)dx$$
Context
The integral comes from the following probability question: "The vertices of a triangle are uniformly random points on a circle. The side lengths in random order are $a,b,c$. Simulations suggest that $P(ab^3+a^3b<c^4)=\dfrac12$. Can this be proved?"
By letting $a=\sin X,\space b=\sin Y,\space c=|\sin(X+Y)|$ where $X$ and $Y$ are uniformly random in $(0,\pi)$, then letting $X=x-y$ and $Y=x+y$, the probability question is equivalent to proving $I=\dfrac{\pi^2}{8}$.
(The probability question was inspired by "A typo leads to a discovery", in particular a comment by @Nilotpal Sinha.)
Related integrals
Many probability questions about a random triangle inscribed in a circle are equivalent to some integral question.
- $P(ab<c^2)=\dfrac35$ is equivalent to $\displaystyle\int_0^{\pi/3}\arccos\left(2 \sin^{2}\left(x\right)-\cos\left(x\right)\right)dx=\frac{\pi^{2}}{5}$, which is proved here.
- $P(ab<cR)=\dfrac12$, where $R$ is the circle's cradius, is equivalent to $\displaystyle\int_{\pi/4}^{\pi/2}\arccos\left(\cos x\sqrt{1+\tan x}\right)dx=\frac{\pi^2}{16}$, which is proved here.
- $P(a^2+b^2<c(a+b))=\dfrac12$ is equivalent to $\displaystyle\int_0^{\pi/6}\arccos\left(\frac{\sin x}{\cos 2x}\right)dx=\frac{\pi^2}{16}$, which is proved here.
- $P\left(\dfrac{1}{a}+\dfrac{1}{b}<\dfrac{1}{c}\right)=\dfrac15$ is equivalent to $\displaystyle\int_{\arccos(1/4)}^{\pi/2}\arccos\left(\cos x\left(2\sin^2x+\sqrt{1+4\sin^4x}\right)\right)dx=\frac{\pi^2}{40}$, which is proved here.
- $P(\text{triangle's centroid lies inside triangle's incircle})=4-\dfrac{24}{\pi}\arctan\dfrac12$ is equivalent to $\displaystyle\int_0^1\frac{1}{\sqrt{1-x^2}}\arccos\left(\frac{3x^3-3x+4x^2\sqrt{2-x^2}}{5x^2-1}\right)dx=\frac{\pi}{2}\arctan\frac{17}{31}$, which is proved here.
These examples suggest that proving $I=\dfrac{\pi^2}{8}$ will involve a lot of substitutions, but I don't know how.
