I have given the following problem from the draft book of Francis Bach: "For $(w,b/R)$ uniform on the sphere and for the ReLU activation, compute the associated kernel as a function of the cosine between the vectors $(x,R)$ and $(x^\prime,R)$.
Now what I first tried is to omit the bias in a first step and to directly compute the integral $$ \frac{1}{V_{n-1}(R)}\int\limits_{\mathbb{S}^{n-1}} \max(0,w^Tx ) \max(0,w^Tx^\prime) \sigma_{n-1}(dw) $$ where $V_{n-1}(R)$ denotes the surface are of the $n-1$ dimensional sphere with radius $R$. However, I got stuck quite immediately. Therefore I did some online research and found the paper of Cho and Saul where they compute the arccos kernel $$2 \int\limits_{\mathbb{R}^n} \frac{1}{(2\pi)^{n/2}} \exp(-\frac{\Vert w \Vert^2}{2}) \Theta(w^T x) \Theta(w^T x^\prime)(w^Tx)^d (w^Tx^\prime)^d d w$$
where $\Theta$ denotes the Heavyside function $\Theta(z)=\frac{1}{2}(1+sign(z))$. Then I noticed that for $d=1$ we have that $\Theta(w^T x) \Theta(w^T x^\prime)(w^Tx)^d (w^Tx^\prime)^d =max(0,w^Tx ) \max(0,w^Tx^\prime)$. Furthermore I noticed how to connect the uniform distribution on the sphere with the standard normal distribution according to this question also here on StackExchange.
Therefore I see that my problem is equivalent to computing the arccos-1-kernel. However, also here I quickly get stuck if I want to explicitly calculate the arccos-1-kernel. These are my calculations so far.
\begin{align*} \frac{1}{V_{n-1}(R)}\int\limits_{\mathbb{S}^{n-1}} \max(0,w^Tx ) \max(0,w^Tx^\prime) \sigma_{n-1}(dw) &= \frac{1}{(2\pi)^{n/2}} \int\limits_{\mathbb{R}^n} \max\biggl(0,\frac{w^Tx}{\Vert w \Vert}\biggr) \max\biggl(0,\frac{w^Tx^\prime}{\Vert w \Vert}\biggr) \exp(-\frac{\Vert w \Vert^2}{2}) dw\\ &= \frac{1}{(2\pi)^{n/2}}\int\limits_{w:\text{ } w^Tx>0, \text{ } w^Tx^\prime>0} \frac{1}{\Vert w \Vert^2} w^Tx w^Tx^\prime \exp(-\frac{\Vert w \Vert^2}{2}) dw\\ &= \frac{1}{(2\pi)^{n/2}}\int\limits_{w:\text{ } w^Tx>0, \text{ } w^Tx^\prime>0} \frac{1}{\Vert w \Vert^2} w^TKw \exp(-\frac{\Vert w \Vert^2}{2}) dw \end{align*} where $K_{i,j}=x_ix_j^\prime$.
Now I am quite stuck on how to further calculate the integral
What I want to state is that I am not looking for a solution of how to calculate the arccos kernel for any $d$ but only for $d=1$. Also I have found some similar results online where one could use spherical harmonics which is also a kind of an overkill to my problem.
Thank you in advance and I'd appreciate any help!
