How to do derivative of expectation (characteristic function)?

Question

I type a lot of stuff here, which may looks like daunting. But most of them are just background of my question and are not relevant on my question. Basically, my question is how to get derivative of expectation (here it is characteristic function)? You can skip the background and directly solve my question at last.

---

Suppose $X$ is integrable. Then the following statements are equivalent:

1. for all $u,v \in \mathbb{R}^p, u \perp v$, we have $E(u^TX|v^T X)=0$;

2. $X$ has a spherical distribution;

Proof:

$1 \Rightarrow 2$. Step 1. It suffices to show that, for any $t_1, t_2 \in \mathbb{R}^p$ such that $||t_1||=||t_2||$, we have $\phi_X(t_1)=\phi_X(t_2)$. (Background: it uses a lemma that is a random vector $X$ has a spherical distribution if and only if its characteristic function is a function of $t^Tt$; that is, $ \phi_X(t)=g_0 (t^Tt)$ for some function $g_0$. Here $\phi_X(t)=E(e^{it^T X})$, where $i=\sqrt{-1}, t \in \mathbb{R}^p$)

Step 2. Let $\mathcal{S} = \{t: ||t|| =||t_1|| \}$. By another lemma (see below), there is a differential function $c: (0, 2\pi) \to \mathcal{S}$ that passes through $t_1$ and $t_2$. It then suffices to show that $\phi_X$ is constant on $\mathcal{C}$.

Lemma: Let $\mathcal{S}=\{ x:||x||=r\}$, where $r>0$, and let $t_1,t_2\in \mathcal{S}$. Then there is a differentiable mapping $c:(0, 2\pi) \mapsto \mathcal{S}$ that passes through $t_1$ and $t_2$; that is, there exist $\alpha_1,\alpha_2 \in (0, 2\pi)$ such that $c(\alpha_1)=t_1$ and $c(\alpha_2)=t_2$. Proof: Let $\mathcal{H}$ be a hyperplane that passes through $t_1,t_2,0$ in $\mathbb R^p$. Then $\mathcal{C}=\mathcal{H} \cap \mathcal{S}$ is a circle centered at the origin with radius $r$. Pick an arbitrary point $t_3$ in $\mathcal{C}$ that is not $t_1$ and $t_2$, and let $t_4$ be the point on $\mathcal{C}$ obtained by rotating $t_3$ counterclockwise 90 degree along $\mathcal{C}$. Then any point $t$ in $\mathcal{C}$ that is not $t_3$ can be written as $t=\cos(\alpha)t_3+\sin(\alpha)t_4$ for some $\alpha \in (0, 2\pi)$. By construction, $\{ c(\alpha): \alpha \in (0, 2\pi) \}$ is a differentiable curve that passes through $t_1$ and $t_2$.

Step 3. A sufficient condition for this is

$$\partial \phi_X[c(\alpha)]/\partial \alpha=0$$

for all $\alpha \in (0, 2\pi)$. The left hand side is

$$\partial \phi_X[c(\alpha)]/\partial \alpha=\partial E(e^{ic(\alpha)^T X})/\partial \alpha\stackrel{?}{=} E[(e^{ic(\alpha)^T X} iX^T\dot{c}(\alpha))] \stackrel{?}{=} E\{ e^{ic(\alpha)^T X} iE[X^T \dot{c}(\alpha)|c(\alpha)^T X] \}$$

Question: how to get the last two equality signs? I don't know how to get derivative with expectation.

Answer 1

Let's start with the first:$$\frac{\partial \mathbb{E}(e^{ic(\alpha)^\top X})}{\partial \alpha}\stackrel{?}{=} \mathbb{E}\left[\left(e^{ic(\alpha)^\top X} iX^T\dot{c}(\alpha)\right)\right].$$ Recall that the expectation is an integral; assuming a continuous random variable $X$, we have expectation $$\mathbb{E}[X] = \int_x\, x \cdot f(x) \,\mathrm{d}x.$$ Very roughly speaking (I'll leave you to iron out the precise details), $\mathbb{E}[X]$ is a sum, and since the derivative of a sum is a sum of derivatives, it holds that for some continuously differentable function $g$ with parameter $\alpha$ (and a few other conditions that make $g$ "nice"), $$\frac{\partial \mathbb E[g(\cdot)]}{\partial \alpha} = \mathbb{E}\left[\frac{\partial g(\cdot)}{\partial \alpha}\right].$$ (This is also known by some people as "differentiating under the integral sign"; you can find a proof as well as the other properties that $g$ satisfies here). Now take $g = e^{ic(\alpha)^\top X}$. Its derivative with respect to $\alpha$ is $\partial_\alpha g = iX^\top c'(\alpha) \cdot e^{ic(\alpha)^\top X}$ (the transpose shifts because we are taking the derivative of the exponent; see the matrix cookbook for more, or simply expand the matrices to clearly see their form).

The second equality follows because of the conditional independence of $X^\top c'(\alpha)$ and $c(\alpha)^\top X$ since $c(\alpha)^\top X$ is an element of the $\sigma$-algebra that is a subset of the $\sigma$-algebra of the probability space under which $X^\top c'(\alpha)$ is generated. (Apologies for the mouthful.) There are many proofs of this fact; one is found here.