Strong consistency of kernel density estimator

Question

I am studying the book Nonparametric and Semiparametric Models written by Wolfgang Hardle and have difficulty with the following exercise:

$\textbf{Exercise 3.13}$ Show that $\hat{f_h}^{(n)}(x) \xrightarrow {a.s.}f(x)$. Assume that $f$ possesses a second derivative and $\Vert > K \Vert_2 <\infty$.

where $\hat{f_h}^{(n)}(x)=\dfrac{1}{n}\sum\limits_{i=1}^n K_h(x-X_i)$ is the density kernel estimator, $\Vert K \Vert_2=(\int_{\mathbb{R}}K^2(s)ds)^{\frac{1}{2}}$ and we set $h=O(n^{-\frac{1}{5}})$ which is the optimal bandwidth parameter in this exercise.

Here is my attempt: I want to use Borel-Cantelli lemma to prove the almost sure convergence. By Chebyshev inequality we have $$ \mathbb{P}(\vert \hat{f_h}^{(n)}(x)-f(x)\vert>\epsilon)\le \dfrac{\mathbb{E}(\hat{f_h}^{(n)}(x)-f(x))^2}{\epsilon ^2}=\dfrac{MSE(\hat{f_h}^{(n)}(x))}{\epsilon^2}, $$ and it is known that $$ MSE(\hat{f_h}^{(n)}(x))=\dfrac{f(x)}{nh}\Vert K \Vert_2^2+\dfrac{1}{4}(f''(x))^2h^4(\mu_2(K))^2+o(h^4)+o(\dfrac{1}{nh}) $$ where $\mu_2(K)=\int_{\mathbb{R}}s^2K(s)ds$. By replacing $h$ with $h_{opt}=O(n^{-\frac{1}{5}})$ we derive that $$MSE(\hat{f_h}^{(n)}(x))=O(n^{-\frac{4}{5}})$$ However, $\sum\limits_{n=1}^{\infty} n^{-\frac{4}{5}}=\infty$ and I was stuck here. Can anyone help me with this?

Answer 1

First of all, you have not explained what the function $K_h(x)$ is. From a look at the book you referenced this is defined as $$K_h(x) = \frac{1}{h} K\left( \frac{x}{h} \right).$$ Therefore, we want to show that $$ \frac{1}{nh} \sum_{i=1}^n K \left( \frac{x-X_i}{h} \right) \xrightarrow{a.s.} f(x), $$ where presumably $X_1, \ldots, X_n$ are i.i.d. with density $f$. We only need that $h \to 0$, $\int_{\mathbb{R}}|K(x)| dx<\infty$ and $f$ is bounded. Other assumptions can also work, however. First, by the strong law of large numbers, for each $h>0$, we obtain $$\frac{1}{nh} \sum_{i=1}^n K \left( \frac{x-X_i}{h} \right) \xrightarrow{a.s.} \frac{1}{h} \mathbb{E} K \left( \frac{x-X_1}{h} \right). $$ But, by changing variables, we have \begin{align} \frac{1}{h} \mathbb{E} K \left( \frac{x-X_1}{h} \right) & = \int_{\mathbb{R}} \frac {1}{h} K \left( \frac{x-y}{h} \right) f(y) dy \\ & = \int_{\mathbb{R}} K(z) f(x-zh) dz. \end{align} Now, as $h \to 0$, we find that $K(z) f(x-zh) \to K(z) f(x)$ and, since, as noted in your reference, $\int_{\mathbb{R}}K(x) dx= 1$, \begin{align} \int_{\mathbb{R}} K(z) f(x) dz = f(x) \int_{\mathbb{R}} K(z) dz = f(x). \end{align} It only remains to justify the interchange of limit and integration but this can be easily done with dominated convergence, as we are assuming that $f$ is bounded.