Showing $\sum_{i=1}^n y_i = O_p(\sqrt{n})$ if $E[y_i] = 0$ and $\sum_{i=1}^n y_i = O_p(n)$ if $E[y_i] \neq 0$?

Question

Let $y_1,\dots,y_n$ be a set of i.i.d random variables with mean $\mu$ and variance $\sigma^2$. Can it be shown that $$ \begin{align} \sum_{i=1}^n y_i = O_p(\sqrt{n}), & \quad \quad \text{if} \ E[y_i] = 0, \\ \sum_{i=1}^n y_i = O_p(n), & \quad \quad \text{if} \ E[y_i] \neq 0. \end{align} $$ using the definition of convergence in probability for Big $O_p$ notation?

I saw a simplified explanation for the results in page 4 of these notes but I'm wondering can it be done directly using the the definition of $O_p$, i.e. $X_n = O_p(n)$ if for all $\varepsilon > 0$, there exists $M,N>0$ such that $P(|X_n/n|>M) < \varepsilon$ for all $n > N$.

Actually I'm not sure about one thing in my answer below, which is can we say that $\mu = O_p(1)$. — sonicboom, Dec 17 '20 at 13:21
I don't think so: https://math.stackexchange.com/q/239288/290189 — GNUSupporter 8964民主女神地下教會, Dec 18 '20 at 09:28
I actually think a constant is $O_p(1)$ now. Let ${X_n}$ be a sequence of random variables where $X_n = \mu \in \mathbb{R}$. Then clearly for any $\varepsilon >0$, if we take $M>|\mu|$ it holds that $P(|X_n| > M) = P(|\mu| > M) = 0 < \varepsilon$ for all $n$. — sonicboom, Dec 18 '20 at 09:54
In section 2.3 of the linked notes, it's state that their mean is $\mu$. I suppose that means $\mu\in\Bbb{R}$. That's not included in your question body. — GNUSupporter 8964民主女神地下教會, Dec 18 '20 at 10:00
I don't get it. Say $X \sim \mathrm{Ber}(p)$ with $p \in (0,1)$, so $\mu = p$ and $\sigma^2 = p(1-p)$. I take $M = (p+1)/2 \in (p,1)$. $P(|X| > M) \ge P(X = 1) = p > 0$. — GNUSupporter 8964民主女神地下教會, Dec 18 '20 at 10:07
You have a random variable that takes on the value either $0$ or $1$. On the other hand, my sequence of constant random variables $X_n$ only ever takes on one value which is the constant $\mu$. Thus once we take $M$ to be larger than $|\mu|$ the probability of $|\mu|$ being larger than $M$ is always zero. — sonicboom, Dec 18 '20 at 10:20
oh you have "where ..." that follows $X_n$. but I don't think that's the intention of the course instructor. that's too restrictive — GNUSupporter 8964民主女神地下教會, Dec 18 '20 at 11:04
I'm not sure what you are saying? My answer below shows why the results in my question hold. I was unsure about one aspect of my answer which is whether a constant can be considered an $O_p(1)$ random variable, but now I think a constant is indeed $O_p(1)$ based on my comments above. So it seems everything is ok now. — sonicboom, Dec 18 '20 at 12:46
A constant can be considered a constant random variable with a degenerate distribution so I think we can apply the $O_p$ notation to it. — sonicboom, Dec 18 '20 at 12:55

score 1 · Answer 1 · answered Dec 17 '20 at 12:09

The CLT says $$ n^{1/2}\bigg(\frac{1}{n} \sum_{i=1}^n y_i - \mu\bigg) \stackrel{d}{\to} N(0,\sigma^2). $$ Since the term on the left hand side converges in distribution it is bounded in probability: $$ n^{1/2}\bigg(\frac{1}{n} \sum_{i=1}^n y_i - \mu\bigg) = O_p(1) $$ which menas $$ \begin{align} \frac{1}{n} \sum_{i=1}^n y_i = \mu + O_p(n^{-1/2}) & \Longleftrightarrow \sum_{i=1}^n y_i = n\mu + O_p(n^{1/2}). \end{align} $$ Now if $E[y_i] = 0$ we get the rate $O_p(n^{1/2})$. Otherwise $\mu = O_p(1)$ since it is a 'constant random variable' and thus $$ \begin{align} \sum_{i=1}^n y_i = nO_p(1) + O_p(n^{1/2}) = O_p(n). \end{align} $$

Actually I'm not sure we can say $\mu = O_p(1)$. – sonicboom Dec 17 '20 at 13:16 — sonicboom, Dec 17 '20 at 13:16

Showing $\sum_{i=1}^n y_i = O_p(\sqrt{n})$ if $E[y_i] = 0$ and $\sum_{i=1}^n y_i = O_p(n)$ if $E[y_i] \neq 0$?

1 Answers1