Big O, little o, Big Op, little op

• Big O and little o
  • A sequence $x_n$ of non-random vectors is said to be $O$(1) if it is bounded and $o$(1) if it converges to zero.
  • If an is a sequence of non-random positive scalars, then  $$ x_n = O(a_n) \text{  means  } \frac{x_n}{a_n} = O(1) $$ (that is, $\frac{x_n}{a_n}$ is bounded), and $$ x_n = o(a_n) \text{  means  } \frac{x_n}{a_n} = o(1) $$ (that is, $\frac{x_n}{a_n}$ converges to zero).
• Big Op and little op
  • A sequence $X_n$ of random vectors is said to be $O_p(1)$ if it is bounded in probability and $o_p(1)$ if it converges in probability to zero.
  • Suppose $X_n$ and $a_n$ are random sequences taking values in any normed vector space, then $$ X_n = O_p(a_n)  \text{  means  } \frac{X_n}{\| a_n \|} \text{ is bounded in probability.} $$ That is, for any $\varepsilon > 0$, there exists a finite $M > 0$ and a finite $N > 0$ such that $$ P \left( \left| \frac{X_n}{a_n} \right | > M \right) < \varepsilon, ~~~ \forall \; n > N. $$ And $$ X_n = o_p(a_n) \text{  means  } \frac{X_n}{\| a_n \|} \text{ converges in probability to zero.} $$ That is, for any $\varepsilon > 0$, $$ \lim _{n \to \infty} P \left( \left| \frac{X_n}{a_n} \right| \geq \varepsilon \right) = 0. $$