This is the trichotomy principle (or law of trichotomy, or etc.) for ordinals. It can also be phrased for arbitrary well-orderings, as follows:
For any two well-orderings $A,B$, exactly one of the following situations occurs: $(i)$ there is a non-surjective order-preserving embedding of $A$ into $B$; $(ii)$ there is a non-surjective order-preserving embedding of $B$ into $A$; $(iii)$ there is an order-preserving bijection between $A$ and $B$.
Intuitively, $(i)$ means $A<B$, $(ii)$ means $B<A$, and $(iii)$ means $A=B$. (Exercise: the principle above does in fact imply trichotomy for ordinals as you've phrased it.) Note also that we can replace "embedding" with "initial segment embedding" (= embedding whose image is an initial segment).
You can find proofs of the trichotomy in any decent introductory set theory textbook - for example, it's Theorem $6.3$ in Kunen's book (phrased there in terms of arbitrary well-orderings).
Note that nowhere do we need to assume that the ordinals (or well-orderings) involved are countable; this is a generally, always-applying principle. Also, to forestall a frequent (in my experience) confusion, note that it does not need the axiom of choice (rather, replacement is the key principle).