Question:
If $A$ and $B$ are square matrices such that $AB = I$, where $I$ is the identity matrix, show that $BA = I$.
Answer: You may use the Cayley-Hamilton theorem to prove that an $n\times n$ matrix $A:=(a_{i,j})\in M_n(k)$ with $k$ any commutative unital ring, commutes with its adjunct matrix $adj(A)$ and inverse matrix $A^{-1}$ if $A^{-1}$ exists.
The Cayley-Hamilton theorem says the following: Let $I(n)$ be the $n\times n$-identity matrix and let
$$p_A(t):=det(tI(n)-A)\in k[t].$$
It follows $p_A(t)$ is a monic polynomial
$$p_A(t)=t^n-tr(A)t^{n-1} + \cdots +(-1)^n det(A)$$
with coefficients in $k$ and $(-1)^ndet(A)$ as "constant term". The Cayley-Hamilton theorem says that
$$p_A(A):=A^n-tr(A)A^{n-1}+\cdots + (-1)^ndet(A)I(n)=$$
$$A^n+a_{n-1}A^{n-1}+\cdots + a_1A +(-1)^n det(A)I(n)=(0)$$
is the zero matrix $(0)$. From this we get an explicit formula for the adjunct matrix, $adj(A)$ in terms of the coefficients $a_i$ of $p_A(t)$ and $A$: We get
$$A(A^{n-1}+a_{n-1}A^{n-1}+\cdots +a_1I(n))=Aadj^*(A)=adj^*(A)A=(-1)^{n+1}det(A)I(n),$$
since $A$ commutes with any power $A^j$ for $j\geq 1$ an integer.
Here we define
$$adj^*(A):=A^{n-1}+a_{n-1}A^{n-1}+\cdots +a_1I(n),$$
and we may define
$$adj(A):=(-1)^{n+1}adj^*(A).$$
From this it follows
$$Aadj(A)=adj(A)A=det(A)I(n).$$
The determinant $det(-)$ has the property that $det(AB)=det(A)det(B)$ and we get the following result:
Lemma: A matrix $A\in M_n(k)$ is invertible iff $det(A)\in k^*$ is a unit.
Proof: If $A$ is invertible there is a matrix $B$ with $AB=BA=I(n)$ hence
$$det(AB)=det(A)det(B)=1\in k^*$$
and it follows $det(A)$ is a unit. Conversely if $det(A)\in k^*$ define
$$A^{-1}:=\frac{1}{det(A)}adj(A)\in M_n(k).$$
It follows $AA^{-1}=I(n)=A^{-1}A,$
hence $A$ is invertible. We get an explicit formula for the adjunct matrix $adj(A)$ in terms of the coefficients $a_i$ of $p_A(t)$.
Henc if $AB=I(n)$ is the identity matrix, it follow $det(A) \in k^*$ is a unit in $k$ and hence $A^{-1}$ exist. We get (since $AB=I(n)$) an equality of matrices
$$A^{-1}=A^{-1}I(n)=A^{-1}(AB)=(A^{-1}A)B=B, $$
hence $B=A^{-1}$ and from this it follows that $BA=A^{-1}A=I(n)$ since $A$ commutes with its inverse $A^{-1}$.
Hence if you are willing to accept the Cayley-Hamilton theorem (which has an elementary proof using elementary properties) the result follows over any commutative unital ring $k$.