Here's a formal (inductive) proof for the formula of $A^k$: we wish to show that
$$
[A^k]_{i,j} = \begin{cases}
b^k & j-i = k\\
0 & \text{otherwise}
\end{cases}
$$
where $[A]_{i,j}$ denotes the $i,j$ entry of $A$. The base case (either $k=0$ or $k=1$) holds trivially. For the inductive step: we note that if $i,j$ are between $1$ and $n$
$$
[A^{k+1}]_{i,j} = [A A^{k}]_{i,j} = \sum_{p=1}^{n} A_{ip}[A^k]_{pj}
$$
We note that $A_{ip}[A^k]_{pj}$ is only non-zero if $A_{ip} \neq 0$ and $[A^k]_{pj} \neq 0$. By our definition of $A$, $A_{ip}$ will only be non-zero if $p = i+1$. On the other hand: by our inductive hypothesis, $[A^k]_{pj}$ will only be non-zero if $p = j-k$. These can only be simultaneously true if $i+1 = j-k$, which is to say that $j-i = k+1$. Thus, we conclude that $[A^{k+1}]_{i,j} = 0$ whenever $j-i \neq k+1$.
Whenever $j - i = k+1$, we compute
$$
[A^{k+1}]_{i,j} = \sum_{p=1}^{n} A_{ip}[A^k]_{pj} =
A_{i,(i+1)}[A^k]_{(j-k),j} = b \cdot b^k = b^{k+1}
$$
The conclusion follows.