We can find a nice formula for $M^{-1}$, where
$$
M = \begin{pmatrix}1&-\gamma&& & \\
&1&-\gamma&&\\
&&\ddots&\ddots&\\
&&&1&-\gamma\\
&&&&1\end{pmatrix}
$$
In particular, it is useful to note that $M = I - N$, where
$$
N = \begin{pmatrix}0&\gamma&& & \\
&0&\gamma&&\\
&&\ddots&\ddots&\\
&&&0&\gamma\\
&&&&0\end{pmatrix}
$$
from there, we could use the Neumann series to compute
$$
M^{-1} = (I - N)^{-1} = I + N + N^2 + N^3 + \cdots
$$
where we note that $N^k = 0$ whenever $k \geq n+1$. With that, you can conclude that
$$
M^{-1} = \pmatrix{1&\gamma&\gamma^2& \ldots & \gamma^n\\
&1&\gamma&\ddots&\vdots\\
&&\ddots&\ddots&\gamma^2\\
&&&1&\gamma\\
&&&&1}
$$
which is equivalent to the result you're looking for.
As for the name of this kind of matrix, I would say that it's an upper triangular Toeplitz matrix.
A formal (inductive proof) for the formula of $N^k$: we wish to show that
$$
[N^k]_{i,j} = \begin{cases}
\gamma^k & j-i = k\\
0 & \text{otherwise}
\end{cases}
$$
where $[A]_{i,j}$ denotes the $i,j$ entry of $A$. The base case (either $k=0$ or $k=1$) holds trivially. For the inductive step: we note that if $i,j$ are between $1$ and $n+1$
$$
[N^{k+1}]_{i,j} = [N N^{k}]_{i,j} = \sum_{p=1}^{n+1} N_{ip}[N^k]_{pj}
$$
We note that $N_{ip}[N^k]_{pj}$ is only non-zero if $N_{ip} \neq 0$ and $[N^k]_{pj} \neq 0$. By our definition of $N$, $N_{ip}$ will only be non-zero if $p = i+1$. On the other hand: by our inductive hypothesis, $[N^k]_{pj}$ will only be non-zero if $p = j-k$. These can only be simultaneously true if $i+1 = j-k$, which is to say that $j-i = k+1$. Thus, we conclude that $[N^{k+1}]_{i,j} = 0$ whenever $j-i \neq k+1$.
Whenever $j - i = k+1$, we compute
$$
[N^{k+1}]_{i,j} = \sum_{p=1}^{n+1} N_{ip}[N^k]_{pj} =
N_{i,(i+1)}[N^k]_{(j-k),j} = \gamma \cdot \gamma^k = \gamma^{k+1}
$$
The conclusion follows.
