"I was thinking that to find inverses separately of the each term in Z5 and then form the matrix?" The way I interpret that sentence you are saying that $(a_{ij})^{-1}=(a_{ij}^{-1})$ which is not true.
To find the inverse of a matrix you can't just take the inverse of each element. Now to answer the question, it depends on how/what can you use to compute the inverse. If you are doing it by hand, then just make a quick addition and multiplication table of $\mathbb{Z_5}$ and just find the inverse exactly as how you would with real numbers except that all addition, subtraction, multiplication, and division should be done in $\mathbb{Z_5}$.
If you are using Matlab for example, then (in general) just take the "normal" inverse (over the reals), multiply every entry in the matrix by the (real) determinant which would turn all entries into integers if you had any decimals/rationals. Then multiply all of the entries in the matrix by the multiplicative inverse of the determinant in $\mathbb{Z_5}$ to "undo" the real multiplication you did. You still have all integers. Then you reduce mod 5 and done.
In your case, the determinant in reals is 6 meaning the determinant over $\mathbb{Z_5}$ is 1. General rule, if the determinant is invertible in whatever number system you working in, the matrix will be invertible too. So in this case your matrix is invertible AND you don't have to do the last two steps where you multiply by the determinant and then its inverse.