The fact that these series converge follows from the integral test; their values are somewhat more complicated, as indicated by the comments: they correspond to evaluating Riemann's zeta function at $k$, and this is highly nontrivial, even for positive integral $k$ (called the zeta constants). There are formulas for even integer values of $k$, though in terms of Bernoulli numbers. Note that your series is actually one less than the zeta constants, since
$$\zeta(k) = \sum_{n=1}^{\infty}\frac{1}{n^k}$$
and your sum starts with $n=2$, not $n=1$.
As mentioned, the Integral Test shows these series converge. But to do it explicitly:
Consider the sequence of partial sums
$$s_n = \sum_{r=2}^n \frac{1}{r^k}.$$
Since all summands are positive, this is an increasing sequence:
$$s_2\leq s_3\leq s_4\leq\cdots$$
so the convergence of the sequence of partial sums is equivalent to showing that the sequence is bounded.
To show the sequence is bounded, consider the function $\frac{1}{x^k}$. This function is decreasing, so if you approximate
$$\int_1^b\frac{1}{x^k}\,dx$$
using a right hand sum, you will get an underestimate for the integral.
Consider $$\int_1^{n}\frac{1}{x^k}\,dx,$$
and approximate it using a right hand sum with the interval $[1,n]$ divided into $n-1$ equal parts (so we break up the integral into $[1,2]$, $[2,3],\ldots,[n-1,n]$). The right hand sum with that partition is given by:
$$\mathrm{Right Sum} = \frac{1}{2^k} + \frac{1}{3^k} + \cdots + \frac{1}{n^k} = s_{n}.$$
Since right hand sums are underestimates, then for every positive integer $n$ we have that:
$$s_n \leq \int_1^n\frac{1}{x^k}\,dx = \frac{1-n^{1-k}}{k-1}.$$
Since the bounds get larger for larger $n$, we have:
$$s_n\leq \frac{1-n^{1-k}}{k-1} \leq \lim_{n\to\infty}\frac{1 - n^{1-k}}{k-1} = \frac{1}{k-1}.$$
So the sequence of $s_n$ is bounded above. Since it is increasing, the sequence of partial sums converges.
Since the sequence of partial sums converges, the series converges as well.
(This is the essence of the Integral Test: if you have a series
$$\sum a_n$$
such that there is a positive, continuous, decreasing function $f(x)$ such that $f(n)=a_n$ for all $n$, then the convergence of the series is equivalent to the convergence of the improper integral
$$\int_1^{\infty}f(x)\,dx$$
in the sense that if the integral converges, then the series converges; and if the integral diverges, then the series diverges. Of course, you can change the lower limit of the integral if necessary.)