The homotopy groups of $H_{>k}$ are finitely generated. Write $H_{>k} = \text{Mat}_d \setminus H_{\leq k}$.
First, $H_{\leq k}$ is a real algebraic variety: it is defined by the polynomial equations that the determinant of every $(k+1)$-minor is zero. It thus has two crucial properties. First, it admits a Whitney stratification (in particular, can be decomposed into finitely many pieces, all of which are smooth manifolds); second, the unit sphere of $H_{\leq k}$ (which we shall denote as $U_k$) is a compact real algebraic variety (defined in addition by the equation $x_1^2 + \cdots + x_{d^2}^2 = 1$), and thus has a finite triangulation.
Now the transversality theory, applied to the finite union of manifolds presenting $H_{\leq k}$, says that every map from a manifold may be perturbed slightly to make it transverse, and this can be done relatively to the boundary if the boundary is already transverse. In particular, if the dimension of the manifold is less than the codimension of $H_{\leq k}$, it may be slightly perturbed to be disjoint from this subset. Calculating $\text{codim} H_{\leq k} = (d-k)^2$, and applying this to spheres and balls, we see that $$\pi_i H_{>k} = 0 \text{ for } i \leq (d-k)^2 -2.$$
The most important point here is that as soon as $k < d-1$, $H_{>k}$ is simply connected.
Now let us calculate the cohomology of $H_{>k}$ (in a sense). It is the complement of the one-point compactification of $H_{\leq k}$ inside $S^{d^2}$. Because $H_{\leq k}$ is the open cone on $U_k$, this one-point compactification may be identified with the unreduced suspension $\Sigma U_k$. Real algebraic varieties are locally contractible, and suspensions of locally contractible spaces are locally contractible, so $H_{\leq k} \cup \{\infty\}$ is as well. We may now apply the Alexander duality theorem to identify $$H^{d^2-q-1}(H_{>k};\Bbb Z) \cong H_{q+1}(U_k;\Bbb Z).$$ Because $U_k$ has a finite triangulation, its homology groups are all finitely generated, and thus the same is true of the cohomology groups of $H_{>k}$. Now, as a general fact, homology groups of a space $X$ are all finitely generated iff the cohomology groups are finitely generated. One direction is a trivial application of the universal coefficient theorem, the other is proposition 3F.12 of Hatcher's algebraic topology book. Hence $H_{>k}$ has finitely generated homology.
Now a simply connected space with finitely generated homology groups has finitely generated homotopy groups. This is a corollary of Serre's Hurewicz mod $\mathcal C$ theorem, where $\mathcal C$ is taken to be the class of finitely generated abelian groups. This proves the result for $k < d-1$.
For $k = d-1$, $H_{>k}$ is of course $GL(d)$, which is a Lie group, deformation retracts onto $O(d)$, and Lie groups are known to have finitely generated homotopy groups. (Alternatively, apply Hurewicz mod $\mathcal C$ to the universal cover $\text{Spin}(d)$ of $SO(d)$, which is still compact and thus has finitely generated homotopy groups; the leftover case $d = 2$ where this argument does not apply is even more trivial.)