[Part of this answer was previously posted as a comment - with the intent to delete it once a good dupe target was found. Unfortunately the comment was wrongly deleted, so I have reposted it as an answer. When a good dupe target is located this answer will be deleted (alas, the approach0 server is currently down so dupe searching is not easy at the moment)].
Yes, using that older language, there are indeed an infinite number of quadratic residues (i.e. integers congruent to nonzero squares) $\!\bmod p,\,$ but there are only a finite number of residue classes. In your example, $\!\bmod 7\!:\,$ the class of $\,4\equiv 2^2$ is $\,[4]_7 = 4+7\Bbb Z = \{\ldots,-3,4,11,\ldots\},\,$ i.e. the set of all $\,x\equiv 4\pmod{\!7}.\,$ Similarly every nonzero polynomial has only a finite number of classes that are roots, but each root has infinitely many congruent "residue" roots in its class. Further, $\!\bmod n\!:\,$ a polynomial has a root iff it has a root in a complete residue system.
It may be helpful to consider an analogy with fractions, e.g. $\,x^2 = 4/9\,$ has only two rational roots $\,x=\pm 2/3,\,$ but there are infinitely many equivalent fractions $\,x = \pm(2n)/(3n),\ n\neq 0.\,$ The "numbers" here - the rationals - are classes of equivalent fractions, usually represented by their least element- just like above. Further, a polynomial has a rational root iff it has a root in the complete residue system of fractions in least terms (to which the Rational Root Test applies).
In older/elementary textbooks that do not mention equivalence classes there is usually a remark that serves to clarify that congruent integers are considered to be the same modular number [i.e. modular numbers are the congruence (equivalence) classes], e.g. Niven and Zuckerman write
"Since $a + m$ is a quadratic residue or nonresidue modulo $m$ according as $a$ is or is not, we consider as distinct residues or nonresidues only
those that are distinct modulo $m$" (my emph.).
It is essential to keep in mind that in the arithmetic (ring) of integers $\!\bmod m\,$ it is the equivalence (congruence) classes (cosets) $\, [a]_n = a + n\Bbb Z\,$ that correspond to our modular "numbers". This is often obscured in treatments that instead work with convenient "normal form" representatives of each class (complete residues systems), such as choosing the least natural in each class as a rep, i.e. the system $\,0,1,2\ldots,n\!-\!1\pmod{\!n}$.
This way of constructing new algebraic objects by working modulo congruences is fundamental in algebra. Indeed, as Andy Magid wrote in his review of Jacobson's great textbook Basic Algebra I
the notion that the cosets of a normal subgroup of a group, while they have intrinsic meaning as subsets of the original group, are best thought of as unities, as elements of a new group, the quotient group, is often the pons asinorum of the Basic Algebra course. Those who cross it successfully usually do learn to think algebraically.
These matters become much clearer when one studies abstract algebra, where the integers $\!\bmod n\,$ are just a special case of a general quotient ring construction, e.g. see this answer for this case.