It sounds like you need a deeper dive into exactly what is a limit.
Limits are one of the more slippery concepts in calculus. In fact, they are so slippery that many teachers wave them off and leave you to a more advanced course like real analysis to get deeply into them.
Unfortunately, I am not sure I have the space here to do them justice.
Limits:
$\lim_\limits{x\to a} f(x) = L$
At a rough level this says, "When $x$ is close to $a, f(x)$ is close to L." It doesn't say anything about what happens when $x = a$ only when $x$ is close to $a.$
So, in your example above.
$\lim_\limits{x\to 2} \frac {0}{x-2} = 0$
$f(2)$ is not defined.
But, for all other $x, f(x) = 0.$ When $x$ is near, but not equal to, $2, f(x)$ is near $0.$
But this is not very precise. A more precise definition:
If we define a neighborhood (open ball... very small interval) in the co-domain (range) that includes $L,$ we can find a corresponding neighborhood in the domain that includes $a$, such that every element in that neighborhood, except possibly $a$ maps into the defined neighborhood in the co-domain.
But this definition is so abstract.
Here is another, but it is not particularly approachable either.
$\forall\epsilon > 0, \exists\delta>0 : 0<|x-a|<\delta \implies |f(x) - L|<\epsilon$
You are now saying, "two greek letters, a backwards E and and upside down A? What am I supposed to make of that?"
Let's unpack it in English, "For any epsilon greater than 0, there exists a delta greater than $0$ such that when the distance between $x$ and $a$ is non-zero and less than delta then the distance between $f(x)$ and $L$ will be less than epsilon.
In the rough definition, epsilon and delta provide some criteria on how close is close.
In the neighborhoods definition, epsilon and delta define the sizes of the neighborhoods.
And since this holds for any epsilon, we can make "close" become arbitrarily small.
I don't know, if I have explained this well. I think I spent about 20 minutes looking at the $\epsilon-\delta$ definition before it finally clicked.
Continuity:
A function is continuous if $\lim_\limits{x\to a} f(x) = f(a)$ for all $x$ in the domain.
Sometimes $f(a)$ is undefined but $\lim_\limits{x\to a} f(x)$ is. This is a "removable discontinuity" and may be permissible to repair this hole.
$f(x) = \frac {0}{x-2}$ is an example of a function with a removable discontinuity.
But a discontinuity like the one we see at $0$ in $f(x) = \sin \frac 1x$ is beyond repair.
Derivatives.
$f'(x) = \lim_\limits{x\to 0}\frac {f(x+h) - f(x)}{h}$
we define the derivative in terms of a limit. If $f(x)$ is not defined at some value of $a,$ we shouldn't plug in $\lim_{x\to a} f(x)$ at that value.
For example, $f(x) =\frac {x^2 - 1}{x-1}$ has a removable discontinuity at $x = 1. f'(1)$ is undefined, even though $\lim_\limits{x\to 1} f'(x) = 1$
I hope this helps.