Yes. Removable singularities aren't really 'singularities'. I like to think of them more as a poor choice of domain. Your function
$$f(x)=\frac{e^{iz}-1}{z}$$
is a great example. Let's consider $f$ as a function from $\mathbb{C} \backslash \{0\}$ to $\mathbb{C}$ - This function is clearly analytic/holomorphic on $\mathbb{C} \backslash \{0\}$ and when we write $f$ as a Laurent expansion on this domain, we notice that what we get is in fact a Taylor expansion. We can extend $f$ to the point $0$ by using this Taylor expansion. Let's call this function $\bar{f}:\mathbb{C} \to \mathbb{C}$. We have that $\bar{f}$ is holomorphic on $\mathbb{C}$ because it is given by its Taylor expansion.
Now, take a curve $C$ in $\mathbb{C} \backslash \{0\}$. We have that (since $f$ and $\bar{f}$ take the same values away from $0$),
$$\int_C f(z) \ dz = \int_C \bar{f}(z) \ dz = 0.$$
The moral is, that although we avoided $0$ in the domain of $f$, it is actually possible to think of $f$ as entire function. This is what I meant about the 'poor choice of domain'. I hope this helps.