In general, if $G$ is a group and $X$ is a set, then $G^X$ is the group whose underlying set is the set of all functions $f\colon X\to G$ with pointwise product; this is isomorphic to the (unrestricted) direct product of $|X|$ many copies of $G$:
$$G^X = \{f\colon X\to G\} = \prod_{x\in X}G.$$
The subgroup of $G^X$ consisting of the almost null functions is often denoted $G^{(X)}$, and is isomorphic to the restricted direct product, aka direct sum, of $|X|$ many copies of $G$:
$$G^{(X)} = \{f\colon X\to G\mid f(x)=e\text{ for almost all }x\in X\} \cong \bigoplus_{x\in X}G,$$
where "almost all" means "all except perhaps for a finite number". Then $X$ is finite, $G^{X}=G^{(X)}$. When $X$ is infinite, this is not the case.
So, $\mathbb{Z}^{\aleph_0}$ is the direct product of countably many copies of $\mathbb{Z}$. This group is not free abelian.
The free abelian group of rank $\aleph_0$ is (canonically isomorphic to)
$$\mathbb{Z}^{(\aleph_0)} = \bigoplus_{n\in\omega}\mathbb{Z},$$
the direct sum of countably infinitely many copies of $\mathbb{Z}$ (where $\omega$ is the ordinal corresponding to the natural numbers.)
It is indeed true that if $n$ is a finite number, then
$$G^{(\aleph_0)}\oplus G^n =\left(\bigoplus_{n\in\omega} G\right) \oplus \left(\bigoplus_{i=1}^n G\right)=\bigoplus_{x\in\omega+n}G \cong \bigoplus_{n\in\omega} G,$$
because $\omega+n$ (ordinal sum) can be bijected with $\omega$. If $g\colon \omega\to \omega+n$ is a bijection, then the morphism
$$h\colon G^{(\aleph_0)}\oplus G^n\to G^{(\aleph_0)}$$
given $h(f) = f\circ h$ gives the desired isomorphism (recall that the elements of $G^{(\aleph_0)}$ is a function from $\omega$ to $G$, so $f\circ h$ is a function from $\omega+n$ to $G$).
In general, if $\{F_i\}_{i\in I}$ is a family of free abelian groups, with $F_i$ of rank $\kappa_i$ (finite or infinite), then
$$\bigoplus_{i\in I}F_i$$
is free abelian of rank $\sum_{i\in I}\kappa_i$ (cardinal sum).