Yes it is true. But not for the commutative hopf algebras in general. You need some more assumptions: first we need the field to be algebraically closed and of characteristic zero. You also need cocommutativity and finite dimensionality of the hopf algebra to have a full "correspondence".
To be more precise:
it can be shown that there is an equivalence of Categories, between the Category of commutative, cocommutative, finite dimensional Hopf algebras $\mathcal{H}$ (over an algebraically closed field, of characteristic zero) and the category $\mathcal{Ab}_{fin}$ of the finite, abelian groups. It is possible to construct fully faithful and essentially faithful functors between $\mathcal{H}$ and $\mathcal{Ab}_{fin}$.
Start from an object of $\mathcal{H}$ i.e. a commutative, cocommutative, finite dimensional Hopf algebra $\mathcal{H}$, over an algebraically closed field, of characteristic zero. The set $G(H)$ of its grouplike elements forms a finite abelian group i.e. an element of $\mathcal{Ab}_{fin}$. It is relatively easy to see that a hopf algebra morphism induces an abelian group homomorphism.
So you get a functor
$$
\mathcal{G} : \mathcal{H} \Rrightarrow \mathcal{Ab}_{fin}
$$
On the other hand, start from a finite abelian group $G$ and take its group hopf algebra $kG$. It is clearly commutative, cocommutative and finite dimensional, i.e. an object of $\mathcal{H}$. On the other hand, an abelian group homomorphism induces -by linear extension, due to the universal property of the group algebra- a morphism of hopf algebras between the corresponding group hopf algebras.
So you get a functor
$$
\mathcal{F} : \mathcal{Ab}_{fin} \Rrightarrow \mathcal{H}
$$
Now, it can be shown that:
$$
\begin{array}{cccc}
\mathcal{G} \mathcal{F} = Id_{\mathcal{A}b_{fin}} & & & \mathcal{F} \mathcal{G} \cong
Id_{\mathcal{H}}\\
\end{array}
$$
Consequently, the functors $\mathcal{G}$, $\mathcal{F}$ constitute an equivalence of the categories $\mathcal{H}$, $\mathcal{Ab}_{fin}$.
In my understanding, it is actually this equivalence of categories, which inspired the introduction of the term quantum groups (implying that the hopf algebra theory may be considered as a kind of "quantum" generalization of the group theory).
Maybe you can also find some interest in this -somewhat related- post: https://math.stackexchange.com/a/2756755/195021