Let us write $H=G/Z(G)$. Then $G$ is a central extension of $Z(G)$ by $H$, and there is an exact sequence $1\to Z(G)\to G\to H\to 1$.
Given a finite group $H$ and an abelian group $A$, you can ask what are all the central extensions $1\to A\to G\to H\to 1$ (central means that $A\subset Z(G)$). This is known to be classified by the cohomology group $H^2(H,A)$, and the trivial element of $H^2(H,A)$ corresponds to the direct product $G=A\times H$.
As an example, it's possible to completely treat the case $A=\mathbb{Z}$. Let $H$ be any finite group, and let $f:H\to \mathbb{Q}/\mathbb{Z}$ be a group morphism (the group of such morphisms is non-canonically isomorphic to the abelianization of $H$). Choose any function $\tilde{f}:H\to \mathbb{Q}$ such that the class of $\tilde{f}(h)$ in $\mathbb{Q}/\mathbb{Z}$ is precisely $f(h)$, for any $h\in H$. Then define $G=\mathbb{Z}\times H$ as a set, and define the multiplication
$$(n,h)\cdot (m,h')=(n+m+\alpha(h,h'),hh')$$
where $\alpha(h,h')=\tilde{f}(hh')-\tilde{f}(h)-\tilde{f}(h')$.
You can check that this gives a group structure on $G$, that $\mathbb{Z}\subset Z(G)$, $G/\mathbb{Z}\simeq H$, and therefore $G/Z(G)$ is finite. Furthermore, any central extension of $\mathbb{Z}$ by $H$ is isomorphic to this for a unique $f$ (the choice of $\tilde{f}$ does not change the isomorphism class of $G$).