The formula simply follows from Fourier inversion theorem. The zero-average restriction simply follows from the fact that the range of $\Delta$ of the torus $\mathbb{T}^2=\mathbb{R}^2/\mathbb{Z}^2$ consists only of such functions.
As for the fundamental solution, the above observation makes it impossible to solve $\Delta \Psi = \delta$ on $\mathbb{T}^2$. A natural substitute is to consider the following compensated equation
$$ \Delta \Psi = \delta - 1 \tag{1}$$
where $1$ represents the constant function with the value $1$. Then this paper constructs such a solution using elliptic function. Specifically, if $\vartheta_1$ denotes the first Jacobi theta function, then with $z = x+iy$,
$$ \Psi(z) = - \frac{y^2}{8} + \frac{1}{2\pi} \log \left| \vartheta_1\left(\frac{\pi z}{2} \middle|i\right) \right|. \tag{2} $$
Perhaps a more transparent formula is
$$ \begin{aligned}
\Psi(z)
&= - \frac{|z|^2}{4} + \lim_{N\to\infty} \sum_{\substack{k \in \mathbb{Z}^2 \\ \|k\|\leq N}} \frac{1}{2\pi} \left( \log|z + k| - \log\max\{|k|,1\} \right) \\\
&= - \frac{|z|^2}{4} + \frac{1}{2\pi} \log |z| + \frac{1}{8\pi} \sum_{k \in \mathbb{Z}^2\setminus\{0\}} \log\left| 1 - \frac{z^4}{k^4} \right|.
\end{aligned} \tag{3} $$
(In the last sum, $k$'s are regarded as elements of $\mathbb{C}$ so that division makes sense.) This formula shows how this $\Psi$ is cooked up; it is a periodic sum of translates of the fundamental solution of $\Delta$ on $\mathbb{R}^2$. Despite their looks, it turns out that both $\text{(2)}$ and $\text{(3)}$ are $\mathbb{Z}^2$-periodic.