The usual formula gives
$$
\Delta^{-1}(x) = \frac{|x|^{2-d}}{\gamma_{d}(2-d)}
$$
as the solution to $\Delta\left[\Delta^{-1}(x)\right]=\delta(x)$ in $d$ dimensions, where
$$
\gamma_{d} = \frac{2\pi^{\frac{d}{2}}}{\Gamma\left(\tfrac{d}{2}\right)}
$$
is the $d$-dimensional solid angle. Consider now $d=2-\epsilon$ for small $\epsilon$. Expanding the above expression we find
$$
\Delta^{-1}(x) = \frac{1}{\epsilon} + \frac{1}{2\pi}\,\log|x| + \cdots
$$
where the dots denote either terms that vanish as $\epsilon\to0$ or are independent of $|x|$. Dropping the $x$-independent terms and sending $\epsilon\to0$ we find
$$
\Delta^{-1}(x)=\frac{1}{2\pi}\,\log|x|
$$
for $d=2$.
Indeed, consider
$$
\frac{1}{2\pi}\int_{\mathbb R^2} \log|x|\ \Delta \chi (x)\,dx = \lim_{\epsilon\to0}\frac{1}{2\pi}\int_{\mathbb R^2\setminus B_\epsilon} \log|x|\ \Delta \chi (x)\,dx
$$
for any smooth test function $\chi$, where $B_\epsilon$ denotes a disk centered at the origin with radius $\epsilon$. Integrating by parts twice we have:
$$\begin{aligned}
\lim_{\epsilon\to0}
\frac{1}{2\pi}\int_{\mathbb R^2\setminus B_\epsilon} \log|x|\ \Delta \chi (x)\,dx
&= 0 - \lim_{\epsilon\to0}\frac{1}{2\pi}\int_{\mathbb R^2\setminus B_\epsilon} \frac{1}{|x|^2} \langle x, \nabla\chi (x)\rangle\,dx\\
&= + \lim_{\epsilon\to0} \frac{1}{2\pi\epsilon}\int_{\partial B_\epsilon}\chi(x)dS(x) + 0 = \chi(0)\,.
\end{aligned}
$$
Therefore,
$$
\Delta^{-1}(x) = \frac{1}{2\pi}\,\log|x|\,.
$$