After a lot of thinking, I think I found a way to prove what I wanted (that is, conclude that $g(0)=0$), whenever $g$ is differentiable with $g'$ bounded.
[Notation: I use $\hat f$ or $\mathcal{F}f$ interchangeably for the Fourier transform of $f$, defined here as $\xi\longmapsto\int_{\mathbb{R}}f(t)e^{-it\xi}\mathrm{d}\nu(t)$, where $\nu = \frac{1}{\sqrt{2\pi}}\lambda$ is a weighted Lebesgue measure.]
For any $f\in\mathcal{C}^\infty_c(\mathbb{R},\mathbb{C})$, its Fourier transform $\hat f$ is Schwartz, and then so is $\xi\longmapsto (1+\xi^2)\hat{f}(\xi)$. By standard properties of the Fourier transform, notice that the latter can be written as $\mathcal{F}(f-f'')$. Therefore we have
$$0 = \int_{\mathbb{R}}\hat fg\ \mathrm{d}\lambda=\int_{\mathbb{R}}\mathcal{F}(f-f'')(\xi)\frac{g(\xi)}{1+\xi^2}\mathrm{d}\lambda(\xi).$$
Since $g$ is continuous and bounded, the Fourier transform of $h:\xi\longmapsto\frac{g(\xi)}{1+\xi^2}$ is well-defined as $h\in L^1(\mathbb{R},\mathbb{C})$. Notice that in addition, $h\in L^2(\mathbb{R},\mathbb{C})$, so $\hat{h}\in L^2(\mathbb{R},\mathbb{C})$.
Using the product property of $\mathcal F$,
$$0 = \int_{\mathbb{R}} \mathcal{F}(f-f'')h\ \mathrm{d}\lambda=\int_{\mathbb{R}} (f-f'')\mathcal{F}h\ \mathrm{d}\lambda.\quad(1)$$
Since $g$ is differentiable with bounded derivative (by assumption), we have that $h$ is differentiable as well, with $h'\in L^1(\mathbb{R},\mathbb{C})$. As $|h(\xi)|\xrightarrow{|\xi|\to\infty}0$, by integration by parts, $\widehat{h'}=[\xi\mapsto i\xi\hat{h}(\xi)]$. It is easily checked that $h'\in L^2(\mathbb{R},\mathbb{C})$ as well, therefore $[\xi\mapsto (1+|\xi|)|\hat{h}(\xi)|]\in L^2$. This implies that $\hat{h}\in L^1(\mathbb{R},\mathbb{C})$ since by Cauchy-Schwartz,
$$\int_{\mathbb{R}}|\hat{h}|\mathrm{d}\lambda = \int_{\mathbb{R}}(1+|\xi|)|\hat{h}(\xi)|\frac{1}{1+|\xi|}\mathrm{d}\lambda(\xi) \leq \left\lVert(1+|\cdot|)|\hat{h}|\right\rVert_{L^2}\left\lVert\tfrac{1}{1+|\cdot|}\right\rVert_{L^2} < \infty.$$
By the Fourier inversion formula for $L^2$ (see e.g. Rudin's RCA, Theorem 9.13), denoting $h_n: t\longmapsto\int_{-n}^{n}\hat{h}(\xi)e^{it\xi}\mathrm{d}\nu(\xi)$, we have that $h_n\longrightarrow h$ in the $L^2$ sense. In particular, it means that there is a subsequence $(h_{n_k})_k$ such that $h_{n_k}\longrightarrow h$ almost everywhere. Since by assumbtion, $h$ is continuous, we have that $h_n(0)\longrightarrow h(0) = g(0)$. Since $\hat{h}$ is $L^1$, by the DCT, we have in particular that $$\int_{\mathbb{R}}\hat{h}\ \mathrm{d}\lambda=\lim_{n\to\infty}h_n(0)=g(0).$$
Now, let $(\phi_n)_n\subset\mathcal{C}^\infty_c(\mathbb{R},\mathbb{C})$ a sequence of smooth functions such that $\operatorname{supp}(\phi_n)=[-n-1{,}n+1]$ and $\phi_n\equiv 1$ on $[-n,n]$. Since $\phi_n\longrightarrow 1$ pointwise and $\hat{h}$ is $L^1$, by the DCT, we have $$\lim_{n\to\infty}\int_{\mathbb{R}}\phi_n\hat{h}\ \mathrm{d}\lambda=\int_{\mathbb{R}}\hat{h}\ \mathrm{d}\lambda = g(0).$$
On the other hand, for any $n$, we have that $\phi_n''$ has support $[-n-1{,}-n]\cup[n{,}n+1]$, and the sequence $(\phi_n)_n$ can be chosen such that there exists some $C>0$ with $|\phi_n''|<C$ for all $n$. Thus, since $\hat{h}$ is $L^1$, $$\lim_{n\to\infty}\left\lvert\int_{\mathbb{R}}\phi_n''\hat{h}\ \mathrm{d}\lambda\right\rvert \leq C\lim_{n\to\infty}\int_{[-n-1{,}-n]\cup[n{,}n+1]}\lvert\hat{h}\rvert\ \mathrm{d}\lambda = 0.$$
By (1), this allows us to conclude that
$$0 = \lim_{n\to\infty}\int_{\mathbb{R}} (\phi_n-\phi_n'')\mathcal{F}h\ \mathrm{d}\lambda = \lim_{n\to\infty}\int_{\mathbb{R}}\phi_n\hat{h}\ \mathrm{d}\lambda - \lim_{n\to\infty}\int_{\mathbb{R}}\phi_n''\hat{h}\ \mathrm{d}\lambda = g(0),$$
which is what we wanted.
Feel free to comment if you find anything incorrect.