Does the power spectral density vanish when the frequency is zero for a zero-mean process?

Question

A wide-sense stationary random time series $\zeta(t)$ is characterized by its mean value and its autocovariance function, which in the Wiener–Khinchin theorem is equivalent to the Fourier transform of the power spectral density $$\langle[\zeta-\langle\zeta\rangle][\zeta’-\langle\zeta\rangle]\rangle = \sigma^2 C(\tau)-\langle\zeta\rangle^2 = \frac 1{2\pi}\int_{-\infty}^\infty S(\omega)exp(-\omega\tau)d\omega$$ where $\sigma^2 C(\tau)$ is the autocovariance function with rms amplitude $\sigma$, $\tau = t – t’$ is the stationary variable for the time series, and $\langle\zeta\rangle$ is the mean value of $\zeta$. The inverse Fourier transform of the autocovariance function is $$S(\omega) + 2\pi \langle\zeta\rangle^2\delta(\omega) = \int_{-\infty}^\infty C(\tau)exp(\omega\tau)d\tau$$ and consists of the absolutely continuous power spectral density and the discontinuous line spectral component $2\pi \langle\zeta\rangle^2\delta(\omega)$ at $\omega = 0$ that only appears when the mean is non-zero.

Conjecture: For a zero-mean random process, i.e. $\langle\zeta\rangle = 0$, it is conjectured that the power spectral density vanishes when the frequency is zero (i.e. $S(0) = 0$). The conjecture rests on the physical interpretation of the zero frequency ($\omega = 0$) component of the spectrum. The zero frequency term corresponds to the DC term in a time series, and the DC term represents the mean value. If the random variable has a zero-mean then the DC term is zero and thus both the mean value $\langle\zeta\rangle$ and the spectrum must be zero at $\omega = 0$. In fact, since the mean value is always contained in the discontinuous line spectral component $2\pi \langle\zeta\rangle^2\delta(\omega)$, the continuous portion of the power spectral density $S(\omega)$ must always be zero at $\omega = 0$ whether the mean is zero or not.

Question: Is this conjecture true or false?

Answer 1

This result proves the conjecture is true. Consider a stationary complex random function $\zeta(t)$. Stationarity requires that the autocorrelation function depend on the difference $t –t’$ only and that the mean value must be a constant (not a member of a distribution function nor depend on $t$). Assume the power spectral density (hereafter called the spectrum) is absolutely continuous everywhere. The Fourier transform of $\zeta(t)$ itself does not exist in general, because stationary random functions are generally neither absolutely integrable nor square integrable, i.e. they do not satisfy Dirichlet’s condition. The correct representation of the random variable $\zeta(t)$ is either as a Fourier series or as a Fourier-Stieltjes integral [see, for example, Akira Ishimaru, Wave Propagation and Scattering in Random Media, (IEEE Press, 1997) Appendix A] Here, the Fourier-Stieltjes integral will be used $$\zeta(t) = \int_{-\infty}^{+\infty} e^{i\omega t}dA(\omega)$$ where $dA(\omega)$ is the random amplitude. Typically, the mean value and the autocorrelation function are found using ensemble averages, but this method provides little insight into the behavior of $dA(\omega)$. Hence, the average value and autocorrelation function for $\zeta(t)$ will be found using temporal averages. The ergodic hypothesis states that for a stationary process the temporal average and the ensemble average are equivalent.
$\mathbf Temporal Averages -$ The temporal average $\overline{\zeta(t)}$ is calculated over a finite interval $T$, and then the limit is taken as $T$ goes to infinity $$\overline {\zeta(t)} = \lim_{T\to \infty} \frac 1T \int_{-T/2}^{+T/2}\zeta(t) dt$$ Substitute the Fourier-Stieltjes integral into the equation and perform the $t$ integration to get $$\overline {\zeta(t)} = \lim_{T\to \infty} \frac 1T \int_{-\infty}^{+\infty} dA(\omega) \int_{-T/2}^{+T/2} e^{i\omega t}dt = \lim_{T\to \infty} \frac 1T \int_{-\infty}^{+\infty} dA(\omega) \frac {2sin(\omega T/2)} {\omega} $$ Use the following expressions to take the limit
$$ \lim_{T\to \infty} \frac {2\sin(\omega T/2)}{\omega} \rightarrow 2\pi\delta(\omega) \quad \mathrm{and} \quad \lim_{T\to \infty} \frac 1T = \lim_{T\to \infty} \frac {\Delta\omega} {2\pi}\rightarrow \frac {d\omega}{2\pi} $$ which gives for the temporal average $$\overline {\zeta(t)} = \int_{-\infty}^{+\infty} dA(\omega) \ \delta(\omega) \ d\omega = dA(0) $$ Finally, for a zero-mean process
$$\overline {\zeta(t)} = \langle \zeta \rangle = dA(0) = 0 $$ Thus, the mean value of $\zeta(t)$ obtained via temporal averaging is $dA(0)$ which is the random amplitude evaluated at the origin.
In the following analysis the random variable $\zeta(t)$ will be decomposed into the sum of zero-mean random variable $\zeta_0(t)$ and the mean $\langle\zeta\rangle$ so that the mean value is everywhere explicit. Now the temporal average for the autocorrelation function of $\overline{\zeta(t)\zeta^*(t’)}$ is given by $$ \overline{\zeta(t)\zeta^*(t+\tau)} = \overline{\zeta_0(t)\zeta_0^*(t+\tau)} + \langle \zeta \rangle^2 = \sigma^2C(\tau) + \langle \zeta \rangle^2 $$ $$ \lim_{T\to \infty} \frac 1T \iint_{-\infty}^{+\infty} e^{-i\omega\tau}dA(\omega)dA(\omega’) \int_{-T/2}^{+T/2} e^{i(\omega-\omega’)t}dt $$ Performing the $t$ integration yields $$ \sigma^2C(\tau) + \langle \zeta \rangle^2 = \iint_{-\infty}^{+\infty} e^{-i\omega\tau}dA(\omega)dA(\omega’) \lim_{T\to \infty} \frac 1T \frac {2sin[(\omega-\omega’) T/2]} {(\omega-\omega’)} $$ The following expressions are used in the limit process
$$ \lim_{T\to \infty} \frac {2\sin[(\omega-\omega’) T/2]}{(\omega-\omega’)} \rightarrow 2\pi\delta(\omega-\omega’) \quad \mathrm{and} \quad \lim_{T\to \infty} \frac 1T = \lim_{T\to \infty} \frac {\Delta\omega} {2\pi}\rightarrow \frac {d\omega}{2\pi} $$ Substituting these expressions into the autocorrelation gives $$ \sigma^2C(\tau) + \langle \zeta \rangle^2 = \iint_{-\infty}^{+\infty} e^{-i\omega\tau}dA(\omega)dA(\omega’) \delta(\omega-\omega’) \ d\omega $$ Finally, performing the $\omega$ integration yields $$ \sigma^2C(\tau) + \langle \zeta \rangle^2 = \int_{-\infty}^{+\infty} e^{-i\omega'\tau}|dA(\omega’)|^2 \qquad \mathrm {Equation \ (1)}$$ The quantity $dA(\omega’)$ can be related to the mean value $\langle\zeta\rangle$ and spectrum $S(\omega)$ by taking the inverse Fourier transform of the autocorrelation function
$$ \int_{-\infty}^{+\infty}\sigma^2C(\tau)e^{i\omega\tau} \ d\tau + \int_{-\infty}^{+\infty} \langle\zeta \rangle^2 e^{i\omega\tau} \ d\tau = \int_{-\infty}^{+\infty}|dA(\omega’)|^2\int_{-\infty}^{+\infty}e^{i(\omega-\omega’)\tau}\ d\tau$$ Performing the $\tau$ integration yields $$ S(\omega)+2\pi \langle \zeta \rangle^2 \delta(\omega)= 2\pi \int_{-\infty}^{+\infty} |dA(\omega’)|^2 \delta(\omega-\omega’) $$ Consider the case of $\omega = 0$ for a zero-mean process, i.e. $\langle\zeta\rangle = 0$; both sides of the expression reduce to $$ S(0)+0 = 2\pi \int_{-\infty}^{+\infty} |dA(\omega’)|^2 \delta(\omega’) $$ The Dirac delta function forces the terms on the right side to be evaluated at $\omega’ = 0$ to produce
$$ S(0) = 2\pi |dA(0)|^2\delta(\omega’) = 2\pi \langle \zeta \rangle ^2 \delta(\omega’) =0 $$ where the previous result $dA(0) = \langle\zeta\rangle$ has been used. Thus, it is shown that the spectrum for a zero-mean process is zero when the frequency is zero, i.e. $S(0) = 0$. It can be seen in Equation (1) for the general case that the required value of $|dA(\omega’)|^2$ must be given by $$|dA(\omega’)|^2 = \frac 1{2\pi}S(\omega’) \ d\omega’+|dA(0)|^2 \delta(\omega’) \ d\omega’ $$ $\mathbf Discussion -$ The vanishing of the spectrum at $\omega = 0$ establishes some new requirements on the autocorrelation function and spectrum. For a zero-mean random variable, when $\tau = 0$ the autocorrelation form of the Wiener-Khinchin theorem reduces to
$$ \sigma^2 = \frac 1{2\pi}\int_{-\infty}^{+\infty}S(\omega)\ d\omega $$ This expression has proven useful and has wide application. Now consider the spectrum form of the Wiener-Khinchin theorem when $\omega = 0$. This expression becomes
$$ S(0) = \int_{-\infty}^{+\infty}\sigma^2C(\tau)\ d\tau =0 $$ Besides the information on the behavior of the spectrum, this gives useful information on the autocorrelation function. For example, the autocovariance function must have both positive and negative excursions, and the area under these excursions must be equal otherwise the integral cannot be zero. This means that the simple Gaussian or exponential forms often used for the autocorrelation function are not valid for a zero mean process – they have no negative excursions anywhere.

Answer 2

The analysis shown by PT looks correct, but there seems to be a contradiction when considering an Ornstein-Uhlenbeck process, like $dx = -xdt+dW(t)$ with Wiener process W(t). There, $<x>=0$ if $<\cdot>$ is a temporal average or an ensemble average for long times $t\to\infty$. Since $S(f)\sim a/(1+f^2)$ with $a\ne 0$, $S(f=0)=a\ne 0$. However, this contradicts the statement of PT.

I appreciate the attempt of PT to explain this gap by the definition of centered processes and intrinsically zero-mean processes, but it does not explain mathematically the contradiction. Interestingly, estimating the spectral power density of an Ornstein-Uhlenbeck process numerically by a DFT, one gains the predicted $S(f)$ and $S(f)\ne 0$, whereas computing numerically the spectrogram based on just the DFT, i.e. $\sim |DFT(f)|^2$, one finds $S(f)\to 0$ for zero-mean processes.

Besides the mathematical aspect, one has to distinguish the notion of energy and power. Energy can be computed for a finite-duration signal $s(t)$ by $E=\int_{-\infty}^\infty S_e(f) df$ with its corresponding energy density $S_e(f)=|\tilde{s}(f)|^2$ with Fourier transform $\tilde{s}(f)$ at frequency f. For infinitely long signals, this energy would diverge and it is better to define the power $P=\int_{-\infty}^\infty S_p(f) df$ with the spectral power density (PSD) $S_p(f)=\lim_{T\to\infty}\frac{1}{T}|\tilde{s}_T(f)|^2$ with the Fourier transform $\tilde{s}_T(f)$ of signal segment $s_T(t)$ with duration $T$.

Hence, $S_p(f)$ is defined as a measure 'energy by time interval' in the limit for infinitely long signals, whereas $S_e(f)$ is the energy density of a finite-duration signal. Real-world zero-mean finite-duration signals show $S_e=|\tilde{s}(0)|=0$, while the PSD is not defined due to its intrinsic assumption of an infinite duration (by the way what the Wiener-Khinchin theorem requires as well). It is interesting to point out as well, that the zero-mean property of the signal does not occur anywhere in the definition of the PSD $S_p$ and hence it is independent from this property. This confirms the physical intuition, that the PSD is a power value in the sense of 'energy per time interval' and not 'mean energy' measure as $S_e$.