11

I encountered a problem at work that, in my opinion, has a fundamental mathematical reasoning to determine its solvability.

Due to an unwitting software configuration, my associate recorded the audio of our meeting twice, resulting in the audio sounding like the sum of the original signal combined with a slightly delayed copy of itself.

The final audio file can be thought as a time-dependent signal $g(t)$ that is related to the original audio signal, $f(t)$, by the following equation

$$g(t) = f(t) + f(t+\varepsilon)$$

Knowing $g(t)$ and $\varepsilon$ is it possible to approximate $f(t)$, either analytically or numerically?

My primary intuition was that it would be possible, but now it feels like trying to unscramble two eggs after they were mixed in a bowl.

Rodrigo de Azevedo
  • 23,223
nickh
  • 485
  • 1
    $f$ cannot always be recovered. For example, if $g(t) = 0$ and $\epsilon = \pi$, $f(t)$ could be any of the infinitely many functions $f(t) = A \sin t$. – Travis Willse Jun 28 '23 at 22:11
  • 1
    More generally, for any $\epsilon$, signals with frequency an odd multiple of $\frac{\pi}{\epsilon}$ cannot be detected. – Travis Willse Jun 28 '23 at 22:17
  • 2
    @TravisWillse: in general I think there will be uncertainty up to functions period $2\epsilon$ where the second half is the negative of the first. You could try to smooth this out, but then precision errors could start to creep into the calculations. – Henry Jun 28 '23 at 22:20
  • 1
    @Henry Right, and any such function will decompose into a (possibly infinite) linear combination of the functions I mention. – Travis Willse Jun 28 '23 at 22:24
  • 2
    Assuming $f(t)$ is nonzero only for $t \in [0, T]$ we get $g(t-\epsilon) = f(t)$ for $t \in [0, \epsilon)$. So we recover $f$ exactly on $[0, \epsilon)$. Then $g(t-\epsilon) = \underbrace{f(t-\epsilon)}_{known} + f(t)$ for $t \in [\epsilon, 2\epsilon)$. In this way we can perfectly recover $f(t)$ for all $t \in \mathbb{R}$ by piecing the solutions together over intervals $[0, \epsilon), [\epsilon, 2\epsilon), [2\epsilon, 3\epsilon)$ and so on. – Michael Jun 28 '23 at 22:50
  • 1
    Nice approach @Michael, I believe that it would be especially useful for larger values of $\varepsilon$ or in performance-sensitive restoration algorithms. – nickh Jun 28 '23 at 22:55
  • 2
    The approach of @Michael will accumulate sampling errors. As the distortions are audible, the delay $ϵ$ is substantial, frequency-dependent cancellation does occur in the spectrum of human perception. Reconstruction from nearly erased data is always an ill-posed problem. – Lutz Lehmann Jun 29 '23 at 05:13
  • 1
    https://dsp.stackexchange.com/q/36264/21075 – Rodrigo de Azevedo Jul 04 '23 at 07:38

4 Answers4

9

A possible avenue, mathematically, may be the Fourier transform.

Let $$ \newcommand{\FF}{\mathcal{F}} \newcommand{\dd}{\mathrm{d}} \FF[f](\xi) := \int_{-\infty}^\infty f(x) e^{-2\pi i \xi x} \, \dd x $$ denote the usual one-dimensional transformation for $f$.

We know two properties:

  • $\FF$ is linear, in the sense that given $\alpha,\beta \in \mathbb{C}$, we have $\FF[\alpha f + \beta g] = \alpha \FF[f] + \beta \FF[g]$

  • $\FF$ satisfies certain time-shift properties. In particular, $$ \FF[f(x-x_0)](\xi) = e^{2\pi i x_o \xi} \FF[f](\xi) $$

Consequently, if $g(x) = f(x) + f(x+\varepsilon)$, then $$\begin{align*} \FF[g](\xi) &= \FF[f(x) + f(x+\varepsilon)](\xi) \\ &= \FF[f](\xi) + e^{-2\pi i \varepsilon \xi} \FF[f](\xi) \\ &= \FF[f](\xi) \Big( 1 + e^{-2\pi i \varepsilon \xi} \Big) \end{align*}$$

Therefore, assuming we know $g,\varepsilon$, $$ \FF[f](\xi) = \frac{\FF[g](\xi) }{ 1 + e^{-2\pi i \varepsilon \xi}} $$ To recover $f$, one uses the similarly-defined inverse Fourier transform, $$ \FF^{-1}[f](x) := \int_{-\infty}^\infty f(\xi) e^{2\pi i \xi x} \, \dd \xi $$ or, more appropriately, $$ f(x) = \FF^{-1}\Big[ \FF[f] \Big](x) := \int_{-\infty}^\infty \FF[f](\xi) e^{2\pi i \xi x} \, \dd \xi $$

However, there are no guarantees it would be easy to calculate this inverse, and depending on the $\varepsilon,\xi$ involved it might not even be well-defined (say, if the exponential in the denominator evaluated to $-1$). Based on the other comments, issues may even arise for certain types of $f$.

PrincessEev
  • 50,606
  • 2
    (+1) An excellent answer. We can see in your derivation the problem with the signals with angular frequency an odd multiple of $\frac{1}{2 \epsilon}$---these are exactly the signals for which the denominator $1 + e^{-2 \pi i \epsilon \xi}$ of your expression for $\mathcal{F}f$ always vanishes. In that sense, those signals are the only components for which your procedure is not defined. In practice there may be numerical issues with components of the signal whose angular frequency is close to the forbidden value, where the denominator becomes very small. – Travis Willse Jun 28 '23 at 22:31
  • Brilliant! When working with discrete-time signals, such as data in a digital audio file, I believe this process translates to determining the DFT of $g$, dividing each sample $G(n)$ by $1+\exp(−2\pi n i\varepsilon)$ so that the IDFT of the result is the recovered data. – nickh Jun 28 '23 at 22:38
  • 1
    Ohhhhhh I see, that explains the comment about the multiples of $1/(2 \varepsilon)$, I don't know why I didn't think to actually see what would cause those undefined values. Thanks Travis! – PrincessEev Jun 28 '23 at 22:39
  • 1
    You cannot really convolve $g$ with the inverse transform of $\frac{1}{1+e^{-2\pi i \epsilon \xi}}$ as it seems to have no inverse transform. – Michael Jun 28 '23 at 22:47
5

Assume that $f$ is (locally approximated by) a straight line, $f(t) = at + b$. This gives us:

$$f(t + \epsilon) = a(t + \epsilon) + b = at + b + a\epsilon$$ $$g(t) = 2at + 2b + a\epsilon$$

We now have two coefficients to solve for, so assume that we know $g(t_1)$ and $g(t_2)$, giving us the system of equations:

$$g(t_1) = (2t_1 + \epsilon)a + 2b$$ $$g(t_2) = (2t_2 + \epsilon)a + 2b$$

Solving for $a$ and $b$ gives:

$$a = \frac{g(t_2) - g(t_1)}{2(t_2 - t_1)}$$ $$b = \frac{(2t_2 + \epsilon)g(t_1) - (2t_1 + \epsilon)g(t_2)}{4(t_2 - t_1)}$$

And plugging these into the formula for $f$ gives:

$$f(t) = \frac{(2(t - t_1) - \epsilon)g(t_2) - (2(t - t_2) - \epsilon)g(t_1)}{4(t_2 - t_1)}$$

If we choose $t_1 = t$ and $t_2 = t + \epsilon$, then this simplifies to:

$$\boxed{f(t) = \frac{3g(t) - g(t + \epsilon)}{4}}$$

If $f$ is expressible as a Taylor series, then the maximum error in this approximation is $\frac{1}{4}\epsilon^2 |f''(t)|$.

Dan
  • 18,262
2

Υοur second feelings are correct from a purely mathematical point of view. For a given $\varepsilon$, there are many functions $f$ such that $f(t) = - f(t + \varepsilon)$ for all $t$. For such an $f$, $g(t) = 0$ for all $t$, and so $g$ gives no information about $f$ apart from that fact that $f(t) = f(t + \varepsilon)$ for all $t$.

It may be possible to retrieve some information about $f$, if you make some assumptions about it based on the physics of the situation (which make my above contrived example highly unlikely). Probably some kind of machine learning approach would be a good place to start. MSE is probably not the best forum to ask if you are interested in following that up.

Rob Arthan
  • 51,538
  • 4
  • 53
  • 105
  • Thank you for your input! As for numerically estimating $f(t)$, I wonder which mathematical conditions should be verified to make it viable. For instance, considering $\varepsilon\to 0$ (the delay is relatively small). – nickh Jun 28 '23 at 22:21
  • 1
    For numerical estimation, I've no doubt @PrincessEev's answer is the way to go. For the machine learning, the kind of thing I have in mind is that you'd sample some actual meetings recording $f$ and calculating $g$ for some typical values of $\epsilon$ (BTW, do you know $\epsilon$?), then you'd attempt to train a system to recover $f$ from $g$. It would be interesting to know how that compares in practice with the Fourier transform approach. – Rob Arthan Jun 29 '23 at 20:33
  • The machine learning approach is a great idea for a project, I wonder how the quality and the algorithmic efficiency of each restoration process differ. And yes, I can determine $\varepsilon$ by inspecting how frequency and power change with time. In a plot such as: http://mres.uni-potsdam.de/index.php/2020/07/08/analyzing-music-in-the-time-and-frequency-domain-using-matlab/ we can identify the time that a frequency outliers takes to repeat itself and estimate $\varepsilon$ like that. – nickh Jun 29 '23 at 22:19
1

You problem is one of deconvolution:

$$ g(t) = f(t) \ast \left( \delta(t) + \delta(t+\epsilon) \right) = f(t) \ast h(t), $$

where $h(t) = \delta(t) + \delta(t+\epsilon)$. In the frequency domain, this is

$$ G(\omega) = F(\omega) \cdot H(\omega). $$

Thus, to recover $f(t)$, we deconvolve $g(t)$ with $h(t)$:

$$ f(t) = \mathcal{F}^{-1} \left( { G(\omega) \over H(\omega) } \right), $$

where $\mathcal{F}$ is the inverse Fourier transform. For the system to be solvable, $H(\omega) \neq 0$.

There are better ways to do deconvolution that are more stable, but the above is the gist of it and where I would suggest you start.

AnonSubmitter85
  • 3,605