If $\delta$ is defined as a distribution, then the object (you wrote it as an integral) is ill-defined. This is because the product of distributions is not defined in general. Mathematica might be using the right-side limit of the Heaviside function to evaluate this ill-defined object.
Here, let $f_x(\tau)=\frac{e^{-x^2/4\tau}}{\sqrt{4\pi\tau}}$ and $H(\tau)$ denote the Heaviside function. The object written as $\int_0^\infty f_x(\tau)\delta(\tau)\,d\tau$ can be interpreted as the functional $\langle \delta,f_xH \rangle$. Inasmcuh as $f_x(\tau)H(\tau)$ is discontinuous at $\tau=0$, the functional is not defined. That is to say, $f_x(\tau)H(\tau)$ is not a test function.
In fact, the functional $\langle \delta,H\rangle$ is not defined. To see this, let $\delta_n(\tau)$ be a regularization of the Dirac Delta. For any test function $\phi$, we have
$$\lim_{n\to\infty}\int_{-\infty}^\infty \phi(\tau)\delta_n(\tau)\,d\tau=\phi(0)$$
Note that there is no requirement that $\delta_n(\tau)$ be symmetric about the origin. Therefore, the integral $\int_{-\infty}^\infty H(\tau)\delta_n(\tau)\,d\tau=\int_0^\infty \delta_n(\tau)\,d\tau$ need not be equal to $1/2$. In fact, the value of the integral can be any value between $0$ and $1$ (See THIS ANSWER).
Inasmuch as the limit, $\lim_{n\to\infty}\int_{-\infty}^\infty H(\tau)\delta_n(\tau)\,d\tau$ depends on the choice of regularization $\delta_n$, the functional $\langle \delta,H\rangle$ fails to exist.