Can entropy be negative?

Question

I've been asked the following question, and I can't solve it definitively. I've solved the first part, but I'm not sure about whether entropy can be negative or not. It wouldn't make much sense for it to be, but just by using inequalities and basic maths, I get the following inequality: $$\sigma^2 < \frac{1}{2e^{}\pi}$$ If that is satisfied, then the entropy is negative, and it certainly makes sense for the variance to be able to take that value. So am I going wrong somewhere, or can the entropy be negative?

Is $p(x)$ the density? If so, then $0 \leq p(x) \leq 1$, so $\log(p(x))$ is always negative, giving $-\log(p(x))$ always positive. And the expectation of a positive quantity is.... — Randall, Oct 31 '22 at 12:35
Also, it would be most helpful if you showed how you were deducing $\sigma^2 < \frac{1}{2e\pi}$ since your entire question hinges on that. — Randall, Oct 31 '22 at 12:38
Does this answer your question? Uniformly random number on $[0,1]$ has zero entropy? — leonbloy, Oct 31 '22 at 12:41
The differential entropy is not the Shannon entropy, it can be negative or zero. — leonbloy, Oct 31 '22 at 12:42
I am deducing that $$\sigma^2 < \frac{1}{2e^{}\pi}$$ by writing the following and working from there: $$\frac{\ln(2\pi\sigma^2)}{2} + \frac{1}{2} < 0$$ — BobMarley, Oct 31 '22 at 12:55
I am pretty sure I am being asked for the Shannon entropy, given the formula supplied. Also, I 100% get that if $$p(x)$$ is the probability density then it's done (and I'm pretty sure that this is the case), but I'm being asked to use the right hand side of the equation to prove it, and the right hand side of the equation seems like it could be negative. — BobMarley, Oct 31 '22 at 12:58
Asked tons of times (one of them, two hours ago)
https://math.stackexchange.com/questions/4565670/how-can-differential-entropy-be-negative — leonbloy, Oct 31 '22 at 14:06
Does this answer your question? How can differential entropy be negative? — Randall, Oct 31 '22 at 14:57

Can entropy be negative?

0 Answers0