2

Let $P$ be an infinite partition of the interval $[0,1]$. Let $P$ have elements $I_i$ which has Lebesgue measure $m(I_i)$.

Then the entropy of $P$ is defined by $\sum_i -m(I_i)\log m(I_i)$. Can this be infinite?

leonbloy
  • 66,202
a12345
  • 1,343
  • Is $m(I_i)$ the measure of the $i$th interval? – Semiclassical Jul 25 '14 at 16:59
  • yes, thanks for your comment – a12345 Jul 25 '14 at 17:03
  • Since $\sum m(I_i) = 1$, the question can be equivalently restated as follows: is there a probability distribution on $\Bbb N$ whose entropy is infinite. As far as I understand, the answer is no as follows from here. – SBF Jul 25 '14 at 17:20
  • Thanks for the link. What is the "mean" that they are talking about? – a12345 Jul 25 '14 at 17:37
  • That I'm not sure I understand: the entropy of a discrete distribution does not depend on its underlying space, so you shall be able to assume the mean to be anything. – SBF Jul 25 '14 at 18:22
  • Related (practically dup) http://math.stackexchange.com/questions/279304 – leonbloy Jul 25 '14 at 18:38

1 Answers1

1

This paper provides an example of a distribution on $\Bbb N$ which has infinite entropy. Thus, in your case $P$ is any partition satisfying $$ m(I_i) = \frac{1}{\lg(i+1)} - \frac{1}{\lg(i+2)}, \qquad i\in \Bbb N, $$ where $\lg i$ is a logarithm base $2$. For example, $I_1 = [1,\frac1{\lg3})$, $I_2 = [\frac1{\lg 3},\frac12)$, $I_3 = [\frac12,\frac1{\lg5})$ etc.

SBF
  • 36,568