I am learning about entropy, and am slightly confused by an exercise in a book. It asks if there exists a discrete random variable $X$ with distribution such that $H(X)=\infty$. My intuition is that there is not, as the set is always countable as it is a discrete random variable, and the probability is less than one, and the definition for entropy is $$H(X)=-\sum_{x\in{X}}P(X=x)log(P(X=x))$$ Any help understanding this would be appreciated. Thank you.
Asked
Active
Viewed 790 times
1
-
1This question is answered here: https://math.stackexchange.com/questions/279304/can-the-entropy-of-a-random-variable-with-countably-many-outcomes-be-infinite – Joe Oct 26 '19 at 12:06
-
Possible duplicate of Can the entropy of a random variable with countably many outcomes be infinite? – leonbloy Nov 02 '19 at 14:54