8

As in the title, I was wondering whether the entropy of a system (it can be any entropy, from Boltzmann to Renyi etc, it is of no importance) is a function or a functional and why? Since it is mostly defined as: $$S(p)=\sum_{i}g(p_i) $$ for some $g$ that has to be continous etc then it has to be a functional. But then I see that $S_{BG}$ for example, which is defined as $S_{BG}=\sum_i p_i \log p_i$ just needs the value of each $p_i$ in order to be defined, right?

The way I see it, it has to be a functional but it is not clear to me why. Also many authors mention the entropy as a function while others call it a functional.

Thank you!

Bazinga
  • 1,543
  • 2
    What is the difference between function and functional supposed to be? To many, the terms are synonyms. – Did Jul 04 '16 at 08:45

1 Answers1

6

A function is a mapping between a set of numbers and another set of numbers. A functional is a mapping between a set of functions and another set of functions. The entropy is defined as the Gibbs functional: $$S(p)=-k\sum_jp_j\log(p_j)$$ where the $p_j$ are functions. So the correct way to define the entropy, following Gibbs, is a functional

Riccardo.Alestra
  • 10,746
  • So... $f:\mathbb R^2\to\mathbb R$, $(x,y)\mapsto f(x,y)=x+2y$, is not a function? – Did Jul 04 '16 at 09:16
  • @Did: https://en.wikipedia.org/wiki/Functional_(mathematics) – Riccardo.Alestra Jul 04 '16 at 09:42
  • Yeah -- but what about the question in my comment? Function or not function? 'Cause if "not function", then I, and a bunch of my colleagues, have to revise our teaching methods on the spot... – Did Jul 04 '16 at 09:47
  • @Did: really the difference between a function and a functional is not so sharp. There are examples of functions that could be considered as functionals. In your example, you can assign a number to $x$ and $y$ and you get a number in $f(x,y)$. In a functional you put a function in a formula and you get a function as a result. – Riccardo.Alestra Jul 04 '16 at 09:55
  • Indeed -- and this could make you want to modify the pair of very definitive assertions which currently open your answer. – Did Jul 04 '16 at 10:19
  • @Riccardo.Alestra But aren't $p_j$ probability values for each $j$? I mean, I do understand the difference between a functional and a function. A functional is defined on a space of functions and a special case of a functional is a function. But what about the entropy? If all it needs is the value of $p_j$ then is it a functional? – Bazinga Jul 04 '16 at 11:28
  • @Did To my best knowledge your example is a function since it is defined on $\mathbb{R}^2$. It should be defined on the dual space in order for it to be a functional. – Bazinga Jul 04 '16 at 11:35
  • 1
    @Mitscaype: yes, but the $p_j$ are fuinctions – Riccardo.Alestra Jul 04 '16 at 11:37
  • @Riccardo.Alestra Moreover, in your reply you say that the entropy is defined as the Gibbs entropy but that does not hold always. There are plenty other entropies which fullfil the Shannon KiKinchin theorems. – Bazinga Jul 04 '16 at 11:38
  • @Riccardo.Alestra Ok, therefore $p_j$ are the functions that show the probability the system will happen to be in the particular $j$ state. Correct? – Bazinga Jul 04 '16 at 11:39
  • 1
    @Mitscaype: correct – Riccardo.Alestra Jul 04 '16 at 11:42
  • @Mitscaype: https://arxiv.org/ftp/arxiv/papers/1107/1107.1739.pdf – Riccardo.Alestra Jul 04 '16 at 11:44
  • @Riccardo.Alestra Ok, I will take a look in the paper. Thanks! – Bazinga Jul 04 '16 at 12:05