- Text
- Entropy,
- Tsallis,
- Statistical,
- Entropies,
- Mechanics,
- Probability,
- Shannon,
- Equation,
- Generalised,
- Axiom,
- Measures,
- Physics

Generalised information and entropy measures in physics

498 C. Becksatisfies all four of the Kh**in**ch**in** axioms. Indeed, up toan arbitrary multiplicative constant, one can easilyshow (see, e.g. [1]) that this is the only entropic formthat satisfies all four Kh**in**ch**in** axions, **and** that itfollows uniquely (up to a multiplicative constant) fromthese postulates. k denotes the Boltzmann constant,which **in** the rema**in****in**g sections will be set equal to 1.For the uniform distribution, p i ¼ 1/W, the Shannon**entropy** takes on its maximum valueS ¼ k ln W;ð11Þwhich is Boltzmann’s famous formula, carved on hisgrave **in** Vienna (Figure 2). Maximis**in**g the Shannon**entropy** subject to suitable constra**in**ts leads toord**in**ary statistical mechanics (see Section 4.3). Inthermodynamic equilibrium, the Shannon **entropy** canbe identified as the ‘physical’ **entropy** of the system,with the usual thermodynamic relations. Generally, theShannon **entropy** has an enormous range of applicationsnot only **in** equilibrium statistical mechanics butalso **in** cod**in**g theory, computer science, etc.It is easy to verify that S is a concavefunction of the probabilities p i , which is an importantproperty to formulate statistical mechanics.Remember that concavity of a differentiable functionf(x) means f 00 (x) 0 for all x. For the Shannon**entropy** one has@@p iS ¼ ln p i 1; ð12Þ@ 2@p i @p jS ¼ 1 p id ij 0; ð13Þ**and** hence, as a sum of concave functions of the p i ,itisconcave.In classical mechanics, one often has a cont**in**uousvariable u with some probability density p(u),rather than discrete microstates i with probabilitiesRp i . In this case the normalisation condition reads11pðuÞ du ¼ 1, **and** the Shannon **entropy** associatedwith this probability density is def**in**ed asZ 1S ¼ dupðuÞ lnðspðuÞÞ; ð14Þ1where s is a scale parameter that has the same dimensionas the variable u. For example, if u is a velocity(measured **in** units of m s 71 ), then p(u), as a probabilitydensity of velocities, has the dimension s m 71 , s**in**cep(u)du is a dimensionless quantity. As a consequence,one needs to **in**troduce the scale parameter s **in** Equation(14) to make the argument of the logarithmdimensionless.Mono for pr**in**tcolour onl**in**eFigure 2. The grave of Boltzmann **in** Vienna. On top of thegravestone the formula S ¼ k log W is engraved. Boltzmannlaid the foundations for statistical mechanics, but his ideaswere not widely accepted dur**in**g his time. He committedsuicide **in** 1906.Besides the Shannon ** information**, there are lotsof other

Contemporary Physics 499are called the Re´nyi entropies [26]. These are def**in**edfor an arbitrary real parameter q asS ðRÞq ¼ 1q 1 ln X p q i :ð15ÞiThe summation is over all events i with p i 6¼ 0. TheRe´nyi entropies satisfy the Kh**in**ch**in** Axioms 1–3 **and**the additivity condition (9). Indeed, they follow uniquelyfrom these conditions, up to a multiplicative constant.For q ! 1 they reduce to the Shannon **entropy**:limq!1 SðRÞ q ¼ S; ð16Þas can be easily derived by sett**in**g q ¼ 1 þ e **and** do**in**ga perturbative expansion **in** the small parameter e **in**Equation (15).The Re´nyi **in**formation**measures** are important forthe characterisation of multifractal sets (i.e. fractalswith a probability measure on their support [1]), aswell as for certa**in** types of applications **in** computerscience. But do they provide a good ** information**measure to develop a generalised statistical mechanicsfor complex systems?At first sight it looks nice that the Re´nyi entropiesare additive for

- Page 1 and 2: kothandamk 5/5/09 20:39 2008 - Styl
- Page 3: Contemporary Physics 497the simple
- Page 7 and 8: Contemporary Physics 501and hence S
- Page 9 and 10: Contemporary Physics 503W ! ? (reca
- Page 11 and 12: Contemporary Physics 505Its inverse
- Page 13 and 14: Contemporary Physics 507exhibits st
- Page 15 and 16: Contemporary Physics 509Lutz’ mic