New trends in physics teaching, v.4; The ... - unesdoc - Unesco

Entropy and Information
Inserting L X lo-’ m, p % kg m/s we obtain Axl = 10- m. The uncertainty at the first
collision is thus about 10 times the diameter of a molecule so that the specific molecule aimed at
wilI be hit only with a probability of about 1 per cent. The detailed, molecular processes occuring
in a gas are thus CO etely undetermined after lom9 s and each repetition of an experiment
- starting from the same initial conditions - wil lead to a completely different course of events,
when detailed questions about the positions and velocities of molecules are asked. The same
applies to any attempt to trace the past positions of molecules. Physicists therefore turn out to
be very poor historians, who cannot even calculate details over a timespan of s when dealing
with individual molecules in a gas.
Any attempt to obtain detailed information about the individual positions of molecules in a
gas is therefore completely useless. A precise prediction of detailed events, corresponding to the
ideals of classical physics, is impossible in principle. It is thus necessary to restrict oneself to
‘ensembles’ of thermodynamic systems which have the same macroscopic properties but differ
in their individual molecular details.
The simple arguments given here have shown the futility of all attempts of detailed calculations
concerning thermodynamic systems. They show that the lack of knowledge about molecular
details lead in general to a further decrease of the available information in the course of time [91 .
These preliminary considerations enable us now to introduce the concept of entropy. Entropy
is a measure of the missing information about the molecular details of gases or other thermodynamic
systems and is defined as follows:
The entropy S of a thermodynamic system is given by
S = 0.7 X k X (missing information) (Eq. 6)
In this relation the missing information is measured in bit and k is the Boltzmann constant
(1.38 X J/K). The conversion factor 0.7k (more accurately k ln2) between information and
entropy wil turn out to be convenient [ lo].
IRREVERSIBLE PROCESSES
I
I
‘The passage of heat from a colder to a hotter body cannot take place without compensation.’
Starting from this apparently trivial statement, Rudolf Clausius was able to prove in 1865 the
existence of a function of state S - the entropy - for every thermodynamic system [ 11 I . His
argument showed furthermore that the entropy remains constant for reversible processes in
isolated systems and increases during irreversible processes. From the point of view of information
theory, this increase of entropy corresponds to a loss of information about the thermodynamic
system as the following examples wil show.
As a first example of a typical irreversible process we consider the expansion of a gas into a
vacuum (see figure 3). During this process information is lost. Before the expansion we were
able to answer the question: ‘Left or right?’ for each molecule. After the expansion this is no
longer possible. Therefore one bit of information is lost for each molecule, leading to a decrease
of N bit of information available about the thermodynamic system. The corresponding increase
of entropy is
AS = 0.7 k N. 0%. 7)
A similar loss of information occurs when two gases consisting of N/2 molecules each are
45