Statistical and Thermal Physics

Using Python

Anders Malthe-Sørenssen1
Dag Kristian Dysthe1

1
Department of Physics, University of Oslo, Norway

Jul 30, 2017

 Preface

This book was developed as a series of lecture notes for the course Fys2160
Statistical and thermal physics at the University of Oslo, Norway. The
motivation for the book project was to develop a text that integrates the
use of computational methods deeply into the exposition, examples and
exercises. In particular, the aim is to demonstrate the whole problem
solution process, from the initial complex physics problem, through
simplified models that have analytical solutions and to more complex and
realistic models that defy analytical approaches, but that can be addressed
using computational models. Our experience is that this approach more
closely reflects the real work processes found in research and industry,
and that the approach make the curriculum more open — there are
more processes and phenomena that can be addressed, and there are
many ways to do that. It also allows students to experience that various
approaches have various advantages and disadvantages.
A particular challenge when introducing programming and computa-
tional methods is to find a reasonable and productive way to include the
computer programs. Here, we have selected a particular programming
language, python, and used this exclusively throughout the book. How-
ever, there are two versions of this book, one using Python and one using
Matlab. Everything but the programs and the associated syntax-specific
comments are the same.
We have also found it useful to include simplifications, analytical and
computational methods in the same worked examples and exercises. This
provides the students with a complete worked example that also can be

v
vi

used as a basis for exploring new phenomena. The code can be found
both in the book and can be downloaded from the book website1 .
All the examples and some of the exercises in the book have been
compiled as iPython notebooks, and you can find and use the notebooks
at the book notebook site2 .
Since the contents of this course is considered an advanced course, the
tradition in most texts is to not include complete worked examples. We
have found this practice unfortunate, since worked examples are powerful
pedagogical tools, and research have shown that going through a worked
example before addressing exercises yourself will improve the learning
outcomes. You will therefore find some worked examples in this book,
and additional sets of worked examples in the worked example site3 .
Statistics is an essential part of statistical physics. We have therfore
included an introductory chapter on statistics. At the University of
Oslo this introduction to statistics served as an important part of the
overall learning outcomes of the bachelor degree, where all physics courses
provide introduction to specific statistical terms. In this text we introduce
basic probability theory, including most common probability densities
and distributions, basic combinatorics, expected values and variance
as well as estimators. We also introduce the concept of Monte Carlo
modelling, both in the basic introduction to statistics, but also as an
important tool to understand physics.
We have selected several computational approaches that are used
throughout the textbooks. We introduce the essentials of Molecular
Dynamics modeling, since this is used to gain insight into the irreversible
nature of systems with many particles and to provide basic estimates
for the properties of real gases, liquids, and solids. We also provide a
general introduction to Monte Carlo methods and use these methods
systematically throughout the text. We start from a general lattice-
based model that is then reused for many different systems. This also
strengthens a central learning outcome — that physics often is about
recognizing how a new model can be mapped onto a well known model
for which we already know the solution. In addition, we also introduce
various algorithmic models, which are stochastic or Monte Carlo type,
but not always following a Metropolis type algorithm. And we will use
continuum methods to address heat and particular transport in diffusion
or flow problems.
1
http://folk.uio.no/malthe/fys2160/index.html
2
http://folk.uio.no/malthe/fys2160/notebooks
3
http://folk.uio.no/malthe/fys2160/workedexamples
 vii

January 2017 Anders Malthe-Sørenssen and Dag Kristian Dysthe

 Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.0.1 Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.0.2 The great success of statistical mechanics . . . . . . . . . . . 4
1.0.3 Integrated numerical methods . . . . . . . . . . . . . . . . . . . . . 4
1.0.4 Molecular dynamics modeling . . . . . . . . . . . . . . . . . . . . . 5
1.0.5 Learning Physics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.0.6 Prerequisites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.0.7 Structure of this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.0.8 Key questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 The Arrow of Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1 One and many particles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.1 Reversible bouncing of a solid ball . . . . . . . . . . . . . . . . . 10
2.1.2 Irreversible bouncing of an elastic ball . . . . . . . . . . . . . . 10
2.1.3 More degrees of freedom . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 Approach to equilibrium — molecular dynamics . . . . . . . . . . . 14
2.2.1 Approach to equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 Approach to equilibrium — algorithmic model . . . . . . . . . . . . 20
2.3.1 Model of atoms on the left side . . . . . . . . . . . . . . . . . . . . 20
2.3.2 Simplifying the system — part 1 . . . . . . . . . . . . . . . . . . 21
2.3.3 Simplifying the system — part 2 . . . . . . . . . . . . . . . . . . 22

ix
x Contents

2.3.4 Fluctuations and the number of atoms, N . . . . . . . . . . 23

2.3.5 Average value of n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.3.6 Dependence on initial conditions . . . . . . . . . . . . . . . . . . 25
2.4 Approach to equilibrium — theoretical model . . . . . . . . . . . . . 26
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.6.1 Discussion questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.6.2 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3 Getting started with Molecular Dynamics modeling . 31

3.1 Atomic modeling basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.1.1 Lennard-Jones potential . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1.2 Initial conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.1.3 Boundary conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.1.4 Integrating the equations of motion . . . . . . . . . . . . . . . . 36
3.2 Simple implementation of a molecular dynamics simulator . . 37
3.2.1 Non-dimensional equations of motion . . . . . . . . . . . . . . 37
3.2.2 Initialization of system . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2.3 Integration of the equations of motion . . . . . . . . . . . . . . 41
3.3 Running a molecular dynamics simulation . . . . . . . . . . . . . . . . 42
3.3.1 Installation of LAMMPS . . . . . . . . . . . . . . . . . . . . . . . . . 42
3.3.2 Starting a simluation in LAMMPS . . . . . . . . . . . . . . . . . 43
3.3.3 Visualizing the results of a simulation . . . . . . . . . . . . . . 48
3.4 Analyzing the results of a simulation . . . . . . . . . . . . . . . . . . . . 49
3.4.1 Matlab implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 49
3.4.2 Python implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 51
3.4.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.5 Advanced models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.5.1 Coarsening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
3.5.2 Hemoglobin in water . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5.3 Lipid bilayers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5.4 Fracture of Silicon crystal . . . . . . . . . . . . . . . . . . . . . . . . 54
3.5.5 Water permeation in a carbon nanotube . . . . . . . . . . . . 54
3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.7.1 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
 xi

4 Probability and Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.1 Motivating example — an ideal gas . . . . . . . . . . . . . . . . . . . . . . 58
4.2 Probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.2.1 Statistical experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
4.2.2 Generating a sequence of random numbers . . . . . . . . . . 61
4.2.3 Measuring the frequency of occurrence . . . . . . . . . . . . . 62
4.2.4 Frequencies and probabilities . . . . . . . . . . . . . . . . . . . . . . 63
4.2.5 Example: Probability of a dice . . . . . . . . . . . . . . . . . . . . 64
4.2.6 Properties of probabilities . . . . . . . . . . . . . . . . . . . . . . . . 66
4.2.7 Probability for simultaneous outcomes . . . . . . . . . . . . . 68
4.2.8 Example: Throwing two dice . . . . . . . . . . . . . . . . . . . . . . 70
4.3 Expected value and Variance . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3.1 Average . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3.2 Expected value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.3.3 The expected value is linear . . . . . . . . . . . . . . . . . . . . . . 72
4.3.4 Expected value of a function g(N ) of a random variable
N ............................................. 73
4.3.5 Example: Expected value of a die throw . . . . . . . . . . . . 73
4.3.6 Variance and Standard deviation . . . . . . . . . . . . . . . . . . 74
4.3.7 Standard deviation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.3.8 Example: Variance and standard deviation of a die
throw . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.4 Toward a theory for n(t) for the MD model . . . . . . . . . . . . . . . 76
4.4.1 One atom model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
4.4.2 Two-atom model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
4.4.3 Many-atom system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
4.4.4 Comparing theory and simulation . . . . . . . . . . . . . . . . . 81
4.5 The binomial distriubtion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
4.5.1 Example: Coin toss . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
4.5.2 Properties of the binomial distribution . . . . . . . . . . . . . 86
4.5.3 Expected value and variance of the binomial distribution 87
4.5.4 Continuum limit of the binomial distribution . . . . . . . . 89
4.5.5 Continuum limit — Analytical derivation . . . . . . . . . . . 89
4.5.6 Continuum limit — Symbolic derivation . . . . . . . . . . . . 93
4.5.7 Probabilities and probability densities . . . . . . . . . . . . . . 95
4.6 Key Concepts in Combinatorics . . . . . . . . . . . . . . . . . . . . . . . . . 96
4.7 Universal distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
xii Contents

4.7.1 Central limit theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4.7.2 Extreme value statistics . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.8 Probability distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
4.8.1 The probability density and the cumulative distribution 99
4.8.2 Uniform distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
4.8.3 Exponential distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 101
4.8.4 Poisson distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
4.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
4.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

5 A thermodynamical example . . . . . . . . . . . . . . . . . . . . . . . . 113

5.1 Ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
5.1.1 Ideal gas — Results from simulations . . . . . . . . . . . . . . 114
5.1.2 Ideal gas — Results from theory . . . . . . . . . . . . . . . . . . . 117
5.1.3 Ideal gas — equipartition principle . . . . . . . . . . . . . . . . 118
5.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

6 Micro- and Macro-states . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

6.1 States of the model gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
6.1.1 Micro- and macro-states for two dice . . . . . . . . . . . . . . . 122
6.1.2 Definition of micro- and macro-states . . . . . . . . . . . . . . 123
6.1.3 Multiplicity of the macrostate of the gas . . . . . . . . . . . . 124
6.2 Behavior of a solid — Observations . . . . . . . . . . . . . . . . . . . . . . 127
6.3 Behavior of a solid — Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.3.1 Einstein crystal model . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
6.3.2 Two Einstein crystals in contact . . . . . . . . . . . . . . . . . . . 134
6.3.3 Time development of the Einstein crystal — Monte
Carlo model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
6.3.4 Time development of the Einstein crystal —
Theoretical argument . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
6.3.5 Approximate multiplicities using Stirling’s formula . . . 142
6.4 The ideal gas — Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
6.4.1 Microstates of a single particle . . . . . . . . . . . . . . . . . . . . 148
6.4.2 Microstates of two non-interacting particles . . . . . . . . . 150
6.4.3 Microstates of N non-interacting particles . . . . . . . . . . 151
 xiii

6.4.4 Interaction between two ideal gases . . . . . . . . . . . . . . . . 152

6.5 Thermal equilibrium, Entropy and Temperature . . . . . . . . . . . 154
6.5.1 Equilibrium of the Einstein crystal . . . . . . . . . . . . . . . . . 156
6.5.2 Temperature in the Einstein crystal . . . . . . . . . . . . . . . . 159
6.5.3 Example: Entropy and energy of the Einstein crystal . 161
6.5.4 Example: Entropy and energy of the ideal gas . . . . . . . 162
6.5.5 Example: Entropy of expansion . . . . . . . . . . . . . . . . . . . 163
6.5.6 Example: Entropy of mixing . . . . . . . . . . . . . . . . . . . . . . 164
6.5.7 Example: Gibb’s paradox . . . . . . . . . . . . . . . . . . . . . . . . . 165
6.6 The laws of thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
6.6.1 Example: Entropy and heat . . . . . . . . . . . . . . . . . . . . . . . 169
6.7 Mechanical equilibrium — Pressure . . . . . . . . . . . . . . . . . . . . . . 171
6.7.1 Example: Pressure of ideal gas . . . . . . . . . . . . . . . . . . . . 174
6.8 The thermodynamic identity . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
6.8.1 The thermodynamic identity and partial derivatives . . 175
6.8.2 Entropy, Heat and Work . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.8.3 Process description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.9 Diffusive equilibrium — Chemical potential . . . . . . . . . . . . . . . 178
6.9.1 Thermodynamical identity . . . . . . . . . . . . . . . . . . . . . . . . 180
6.9.2 Example: Chemical potential of the Einstein crystal . . 180
6.9.3 Example: Chemical potential of the ideal gas . . . . . . . . 181
6.10 Simulations of the microcanonical system . . . . . . . . . . . . . . . . 182
6.10.1 Molecular dynamics simulations . . . . . . . . . . . . . . . . . . . 182
6.10.2 Stochastic simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.10.3 Monte Carlo simulations . . . . . . . . . . . . . . . . . . . . . . . . . 186
6.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 188
6.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189

7 The canonical system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

7.1 Boltzmann statistics in the Einstein crystal . . . . . . . . . . . . . . . 194
7.1.1 Einstein crystal – numerical example . . . . . . . . . . . . . . . 194
7.1.2 Einstein crystal in heat bath — Theory . . . . . . . . . . . . 198
7.1.3 Comparison with numerical model . . . . . . . . . . . . . . . . . 200
7.2 The partition function and Boltzmann statistics . . . . . . . . . . . 201
7.2.1 Average values and fluctuations . . . . . . . . . . . . . . . . . . . 202
7.2.2 Example: Two-state system . . . . . . . . . . . . . . . . . . . . . . . 203
xiv Contents

7.2.3 Example: Fluctuations and heat capacity . . . . . . . . . . . 206

7.2.4 Sharpness of the canonical system . . . . . . . . . . . . . . . . . 206
7.2.5 Example: Einstein crystal in the canonical system . . . 207
7.2.6 Example: Single spin system . . . . . . . . . . . . . . . . . . . . . . 209
7.2.7 Example: N -spin system . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.2.8 Example: Ising spin system . . . . . . . . . . . . . . . . . . . . . . . 211
7.2.9 Degenerate states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
7.2.10 Example: System with degenerate states . . . . . . . . . . . . 214
7.3 Thermodynamics in the canonical ensemble . . . . . . . . . . . . . . . 215
7.3.1 Helmholtz free energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.3.2 A microscopic expression for Helmholtz free energy . . 217
7.3.3 Helmholz free energy decreases in approach to
equilbrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
7.3.4 Helmholtz free energy is a minimum in equilibrium . . 225
7.3.5 Maxwell relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.4 Composite systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.4.1 Aspects of motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226
7.4.2 System of two particles . . . . . . . . . . . . . . . . . . . . . . . . . . . 228
7.4.3 System of many particles . . . . . . . . . . . . . . . . . . . . . . . . . 229
7.4.4 Example: Ideal gas in the canonical ensemble . . . . . . . 230
7.4.5 Example: Comparing microcanonical and canonical
analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
7.4.6 Example: Extension of a long polymer . . . . . . . . . . . . . 237
7.4.7 Example: Ising model in 1d and 2d . . . . . . . . . . . . . . . . 240
7.4.8 Example: Grain boundaries . . . . . . . . . . . . . . . . . . . . . . . 240
7.5 On the great importance of kT . . . . . . . . . . . . . . . . . . . . . . . . . 241
7.6 Transformation between ensembles . . . . . . . . . . . . . . . . . . . . . . 241
7.6.1 Example: Fuel cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
7.6.2 Example: Muscle cell . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
7.7 Equipartition and velocity distributions . . . . . . . . . . . . . . . . . . 247
7.7.1 Equipartition theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
7.7.2 Maxwell Velocity Distribution . . . . . . . . . . . . . . . . . . . . . 248
7.8 Statistical entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
7.9 General Monte-Carlo framework . . . . . . . . . . . . . . . . . . . . . . . . 252
7.10 Simulations of a canonical system . . . . . . . . . . . . . . . . . . . . . . . 253
7.10.1 Monte Carlo estimates of integrals . . . . . . . . . . . . . . . . . 253
 xv

7.10.2 Monte Carlo method for a canonical system . . . . . . . . . 255

7.10.3 Detailed balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
7.11 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
7.13 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269

8 Thermodynamics and Heat engines . . . . . . . . . . . . . . . . . . 271

8.1 Thermodynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272
8.2 Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
8.2.1 A simple piston . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
8.2.2 Net work and efficiency of the machine . . . . . . . . . . . . . 278
8.2.3 Limits on efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280
8.3 The Carnot cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
8.3.1 Carnot cycle for ideal gas . . . . . . . . . . . . . . . . . . . . . . . . . 283
8.4 Carnot refrigerator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
8.5 Realistic model for Stirling Engine . . . . . . . . . . . . . . . . . . . . . . 287
8.6 Additional aspects of heat engines and thermodynamics (not
releavnt for exam) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
8.6.1 Path dependence of Heat and Work . . . . . . . . . . . . . . . . 292
8.6.2 Heat and work at constant temperature and constant
pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
8.6.3 Chemical Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
8.6.4 Joule-Thompson effect . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
8.7 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
8.9 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297

9 Chemical potential and Gibbs Distribution . . . . . . . . . . 299

9.1 Chemical potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
9.1.1 Chemical potential in the canonical system . . . . . . . . . 300
9.1.2 Thermodynamic identity for diffusive systems . . . . . . . 303
9.1.3 Chemical potential for multi-particle systems . . . . . . . . 304
9.1.4 Example: Chemical potential of the Einstein crystal . . 304
9.1.5 Example: Chemical potential of the Ideal gas . . . . . . . . 304
xvi Contents

9.1.6 Potential energy and the chemical potential . . . . . . . . . 305

9.1.7 Example: Barometric pressure formula . . . . . . . . . . . . . 307
9.1.8 Example: Batteries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.1.9 Example: Helmholtz free energy for system in potential
field . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 308
9.1.10 Example: Modeling potential gradients . . . . . . . . . . . . . 309
9.2 Phase equilibrium at constant volume . . . . . . . . . . . . . . . . . . . . 311
9.2.1 Equilibrium between an ideal gas and an ideal crystal 311
9.2.2 Vapor pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314
9.3 Gibbs Free Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
9.3.1 Definition of Gibbs free energy . . . . . . . . . . . . . . . . . . . . 316
9.3.2 Minimum Free Energy principle . . . . . . . . . . . . . . . . . . . 317
9.3.3 Gibbs free energy and the chemical potential . . . . . . . . 319
9.3.4 Gibbs free energy for a system of several species . . . . . 319
9.4 Phase coexistence at constant pressure . . . . . . . . . . . . . . . . . . . 320
9.4.1 Phase coexistence in a simple model . . . . . . . . . . . . . . . 320
9.4.2 General phase coexistence . . . . . . . . . . . . . . . . . . . . . . . . 326
9.4.3 General phase coexistence and Gibbs free energy . . . . 329
9.4.4 Clausius-Clapeyron’s equation . . . . . . . . . . . . . . . . . . . . 330
9.4.5 Example: Transition from graphite to diamond . . . . . . 333
9.4.6 Latent heat and enthalpy . . . . . . . . . . . . . . . . . . . . . . . . . 333
9.5 Van der Waals fluid model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
9.5.1 Molecular dynamics simulation of Lennard-Jones gas . 334
9.5.2 Modifications to the ideal gas law . . . . . . . . . . . . . . . . . . 337
9.5.3 Properties of the van der Waals system . . . . . . . . . . . . . 342
9.5.4 Surface tension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
9.5.5 Lattice gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
9.5.6 Stability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 349
9.6 Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
9.6.1 Ideal mixture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
9.6.2 Binary mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
9.6.3 Dilute solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
9.6.4 Example: Vapor pressure above a dilute solution . . . . . 357
9.6.5 Example: Osmotic pressure . . . . . . . . . . . . . . . . . . . . . . . 359
9.6.6 Example: Change in boiling point above a dilute
solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
9.6.7 Example: Henry’s law and the solubility of gases . . . . . 361
9.6.8 Non-ideal mixtures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
 xvii

9.6.9 Surface growth processes . . . . . . . . . . . . . . . . . . . . . . . . . 361

9.7 Chemical reactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
9.7.1 Chemical equilibrium for Ideal Gas systems . . . . . . . . . 363
9.7.2 Example: Disassociation of hydrogen . . . . . . . . . . . . . . . 364
9.7.3 Example: pH and the Ionization of water . . . . . . . . . . . 365
9.7.4 Example: Kinetics, reaction rates, and catalytic
processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
9.7.5 Example: Dissolved Oxygen . . . . . . . . . . . . . . . . . . . . . . . 366
9.7.6 Example: Charge distribution of the Hemoglobin
Molecule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 366
9.8 Old Stuff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
9.8.1 Intensive, Extensive Variables, and G . . . . . . . . . . . . . . 367
9.8.2 Multi-component systems . . . . . . . . . . . . . . . . . . . . . . . . 367
9.8.3 Matlab MD script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
9.9 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
9.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
9.11 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370

10 Fermi and Bose gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371

10.1 Gibbs factor and Gibbs sum . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
10.1.1 Averages using Gibbs sum . . . . . . . . . . . . . . . . . . . . . . . . 374
10.1.2 Gibb’s sum for systems with several species of particles 375
10.1.3 Example: CO poisoning . . . . . . . . . . . . . . . . . . . . . . . . . . 376
10.1.4 Example: Diffusive equilibrium in the Einstein crystal 378
10.1.5 Example: Vacancies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 378
10.2 Fermi-Dirac and Bose-Einstein distribution . . . . . . . . . . . . . . . 378
10.2.1 Bose-Einstein distribution . . . . . . . . . . . . . . . . . . . . . . . . 382
10.2.2 Classical limit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
10.3 Fermi gases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
10.3.1 Zero temperature limit . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
10.3.2 Density of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392
10.3.3 Behavior at T = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
10.3.4 Qualitative behavior at low temperatures . . . . . . . . . . . 395
10.3.5 Quantitative description of low temperature behavior 398
10.4 Photon Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
10.4.1 Entropy of the photon gas . . . . . . . . . . . . . . . . . . . . . . . . 403
xviii Contents

10.4.2 Radiation from a black body . . . . . . . . . . . . . . . . . . . . . . 403

10.5 Phonon Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
10.6 Boson Gas and Einstein Condensation . . . . . . . . . . . . . . . . . . . 408
10.6.1 Spacing of levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
10.6.2 Approaching T = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 409

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 415
 List of Exercises

Exercise 2.1: Direction of time for a bouncing dimer . . . . . . . . . . . 28

Exercise 2.2: Time development for gas with two atom types . . . . 28
Exercise 2.3: One-way hole . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Exercise 2.4: Two-size particles and one-size hole . . . . . . . . . . . . . . 29
Exercise 2.5: Irreversible processes . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
Exercise 2.6: Bouncing point masses . . . . . . . . . . . . . . . . . . . . . . . . . 30
Exercise 2.7: Example of irreversible processes . . . . . . . . . . . . . . . . 30
Exercise 2.8: Lifting a ball . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
Exercise 2.9: Atomic modeling of mixing . . . . . . . . . . . . . . . . . . . . . 30
Exercise 3.1: Evolution of density . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Exercise 3.2: Temperature variation . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Exercise 3.3: Coarsening . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
Exercise 4.1: Deck of cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
Exercise 4.2: Dice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
Exercise 4.3: Casino in St. Petersburg . . . . . . . . . . . . . . . . . . . . . . . . 107
Exercise 4.4: Coin-flip game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
Exercise 4.5: Waiting time statistics . . . . . . . . . . . . . . . . . . . . . . . . . 108
Exercise 4.6: Poisson statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Exercise 4.7: Gamma distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
Exercise 4.8: Stirling’s formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Exercise 4.9: Conducting strip . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
Exercise 4.10: Velocity distribution in Argon gas . . . . . . . . . . . . . . 112
Exercise 6.1: Microstates and macrostates of a random walk . . . . 189
Exercise 6.2: Flipping 30 coins . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
Exercise 6.3: Stirling’s approximation . . . . . . . . . . . . . . . . . . . . . . . . 190

xix
xx List of Exercises

Exercise 6.4: Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190

Exercise 6.5: Gas in one- and two-dimensions . . . . . . . . . . . . . . . . . 191
Exercise 7.1: Numerical values for the partition function . . . . . . . . 262
Exercise 7.2: Changing the energy scale . . . . . . . . . . . . . . . . . . . . . . 263
Exercise 7.3: Thermal excitations in the Hydrogen atom in the Sun 263
Exercise 7.4: Thermal excitations of the water molecule . . . . . . . . 263
Exercise 7.5: Three-level system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Exercise 7.6: Partition function of two systems . . . . . . . . . . . . . . . . 264
Exercise 7.7: Arrhenius processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
Exercise 7.8: Energy fluctuations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
Exercise 7.9: Hydrogen gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
Exercise 7.10: One-dimensional ideal gas . . . . . . . . . . . . . . . . . . . . . 266
Exercise 7.11: Heat capacities of an ideal gas . . . . . . . . . . . . . . . . . . 267
Exercise 7.12: Compressibility of an ideal gas . . . . . . . . . . . . . . . . . 267
Exercise 7.13: Equipartion for a linear degree of freedom . . . . . . . . 267
Exercise 7.14: Speed of an oxygen molecule . . . . . . . . . . . . . . . . . . . 268
Exercise 7.15: Escape velocities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
Exercise 7.16: Informational entropy . . . . . . . . . . . . . . . . . . . . . . . . . 268
Exercise 8.1: Adiabatic process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
Exercise 8.2: Adiabatic compressibility . . . . . . . . . . . . . . . . . . . . . . . 295
Exercise 8.3: Crossing adiabatic curves . . . . . . . . . . . . . . . . . . . . . . . 295
Exercise 8.4: Work of a gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Exercise 8.5: String engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Exercise 8.6: Three-point cycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 296
Exercise 9.1: Maxwell relations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370
Exercise 9.2: Compressibility of the van der Waals gas . . . . . . . . . 370
 Introduction
1

Thermal physics is the physics of macroscopic objects — objects that

consist of many individual elements such as atoms, molecules or large
compounds. In thermal physics we bring the tools from other parts of
physics together and introduce entirely new concepts needed to address
fundemental and practical problems. We combine mechanics, electromag-
netics and quantuum mechanics to describe processes on macroscopic
scales. Thermal physics is needed to address many of the large scale
phenomena seen around you from the scales of nanometers to lightyears.
Thermal physics is about many of the questions we wonder about: How
does you blood carry oxygen? How does an ionic pump in a cell in your
body work? What determines chemical equilibria? Why is ice frozen while
water flows? How can we desalinate water, generate energy, conserve
energy, or understand the biological and geological processes responsible
for the world we live in? Thermal physics is also about limits: How much
energy can we get from heat? How does the universe evolve in the really
long run? Thermal physics is about fundamental questions: What are
the processes responsible for the complex patterns and structures in the
Earth’s crust? Why do many processes only spontaneously occur in one
direction in time? Can we build a machine that generates energy from
nothing? Why is more different?
Thermal and statistical physics are about the wonders of the world, but
as with all physics, we need tools to address and understand the wonders.
However, as you develop these tools and improve your knowledge and
insights into thermal and statistical physics, you will see that these tools
allow you to pose further questions and gain further insights. But they

1
2 1 Introduction

also allow you to be more precise with both your observations of nature
and in your ability to predict and utlize nature. Thermal physics contains
many of the elements needed to innovate to address some of the key
questions that face us, and the Earth, now: How can we generate, store
and distribute energy efficiently and in environmentally sustainable ways?
How can we generate clean water and air? How can we generate energy
efficient homes, cities and societies?
Thermal physics is the physics of systems of many. It may therefore be
tempting to ask what the limits of thermal physics are. What happens
when we get better at engineering systems that are very, very small. Can
we then escape from some of the limitations of statistical physics? Is
the physics of nano-scale systems fundamentally different from that of
macroscopic systems? Is there a need for new physical concepts on the
meso-scale, on the scale between individual atoms and molecules and the
thermal physics scale of many atoms? These are questions and concepts
that are still under development, as we develop better computational
and experimental tools to address systems with thousands to billions of
atoms.
Many of the key concepts of thermal and statistical physics can also
be applied in completely different fields, but this should be done with
care and caution. If you have a background in statistics you may indeed
recoqnize many of the concepts introduced in this text, such as the strong
similarity between the partition function and the likelihood function.
Indeed, concepts from statistical physics, and particular algorithmic
modeling approaches, are being applied across fields, n areas such as life
science, epidemology, economics, sociology, finance, and marketing. This
illustrates that the concepts you learn in statistical physics are useful and
powerful, and that many of their applications are yet to be discovered.
The goal of this text is to provide you with a first introduction to
key elements of thermal and statistical physics for students with no
background in thermodynamics or statistical physics. The approach of
this book is to start from the microscopic and develop the thermodynamic
concepts from there. This means that we will start from an atomic
picture, similar to what you have already learned to master in your basic
courses in mechanics and quantuum mechanics and move slowly towards
thermodynamics. I believe this coupling between the microscopic and
the macroscopic to be conceptually important, indeed one of the great
achievements of physics, but it is fully possible to apply thermodynamic
principles without understanding the underlying microscopic foundations.
When we have motivated thermodynamics from statistical physics, we will
1 Introduction 3

continue to apply and use thermodynamic concepts to address processes

without always reverting to a microscopic picture. If you are primarily
interested in the applictions, you will therefore have to bear with me
through the first, conceptual introduction.

1.0.1 Thermodynamics
Thermal and statistical physics is the study of macroscopic objects
with the aim of understanding and predicting their behavior from the
microscopic interactions between the particles making up the system.
In our study of thermal physics we will discover new laws that extend
beyond the laws of energy conservation or Newton’s laws that we already
know from mechanics, but these new laws are only valid for systems
with many particles. A wonderful result in statistical physics is that
many collective properties of macroscopic systems do not depend on the
microscopic details. For example, all liquids have common properties
that do not depend on the details of the interatomic interactions of the
atoms or molecules making up the liquid. The behavior is the same if
the liquid is made of metal, glass, water, polymers (macromolecules) or
liquid hydrogen! There are common properties of matter that simply
depends on the statistical behavior of systems of many particles, and
thermal and statistical physics is the study of these properties. Thermal
physics and thermodynamics contain the laws describing macroscopic
objects and statistical physics is the theoretical framework that allows
us to derive the laws of thermodynamics from our understanding of the
underlying microscopic interactions.
In thermodynamics we introduce new physical concepts such as tem-
perature, thermal energy (heat), and heat capacity. We introduce the
concepts needed to discuss thermodynamic processes and thermodynamic
equilibrium. We introduce the basic notions needed to characterize the
equilibrium chemical processes such as free energies and chemical po-
tentials and use these concepts also to discuss phase transitions and
mixtures, including solutions.
Macroscopic objects have their own laws and behaviors which we must
understand, learn, and preferrably develop a theoretical foundation for.
This is the aim of thermodynamics. Thermodynamics can be developed
as an axiomatic system, based on a few basic laws, but it can also be
developed from a microscopic model of a system — using statistical
mechanics.
4 1 Introduction

1.0.2 The great success of statistical mechanics

Statistical mechanics is the theory that allows us to demonstrate the
validity of thermodynamics and allows us to calculate many of the
fundamental thermodynamic properties of a system. For example, we
will later show how we can find the ideal gas law from the quantuum
mechanical behavior of a particle in a box. Indeed, statistical mechanics
is a great success in physics from both a conceptual and a practical point
of view. It gives us insight into the laws of thermodynamics, since we can
derive them, and it gives us tools to calculate real material properties
from first principles. Statistical mechanics also gives insight into why
some of the microscopic details are exactly that — details that do not
matter — and why some details are important. In this text you, we start
from a microscopic picture and derive the laws of thermodynamics before
we apply and extend them.

1.0.3 Integrated numerical methods

This text grew from a need to have a text that integrates the use of
numerical methods into the exposition, examples and exercises of a
course in thermal physics. Computations have always been at the center
of physics. It can indeed be argued that Newton invented calculus in order
to calculate motion. But with the advent of computers and with their
rapid growth in computational power, computating is now an integrated
part of all aspects of physics, and it is also central to how physical
is developed and applied in research and industry. However, in most
textbooks, only analytical tools are introduced and taught. In this text,
I have attempted to fully integrate the use of computational tools. This
provides us with some advantages, since we are not limited to problems
that can be solved analytically, but it does add new challenges as new
computational methods need to be learned and mastered.
Integration of computational methods allows us to develop examples
and exercises that more closely follow a work-flow similar to that used in
research and industry, and it allows us to use realistic data for analysis.
It will also teach you computational skills that we expect to be useful
later.
The basic philosophy of this text is that the students should implement
and run simulations themselves instead of using packages or applets that
can be tweaked. We have found that this provides students with useful
tools and insight, but also that it is very fun and inspiring to be able
1 Introduction 5

to generate realistic physical simulations yourself and to visualize and

understand your own data. Again, this comes at a cost, since you will not
be able to develop as advanced code as a commercial software package can
provide, but we believe that the satisfaction you gain by implementing it
yourself is more important and enjoyable.
We have also integrated the use of symbolic calculations in python
in both the exposition and the examples. This is to introduce you to a
useful tool in its natural context — this is how physicits and engineers
actually solve problems. We use the symbolic toolbox in python in order
to keek everything within a single programming framework, even if we
know that for example Mathematica provides a better set of tools for
many symbolic problems.
This text is part of a series of texts, where each text introduces key
computational concepts relevant for the particular type of physics intro-
duced in the text. For thermal and statistical physics, we will introduce
new concepts in algorithmic and stochastic modelling, molecular dy-
namics modeling, statistical descriptions of large data sets, optimization
methods needs for minimization of free energies, and continuum modeling
methods needed for kinetic theory and transport.

1.0.4 Molecular dynamics modeling

A particular focus of this text, will be the introduction of molecular
dynamics (MD) modeling. Molecular dynamics is the integration of
the trajectories of many interacting atoms forward in time. There are
now many commercial and open source tools that allow you to use
such simulations to measure thermodynamic properties of many types
of systems, and molecular dynamics may therefore be a tool that is
practically useful for you in your future professional career. In addition,
it is a tool that allows us to observe the collective effects and to measure
the statistical properties of systems of many atoms, and hence molecular
dynamics also provide a powerful method to gain intuition into the
concepts in statistical and thermal physics. Molecular dynamics is also a
tool that is used to understand basic atomic and molecular processes in
chemistry, material science, and life science. With the advent of ever more
powerful techniques that combine quantuum mechanical calculations with
molecular dynamics simulations of many atoms, simulations tools are
starting to become real predicitive tools that can be used for calculations
of material properties, rates of chemical processes, or the interaction of
6 1 Introduction

chemical compounds with biologically active molecules or cell machinery.

Indeed, molecular dynamics simulations is now a routine part of research
and development in areas such as the development of novel materials,
catalysis in chemical processing or in pharmaceutical research.
We have chosen to use the open source tool LAMMPS1 to integrate
the motion of atoms. However, our focus is on the primary principles
and on the analysis of data from simulations in order to gain a firm
understanding of the key concepts. It is therefore simple to replace this
with another tool of your choice.

1.0.5 Learning Physics

The typical advice you get on how to learn physics is usually based on
solid, pedagogical research. You learn physics by doing physics, by trying
to apply the concepts introduced in new situations. This is why most
textbooks in physics contains many worked examples and exercises: You
learn best by first reading the exposition, working your way through a
worked example, then work on the shorted exercises before you start
working on larger projects. This book therefore contains these three
elements:
Worked examples: Worked examples are used to motivate the theory,
and are then integrated into the text, or are provided as examples as
to how to solve an exercise or a problem. They typically contain both
analytical, symbolic and computational methods.
Exercises: Short problems that adress a key concept.
Projects: Long problems that combine all the key concepts introduced
in a chapter into a coherent story. These problems will typically contain
both analytical, symbolic, and computational approaches and various
levels of refinement of a theory.

1.0.6 Prerequisites
An introductory course in thermal and statistical physics typically follows
courses in mechanics, electromagnetism, and quantuum physics. However,
this text does not require any significant knowledge of electromagnetism
— the few concepts needed, such as for electromagnetic waves, will be
1
http://lammps.org
1 Introduction 7

introduced when needed. Basic concept from mechanics such as Newton’s

laws, mechanical energy and work is used throughout the text.
A few key concepts from quantuum mechanics is needed, but we will
not need to solve the Schrodinger equation. The concepts needed from
quantuum mechanics is the quantization of energy states for a particle in
a box (used for an ideal gas), for a two-state system, and for an harmonic
oscillator. In order to calculate the rotational behavior of a diatomic
gas, you also need to apply the formula for the rotational energy states.
However, in all these cases no derivations are needed, we only apply a
given formula for the quantization of states. In addition, we need the
concepts of Fermions and Bosons to describe Fermi and Bose gases, but
the only concept we use is that two Fermions cannot be in the same
quanuum mechanical state.
It is not necessary to have an introductory programming course in
order to follow this text, but it may be an advantage to have some
experience in scripting in either Python or Matlab.

1.0.7 Structure of this book

1.0.8 Key questions
In our studies of thermal and statistical physics, we will ask and answer
many fundamental as well as practical questions:
• What is temperature?
• Can a machine run forever?
• Why does a movie look strange when played backwards?
• How efficient can an engine or a heat-exchanger work?
• Can we generate electricity by lowering the temperature of the sea?
• How much energy does it take to boil a liter of water?
• Why does water droplets form on the outside of may cold glass of
water?
• Why do clouds form?
These and many other questions will be addressed in this text. Stay with
us!
 The Arrow of Time
2

Abstract In this chapter we introduce the notion of the arrow in time

and provide the first insights into the difference between a system with
few and many particle. We introduce heterogeity in the distribution of
energy in simple mechanical models, such as a bouncing ball and in a
monatomic gas. For the monatomic gas, we introduce two modelling
approaches that will be used frequently throughout the book: Molecular
dynamics modeling and algorithmic modeling. We show how a system
starting outside equilibrium approaches equilibrium irreversibly, relating
the behavior of many atoms to the arrow of time.
Thermal and statistical physics is about the behavior of systems of
many particles. But in what way is a system of many particles different
from that of a few particles. Mainly in the number of particles! There are
concepts and effects that only make sense in systems with many particles.
Let us now try to gain intuition into what the effects of many particles
are.

2.1 One and many particles

Let us start exploring from a simple macroscopic system consisting of

either a single particle or subdivided into many particlesd. A system you
are well aquinted with from mechanics is that of a bouncing ball. Does
the arrow of time appear in the dynamics of a bouncing ball?

9
10 2 The Arrow of Time

2.1.1 Reversible bouncing of a solid ball

Fig. 2.1b illustrates a classical problem in mechanics — a single ball
bouncing on an elastic floor. The ball is affected by two potentials:
Gravity UG = mgz, and a linear spring modeling the interaction with
the floor, UF = −(1/2)kz 2 when z < 0 and 0 when z > 0. The figure
illustrates the motion of the center of mass of this ball. This curve is
completely symmetric in time — if we reverse time, t → −t, the curve
would look the same. You would not be able to see if we play a movie
of the ball backward or forward. This is because the motion of this
individual particle is reversible in time. You can see this directly from
Newton’s second law:
d2 r d2 r
m = −∇U (r) = m . (2.1)
dt2 d(−t)2
We get the same equation if we replace t with −t. Indeed, this is also
obvious from the conservation of energy, which gives
1 2 q
mv = U (r) ⇒ v = ± 2U (r)/m , (2.2)
2
where there are two possible signs corresponding to two possible directions
of movement.
For a single particle (affected by conservative forces) motion is re-
versible in time, and it would not look unphysical if we played a movie
of the motion backward. This will always be the case for the systems
we are looking at here — the motion of individual particles will always
be conservative and reversible. Irreversibility comes from many particles.
But how?

2.1.2 Irreversible bouncing of an elastic ball

Now, let us study our first system with many particles: Instead of mod-
eling a solid ball, we study a ball that deforms elastically consisting
of a line of masses connected with springs as shown in Fig. 2.1b. Only
the bottom-most mass interacts with the floor, just like before, and the
overall mass and the initial height is the same as before. What happens
if we drop this ball? Fig. 2.1c shows the position of the center of mass,
zcm (t) as a function of time. Hmmm. This does not look the same as in
Fig. 2.1b. The ball does not bounce back up to its original height, but
bounces around at a lower height. What has happened?
2.1 One and many particles 11

0.015

a 0.01 b
0.005


0

-0.005

-0.01
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.15

0.1
c

 0.05

0.1



0.05

0.1


0.05

0
0 2 4 6 8 10 12 14 16 18 20


Fig. 2.1 The motion of a single ball bouncing on an elastic floor shown in a plot of z(t)
a Illustration of the experiment. b Ball modelled as a point particle. c Ball modelled as
two connected particles.

The total energy in the system has not changed. The motion is con-
servative and the change in energy is negligible. However, we can divide
the energy into two parts, the energy of the center of mass and the
internal energy in the form of motion relative the center of mass and of
internal compression or elongation of the springs. Fig. 2.1c shows the
center-of-mass energy, Ecm and the internal energy ∆E = ETOT − ecm as
a function of time. We see that initially all the energy is the in the center
of mass motion, but after a short while, energy is distributed between
the center of mass motion and the internal energy.
The initial point therefore seems to be a special point, whereas config-
urations further on seems more representative. We are not surprised as
the system bounces a bit up and down as time progresses, but we would
be very surprised if the system suddenly bounced so that all the energy
ended up in the center of mass motion, and no energy was in the internal
motion, corresponding to what would happen if we played a movie of
12 2 The Arrow of Time

the motion backwards. Notice that it is not physically impossible to go

backwards. Indeed, if we reversed all the velocities after a short time,
the system would return to its initial state where all the energy is in the
center of mass motion.
It seems very unlikely that the system would move to this special state,
corresponding to the initial state. Improbable, but not impossible. And
we are now at the core of the arrow of time, motion forward in time is
not the only possibility, but the most probable possibility. And we see
that the system develops towards a state where the energy is distributed
throughout all the possible degrees of freedom in the system, and that
this state corresponds to an equilibrium. Still, all these words are vague
and without a precise meaning, but we can see the initial contours of the
difference between one particle and many particles. In systems with many
particles there are many ways to distribute the energy, and the more
probable states corresponds to a homogeneous distribution of energy.
We are now onto the new laws of physics of many particles. The
are related to likelihoods — to probabilities — and they are therefore
different than the law’s of Newton, which are absolute laws. We will start
to address the differences in a simple atomic system — an atomic gas.

Finding the motion of an elastic object: We model a one-

dimensional elastically deforming ball as a set of masses, m, con-
nected by springs of spring constant k. The springs are enumerated
i = 1, . . . , N from the bottom to the top. The bottom-most mass
also interacts with the floor. The force on particle i is then

mi ai = Fi = −mi g − k (zi+1 − zi − b) + k (zi − zi−1 ) − Fw , (2.3)

where Fw = −kw zi when for i = 1 when zi < 0 and 0 otherwise.

The motion is solved using the following program:
from pylab import *
N = 10 # Nr of elements
k = 100000
kw = 1000000 # Spring constants
d = 0.1 # Size in m
m = 0.1 # Mass in kg
t = 20.0 # Simulation time
g = 9.81 # Acc of gravity
z0 = 0.1 # Initial height
dt = 0.00001 # Timestep
nstep = ceil(t/dt)
2.1 One and many particles 13

z = zeros((N,nstep),float)
vz = zeros((N,nstep),float)
z[:,0] = d*(array(range(N)))/N+z0 # Initial positions
l = d/N # Distance between nodes
for i in range(nstep): # Calculate motion
dz = diff(z[:,i])-l
F = -k*append(0.0,dz) + k*append(dz,0.0) - m*g # Internode forces
F[0] = F[0] - kw*z[0,i]*(z[0,i]<0) # Bottom wall
a = F/m
vz[:,i+1] = vz[:,i] + a*dt
z[:,i+1] = z[:,i] + vz[:,i+1]*dt;

Notice the use of vectorized math to find all the differences zi+1 − zi
at once using ’diff’. The resulting motion is shown in Fig. 2.1c.

2.1.3 More degrees of freedom

The behavior of many particles becomes even clearer in a system with
more degrees of freedom, as illustrated in Fig. 2.2, where we have simu-
lated a two-dimensional ball bouncing against a flat surface. The ball
bounces lower and lower as time progresses, but the fluctuations are now
smaller, probably because we have much more particles to distribute the
energy amongst. If we played the motion in this system backwards — if
we reversed time — it would look strange and unphysical.
Energy is conserved in this system for both forward and backward
motion. This means that energy conservation is not the only law that is
important for macroscopic behavior. We need more laws! The new laws
are related to how the energy is distributed in the object. From Fig. 2.2
we see that in the first few moments as the ball falls down, all the energy
is in one particular degree of freedom, the motion of the center of mass
of the ball. But as the ball bounces, the energy is distributed among
many degrees of freedom. We will learn more about how we describe the
distribution of energy between different degrees of freedom throughout
this book, through the introduction of the concept of entropy. However,
first we simply need to acknowledge that energy conservation is not
enough to describe the arrow of time. We need something more!
14 2 The Arrow of Time

0.95

0.9

0
0.85
z /z
0.8

0.75

0.7
0 0.5 1 1.5 2 2.5
t /T

Fig. 2.2 Illustration of a computer simulation of a bouncing ball. Energy is conserved in

the simulation, yet the ball does not bounce as high in its second bounce as its starting
point. The color scale indicates the magnitude of the velocity. We see that initially, the
whole ball has the same velocity, but after the ball has hit the ground, the distribution of
velocities is heterogeneous.

2.2 Approach to equilibrium — molecular dynamics

Let us see how much insight we can gain from the atomic hypothesis
and the motion of all the atoms in a system. Given the precise motion
of all the atoms, we should be able to understand the behavior of the
macroscopic system. However, as we shall see, it turns out not to be that
simple. Description or direct calculation does not always provide insights
by themselves. But we may be able to build understanding based on
description and calculation.
Here, we use a molecular dynamics simulation program to find the
motion of a set of atoms and then analyze the motion. You can read
more about these tools in chapter (3).
How do we determine the time development of a system of atoms? This
is a problem in quantum mechanics: The motion and interaction of atoms,
including the full motion of the electrons, needs to be determined from the
basic equations of quantum mechanics. However, it turns out that we can
learn very much without a quantuum mechanical approach. The atomic
hypothesis in itself combined with Newtonian mechanics and a simplified
description of the interactions between atoms provide valuable insights
as well as good predictions of the behavior of macroscopic systems.
2.2 Approach to equilibrium — molecular dynamics 15

How can we find the interactions between atoms? Again, we may use
quantuum mechanics to find the effective potential energy for a system of
many atoms. Here, we will start from a very simple model for atom-atom
interactions: We will assume that the energy only only depends on the
distance between pairs of atoms.
One of the simplest models for atom-atom interactions comes from a
description of the interactions between nobel-gas atoms, but is widely
used to model many types of atomic interactions. There are two main
contributions to the interaction between two noble-gas atoms:
• There is an attractive force due to a dipole-dipole interaction. The
potential for this interaction is proportional to (1/r)6 , where r is the
distance between the atoms. This interaction is called the van der
Waals interaction and we call the corresponding force the van der
Waals force.
• There is a repulsive force which is a QM effect due to the possibility
of overlapping electron orbitals as the two atoms are pushed together.
We use a power-law of the form (1/r)n to represent this interaction.
It is common to choose n = 12, which gives a good approximation to
the behavior of Argon.
The combined model is called the Lennard-Jones potential (see Fig. 2.3):
 12  6 !
σ σ
U (r) = 4 − . (2.4)
r r

The force is given as the derivative of the potential.

6
Fig. 2.3 Plot of the
4
Lennard-Jones potential
2
showing the repulsive part
U=00

0
(blue), the attractive van-
-2
der-Waals interaction (red)
-4
and the combined Lennard-
-6
Jones poential (black). 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4
r=<

We will use this potential as a basic tool to understand the behavior

of macroscopic objects. To solve the equations of motion we will use
molecular dynamics simulation package called LAMMPS, which is an
efficient package for integrating the equations of motion of the system.
16 2 The Arrow of Time

(Learn more about molecular dynamics and how to set up and analyze
simulations in Chap. 3).

2.2.1 Approach to equilibrium

Let us address how a gas approaches equilibrium by starting the system
outside equilibrium and studying the approach to equilibrium in detail.
We want to model a gas of atoms: a system where the atoms are rather
far apart from each other. (We will be more specific later). We start with
a box for the gas and divide the box into two equal parts. We fill one
part of the box with atoms and keep the other part empty. The atoms
start with random velocities. (But with zero total momentum). From
t = 0 we allow the system to develop in time.
We run this simulation for 2000 timesteps in LAMMPS using
Terminal

lammps < in.gastwosection01.lmp

where the input file is in.gastwosection01.lmp1

# 2d Lennard-Jones gas
units lj
dimension 2
atom_style atomic
lattice hex 0.25
region mybox block 0 20 0 10 -0.5 0.5
create_box 1 mybox
region 2 block 0 10 0 10 -0.5 0.05
create_atoms 1 region 2
mass 1 1.0
velocity all create 0.5 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nve
dump 1 all custom 10 twosec01.lammpstrj id type x y z vx vy vz
restart 2000 mydata.restart01 mydata.restart011
thermo 100
run 2000

The initial setup is shown in Fig. 2.4a. After 2000 timesteps (Fig. 2.4b)
the system has expanded to fill almost the whole area, but we can still see
that there are more atoms on the left side and the atoms still appear to
1
http://folk.uio.no/malthe/fys2160/in.gastwosection01.lmp
2.2 Approach to equilibrium — molecular dynamics 17

be lumped together inhomogeneously. The resulting dynamics is shown

in the movie of the simulation in gastwosection01.mp42 .

Fig. 2.4 Snap-shot from VMD of initial configuration of simulation of gas system (left)
and after 2000 timesteps (right).

What if we ran the simulation backwards in time, starting from the

state after 2000 timestep? The resulting behavior can be seen by running
the simulator using in.gastwosection11.lmp3 , which results in the movie
shown in gastwosection11.mp44 .
# 2d Lennard-Jones gas
read_restart mydata.restart01
# Reverse all velocities
variable vx atom -vx
variable vy atom -vy
velocity all set v_vx v_vy NULL
fix all nve
dump 1 all custom 10 twosec11.lammpstrj id type x y z vx vy vz
restart 2000 mydata.restart11 mydata.restart111
thermo 100 # Output thermodyn variables every 100 timesteps
run 2000 # Number of timesteps

It almost gets back to where it started, with some small discrepancies

due to numerical errors. The resulting behavior seen in the movie is
clearly very different from the behavior we would expect. Indeed the
behavior seems unphysical: We would not expect the atoms suddenly to
organize themselves so that they all are on one side. Instead, we would
expect the atoms to continue to be well mixed, and this is also what
we would see if we continued to run the simulation forward in time. If
we run the simulation for 10000 timesteps starting from the ordered
2
http://folk.uio.no/malthe/fys2160/gastwosection01.mp4
3
http://folk.uio.no/malthe/fys2160/in.gastwosection11.lmp
4
http://folk.uio.no/malthe/fys2160/gastwosection11.mp4
18 2 The Arrow of Time

state at t = 0, as is done by the input file gastwosection02.lmps5 and

visualized in the movie gastwosection02.mp46 , we do not observe that
the particles organize themselves regularly. Instead, we simply see that
the particles mix homogeneously, and remain mixed while they move
randomly about. Somehow, this mixed state seems more probable than
the segregated state we started at, and the development in the movie in
gastwosection02.mp47 appears to be physically reasonable. At least it is
not clearly unphysical, as for the segregating system.
How can we quantify this behavior? We can measure the number of
particles (or the relative number of particles) on the left and the right
half of the system as a function of time. Let us use n(t) for the number
of atoms on the left side. Since the number of atoms is conserved and
equal to the total number of atoms N , the number of atoms on the right
side is N − n(t). It is therefore enough to characterize the behavior of
the number of atoms on the left side of the system.
How can we measure this from the data? We can read the data from
the simulation directly into python using the pizza.py program and use
dump to read and interpret the data from the files. You can learn more
about how this works in Chap. 3. Here, I will simply use these routines
to read data from the simulation.
To start python using pizza.py you type
Terminal

python -i ~/pizza/src/pizza.py

if this is whereyou have downloaded/installed pizza.py. You should

now have access to all the essential tools. We use the following script to
measure the number of atoms on the left side of the system:
#start1
from pylab import *
d = dump("twosec02.lammpstrj") # Read output states
t = d.time()
nt = size(t)
nleft = zeros(n,float) # Store number of particles
# Get information about simulation box
tmp_time,box,atoms,bonds,tris,lines = d.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir
for it in range(nt):
xit = array(d.vecs(t[it],"x"))
# Find list of all atoms in left half
5
http://folk.uio.no/malthe/fys2160/gastwosection02.lmps
6
http://folk.uio.no/malthe/fys2160/gastwosection02.mp4
7
http://folk.uio.no/malthe/fys2160/gastwosection02.mp4
2.2 Approach to equilibrium — molecular dynamics 19

jj = find(xit<halfsize)
nleft[it] = size(jj)
plot(t,nleft), xlabel(’t’), ylabel(’n’), show()
#end1
np.savetxt(’tmp.d’, (t,Npart[0,:]))

and the resulting plot that shows the number of particle on the left
is shown in Fig. 2.5. Now, this is interesting. We see that the system
clearly moves in a particular direction. But it is also clear that we
need more data to be really sure about what is happening. Let us
run the simulation for 50000 timesteps and replot the data. (Using
in.gastwosection20.lmps8 and nrleft02.py9 ). The resulting plot in Fig. 2.5
now much more convincingly shows what happens: We see that n(t)
converges towards an approximately constant value and then fluctuates
around this value.

Fig. 2.5 Plot of the number 250

of atoms on the left half 200

(n) as a function of time (t)
for 10000 timesteps (top) 150
n

and for 50000 timesteps

100
(bottom)
50
0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000
t

250

200

150
n

100

50
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
t x 10
4

From this plot it is also clear that time is not fully reversible. It looks
like we might move back and forth a bit in time, but if we go too far
back we clearly end up in a situation that is different – all the atoms
suddenly segrate. This seems unphysical. It is clearly not impossible –
since it did indeed happen and would happen if we were able to run the
system perfectly backwards in time. (You can achieve this numerically
by having a sufficiently small time-step in your simulation). However, it
seems to be unlikely, in the sense that even a tiny change in the positions
or velocities, such as that generated by rounding errors, would mean
8
http://folk.uio.no/malthe/fys2160/in.gastwosection20.lmps
9
http://folk.uio.no/malthe/fys2160/nrleft02.py
20 2 The Arrow of Time

that we would not be able to reverse the simulation. If you take the
50000 timestep simulation and reverse it, you will not get back to the
segregated system. Instead, the system will fluctuate around the average
value with no large excursion.
This arrow of time is central to our understanding of macrosopic
systems. And it is clearly a new law — a law in addition to conservation
of momentum or conservation of energy.

2.3 Approach to equilibrium — algorithmic model

The atomic model directly illustrates the arrow of time, and we can
use molecular dynamics to directly measure the approach towards an
equilibrium situation: From the trajectories of all the atoms we can
measure a macroscopic quantity such as n(t). But we can gain even
more insight into the process by trying the extract the most important
part of the process — the part of the process that really introduced
the arrow of time — and leave out all the unnecessary details. We
do this by introducing a simplified model that only contains the most
important feature of the process — we introduce an algorithmic model
of the process. This method of defining a model which is as simple as
possible is a common method used in statistical physics, which we will
return to many times throughout this book.
How would such as simplified, algorithmic look like? What is the most
important features of the process? It is usually not easy to know what
the most essential feature are, and often you would need to experiment
and test many different model before you arrive at the simplest (or at
least a very simple) model that reproduces the main effects. However,
here we will simply pick the right model immediately, unfairly giving you
the impression that this is simple in practice. It is not.

2.3.1 Model of atoms on the left side

Let us try to analyze the core features of the system: What is the system?
It is a closed box with an imaginary wall in the middle. Initially, there
are N atoms on one side of the wall. On the other side of the wall there
are no atoms. As the atoms move about they get mixed, so that the
distribution of atoms becomes more homogeneous after some time.
2.3 Approach to equilibrium — algorithmic model 21

We could make this system more idealized by introducing an actual

wall in the middle of the system. A wall with a small hole. This would also
slow down the equilibration process, since it would take some time for the
atoms to hit the hole and thus move to the other side to homogenize the
distribution of atoms. However, we expect this system to retain many of
the features of the original system: All the atoms starts on one side and
as time progresses we expect the system to become more homogeneous.
We could also model this system using molecular dynamics. But it would
take a long time to find its behavior, since it would take a long time
for an atom to actually hit the hole. We would spend a lot of time
modeling atomic motion as the atoms bounced around inside the box,
and we would spend little time modelling the actual process of interest,
where an atom moves from one side of the box to the other. Instead,
we should simplify the process and introduce an algorithmic model that
only captures the most important part of the process — the motion of
an atom from one box to another.

2.3.2 Simplifying the system — part 1

First, let us make a few simplifying assumptions about the system. Let
us assume that all the atoms are independent. (This turns out to be a
reasonable assumption and indeed the assumption we will use to define an
ideal gas). With this assumption we get a new picture of what happens in
the system. Each of the two boxes consists of many atoms that bounces
around independenty. This means that each atom will have a given
probability to go through the hole every time it hits the dividing wall.
Another way to formulate this is that for each atom there is a certain
probability per unit time that it will pass through the hole and move
to the other side. Since the atoms are independent and identical, this
probability must be the same for all the atoms. Now we have transformed
the problem into a rule-based model, which we can use to find the
behavior of the system:
• The system consists of N indepedent atoms seperated by a wall with
a small hole. The position of each atom is given by si , so that it atom
i is in the left box, si = 1, and if it is in the right box si = 0.
• For each atom, there is a probability p = R∆t that the atom will move
through the hole from one box to the other during a time interval ∆t.
• There are n(t) = i si atoms on the left side and N − n(t) on the
P

right side.
22 2 The Arrow of Time

This means that we can formulate a simple rule for how the system
develops in time. At a time t, the state of the system is given by the
positions si (t) of all the N atoms. In the short time interval ∆t from
t to t + ∆t, each atom i has a probability p = R∆t to move from the
box it currently is in and into the other box10 , that it, each atom has a
probability p to change its state si into the opposite state.

2.3.3 Simplifying the system — part 2

However, this model is still more complicated than necessary, because
for a small time step ∆t, we will spend a lot of time not doing anything.
Instead, we could divide time into time intervals ∆t, which corresponds
to the typical time it takes for an atom to pass through the hole. Then,
the rule would be:
• At each time interval ∆t, corresponding to the typical time it takes
for an atom to pass through the hole, we choose an atom at random
and move it to the other side.
This is a possible model, but it is still a bit too elaborate. We can
make the model even simpler and more to the point, because we are
not interested in where each atom is – we are only interested in how
many atoms there are at each side. We are only interested in n: the
number of atoms on the left side, and N − n, the number of atoms on
the right side. If we pick an atom at random an move it to the other side,
the probability that we pick an atom on the left side is n/N . (And the
probability that we pick an atom on the right side is (N − n)/N .) The
probability that we move an atom from the left to the right hand side is
therefore n/N . We therefore have arrived at a simple algorithm:
• Draw a random number r between 0 and 1.
• If r ≤ n/N we move an atom from left to right (n → n − 1), otherwise
we move an atom from right to left (n → n + 1).
• Increase time by a unit ∆t corresponding to the typical time it takes
for an atom to pass through the hole.
10
You may ask whether these probabilities always must be the same: Does the probability
for an atom to move from the left to the right have to be the same as the probability to
move from the right to the left? We will return to this question later. For now we will
assume that the system is symmetric, and that there is no difference between the two
sides.
2.3 Approach to equilibrium — algorithmic model 23

We use this algorithm to study the approach towards equilibrium. The

algorithm is implemented in the following program:
from pylab import *
N = 210 # Number of particles
nstep = 4000 # Number of steps
n = zeros(nstep)
n[0] = 210 # Initial number on left side
for i in range(1,nstep):
r = rand(1)
if (r<n[i-1]/N):
n[i] = n[i-1] - 1 # Move atom from left to right
else:
n[i] = n[i-1] + 1 # Move atom from right to left
plot(range(0,nstep),n/N)
xlabel(’t’),ylabel(’n/N’)
show()

If we run this program for N = 210, which corresponds to the number

of atoms in the molecular dynamics simulations, we get the n(t) curve
shown in Fig. 2.6.

Fig. 2.6 Plot of the number 250

of atoms on the left half 200

(n) as a function of time (t)
for 4000 timesteps of the 150
n

discrete algorithm.
100

50
0 500 1000 1500 2000 2500 3000 3500 4000
t

2.3.4 Fluctuations and the number of atoms, N

This simplified model allows us to study the approach to equilibrium and
the fluctuations at equilibrium efficiently, for example by addressing the
dependency on the number of atoms, N . What happens as we increase
the number of atoms? Based on our discussions boave, we expect the
system to become more irreversible, even if we still do not know what we
precisely mean by that. Let us therefore look at the data. We implement
a short script to plot the behavior for various values of N :
from pylab import *
N = array([16,32,64]) # Number of particles
nstep = 4000 # Number of steps
for j in range(0,size(N)):
NN = N[j]
24 2 The Arrow of Time

n = zeros(nstep)
n[0] = NN # Initial number on left side
for i in range(1,nstep):
r = rand(1)
if (r<n[i-1]/NN):
n[i] = n[i-1] - 1 # Move atom from left to right
else:
n[i] = n[i-1] + 1 # Move atom from right to left
plot(range(0,nstep),n)
show()

The resulting behaviors for N = 20, 40, 60 and 80 are shown in Fig. 2.7.
We see that the fluctuations becomes smaller when N increases. This
means that the system stays closer to the equilibrium value for n —
corresponding to the average n over long times. The probability for a
large excursion becomes smaller when the number of particles N becomes
larger. This also means that when the number of particles is very small,
it is fully possible to have a fluctuation where all the particles are on the
left side (n = 1), but as N becomes larger, this becomes more and more
unlikely. This illustrates the principle of the physical law we are exploring
here: A behavoir far from the average becomes less and less probable
— not impossible, only improbable — as the number of atoms increase.
The physical laws for macroscopic systems are therefore a consequence of
the many number of atoms in a typical macroscopic system. In a liter of
gas (at room temperature and atmospheric pressure) there are typicall
1023 gas molecules. A very large number. And it is the largeness of this
number that ensures the regularity of the physical laws. With N = 1023
the fluctuations becomes very, very small indeed.

Fig. 2.7 Plot of n(t)/N for N = 20

0.8
various system sizes N .
0.6
n

0.4

0.2

0.8 N = 40

0.6
n

0.4

0.2

0.8 N = 60

0.6
n

0.4

0.2

0.8 N = 80

0.6
n

0.4
0.2
0 50 100 150 200 250 300 350 400
t
2.3 Approach to equilibrium — algorithmic model 25

We also observe a clear transition from the initial state at t = 0 to the

equilibrium state where there are (small) fluctuations around an average
value of n. And we see that the transition time becomes longer for a
larger system. For now, this will only be an interesting observation, but
we may return to such transient effect later on.

2.3.5 Average value of n

What is the equilibrium value of n in this system? This is not a clear
question, since we have not defined what equilibrium means. However,
from our data in Fig. 2.7 it is clear that n(t) approaches some value after
a time, although it continues to fluctuate around this value. What is this
value? We could find this value as the average value of n, after an initial
transient time of 1000 steps, using the function average,
>> average(n[1000:])
0.50569523809523831

This is very close to n(t)/N = 1/2, which is what we would expect

theoretically. Why? Because there is nothing special about the left hand
side compared to the right hand side: They are equally large boxes
and otherwise similar. We therefore expect the number of atoms on
the left and on the right hand side to be equal, on average, and hence
n/N = (N − n)/N , which gives n/N = 1/2. In practice, we are not going
to measure exacly the same number of atoms on each side, since there
are small fluctuations.
Test your understanding: What do you think will happen if we gradually increase
N ? Make a short program to plot the standard deviation of n(t) as a function of N and
comment on the results.

2.3.6 Dependence on initial conditions

Fig. 2.7 shows that the system approaches an equilibrium state after a
short transition time, where the transition time appears to increase with
the number of atoms, N , in the system. After this transition time, the
system appears to have stabilized, with small fluctuations around an
average value, hni/N = 1/2. During the short transition time, the initial
conditions influence the state of the system: The value of n(t) depends
on the initial value, n(0), over some interval τ0 . This is also illustrated in
Fig. 2.8, which shows the behavior for three different initial conditions,
n(0) = 1, n(0) = 1/2 and n(0) = 0.
26 2 The Arrow of Time

Fig. 2.8 Plot of n(t)/N for 1

n(0)/N=1
various initial conditions 0.8 n(0)/N=1/2
n(0)/N=0
n(0): n(0) = 1 (blue), 0.6

n=N
n(0) = 1/2 (red), n(0) =
0.4
1/2 (green).
0.2

0
0 100 200 300 400 500 600 700 800 900 1000
t

We could call this transition time the memory extent for the system:
The system remembers its initial conditions for some time interval τ0 ,
and after this time, it has reached the equilibrium state. We expect that
the behavior of the system does not depend on the initial state after this
transition time. This means that when the system reaches its equilibrium
state, the behavior does no longer depend on the initial condition — the
behavior is instead general, depending on the properties of the system
when it is in the equilibrium state.

2.4 Approach to equilibrium — theoretical model

From these models we observe that macroscopic quantities, such as n,

may become independent of time and of the initial state: If only the
number of particles N is large enough the fluctuations will be very small
and the quantity will be approximately independent of time. In the
example above, we found that the gas system approaches n(t) = 1/2
independently of the initial condition of the gas. However, this prompts us
to return to our main question: Why do these systems have a particular
direction of time? In what way is the situation where all the atoms are on
one side (n(0)/N = 1) different from the states with n/N ' 1/2 observed
later, when the system has reached what we call equilibrium?
First, let us introduce a way to describe the state of the system. We use
the term microstate to describe a particular arrangement of the atoms.
This could be the positions of all the atoms, ri , or we could simplify it
a bit further, and only specify if an atom is on the left or on the right
side of the box. We can denote this by introducing si = 1 when atom i
is on the left side, and si = 0 when atom i is on the right side. We can
describe the microstate of the system by listing all the si for all the i
atoms in the system. Using this notation we can write a microstate as

(s1 , s2 , s3 , . . . , sN ) , (2.5)
2.5 Summary 27

For the N = 2 system there are 4 possible microstates:

Microstate n Description
(1, 1) 2 (atom 1 on the left, atom 2 on the left)
(1, 0) 1 (atom 1 on the left, atom 2 on the right)
(0, 1) 1 (atom 1 on the right, atom 2 on the left)
(1, 1) 0 (atom 1 on the right, atom 2 on the right)
Of these microstates, there is one microstate with n = 2, one microstate
with n = 0 and two microstates with n = 1.
Now, if we assume that the system cycles through all the microstates
and spends the same amount of time in each microstate, we would
assume that the probability for each of the microstates are the same.
Each microstate is equally probable. (This is a fundamental assumption.
We will return with more arguments as to why this is true later).
This assumption would mean that the system will take the value
n = 2, n = 1, n = 1, and n = 0 with the same frequency and the same
probability, which will mean that the value n = 1 is twice as likely as
the states n = 2 and n = 0. For the N = 2 the difference between the
average value n = 1 and the initial condition where all atoms are on one
side, n = 2 or n = 0, is not that large. But as N increases we will see that
the state with n = N becomes increasingly unlikely. For large N almost
all the states occur near n = N/2 and there are very few states with
n = N or n = 0. That is what we are going to mean by equilibrium: It is
the most probably state. But not the most probably microstate, because
any microstate is equally probable, but the most probably macrostate:
The most probably value of n in this case. We will dive deeper into these
arguments and these distinctions in the next chapters.
(You may already now realize that the probability distribution for n is
given by the binomial formula, P (n, N ) = N !/(n!(N − n)!)(1/2)N , which
is very sharp around n = N/2 as N becomes large. We will address these
issues in detail in the following chapters).

2.5 Summary

• On an atomic level all laws are reversible in time.

• On a macroscopic level, time has a specific direction.
• Energy conservation is not a sufficient principle. We need new princi-
ples describing the distribution of energy.
28 2 The Arrow of Time

• We can understand the direction of time as a development towards a

most likely state in the system.

2.6 Exercises

2.6.1 Discussion questions

Exercise 2.1: Direction of time for a bouncing dimer
Fig. 2.9 shows a bouncing dimer (N = 2). However, in one of the
figures the time axis has been reversed. Which figure shows the correct
progression of time and what arguments can you use to find out?

Fig. 2.9 Plot zcm (t) for a 0.14

bouncing dimer, that is a 0.12

mass-spring system with 0.1

two (N = 2) masses. Which

zcm

0.08

figure shows the correct 0.06

progression of time? 0.04

0.02
0 2 4 6 8 10 12 14 16 18
t=t0

0.14

0.12

0.1
zcm

0.08

0.06

0.04

0.02
0 2 4 6 8 10 12 14 16 18
t=t0

Exercise 2.2: Time development for gas with two atom types
A system consists of two types of atoms, A and B, in gas form in a
rectangular box. Initially, all the A atoms are on the left side of the
box and all the B atoms are on the right side. Which of the figures in
Fig. 2.10 represent realistic time developments of the system? Explain
your answer.
2.6 Exercises 29

Fig. 2.10 Plot nA (t) (blue) 1

(a)

and nB (t) (red) for a two-

n(t)=N
0.5
atom system. However,
only one of the figures are 01
from a realistic simulation. (b)

n(t)=N
0.5

10
(c)

n(t)=N
0.5

01
(d)

n(t)=N
0.5

0
0 100 200 300 400 500 600 700 800 900 1000
t

Exercise 2.3: One-way hole

A box consists of atoms of one type. Initially, all the atoms are on the left
side. There is a hole separating the left from the right side and the atoms
can only progress from the left to the right through this hole. Sketch the
time development of n(t). Does this behavior seem realistic to you?

Exercise 2.4: Two-size particles and one-size hole

A box consists of two atoms, A and B, where atom A is smaller than
atom B. Initially, the atoms are uniformly distributed throughout the
box. Suddenly, we place a sieve in the middle of the box, so that only
the A atoms may pass through the sieve, but not the B atoms. Sketch
the time development of the number of particles nA (t) (nB (t)) of type A
(type B) on the left side as a function of time.

Exercise 2.5: Irreversible processes

List a few processes from your kitchen that clearly are not reversible and
explain why you think the processes are irreversible.
30 2 The Arrow of Time

2.6.2 Problems
Exercise 2.6: Bouncing point masses
Discuss the difference between one bouncing point mass and a system
consisting of two bouncing point-particles connected by a spring. Calcu-
late the time development for a one-dimensional system and comment
on the behavior.

Exercise 2.7: Example of irreversible processes

List more examples of processes that only can go in one direction. Try
to articulate why the reverse process does not happen spontaneously.

Exercise 2.8: Lifting a ball

If you take a ball from a table and place it on the floor – can this process
spontaneously occur in reverse? If you take a ball from the floor and
place it on the table – can this process spontaneously occur in reverse?
Explain.

Exercise 2.9: Atomic modeling of mixing

We start a molecular dynamics simulation from a gas state where all the
atoms are distributed uniformly in space with random initial velocities.
Color the left half red and the right half blue. What is the time develop-
ment of the number of red particles in the left side? Will you ever return
to the starting position?
 Getting started with Molecular
Dynamics modeling 3

Abstract In this chapter we provide a quick introduction to molecular

dynamics modeling. In molecular dynamics the motion of a set of atoms is
determined from a model for the inter-atom interactions. We demonstrate
the basic physical formulation for a Lennard-Jones model for a gas and
provide a python implementation of the molecular dynamics algorithm.
This python implementation is too slow for any practical application,
and we therefore introduce an open-source integrator to determine the
motion of all the atoms. Installation and use of the LAMMPS simulator
is described in detail. The simulation produces a set of trajectories for
all the atoms in the model, and we also demonstrate how to read these
trajectories into python and use this data-set to characterize the behavior
of realistic systems.
Statistical mechanics allows us to go from the atomic hypotesis to
theories for macroscopic behavior. We will do this by developing strongly
simplified models of the physical systems we are interested in: the ideal
gas and the ideal crystal models. However, we can also study the behavior
of a system in detail by calculating and following the motion of each
atom in the system. In this book we will use a modeling tool called
molecular dynamics. This is a useful tool that you may indeed use in
your professional career as a researcher. Also, molecular dynamics allow
you to pose and answer many questions about both atomic scale systems
and many-particle systems — questions with non-trivial answers as well
as real research questions. The tools we introduce here are indeed the
tools that are used to understand nano-scale structures of materials, to
understand and engineer chemical processes such as catalysis, to model

31
32 3 Getting started with Molecular Dynamics modeling

and understand biological processes and to understand the interaction

between drugs and living organisms. Molecular dynamics modeling is an
important field of research and engineering, and our quick introduction
here is in no way exhaustive, but should give you a quick introduction
for the capabilities of the tool.
This chapter starts by motivating the use of classical mechanics to
understand atomic motion. We introduce the Lennard-Jones model and
standard boundary and initial conditions for atomic modeling studies.
We introduce standard integration methods and a very simple illustrative
implementation in python. A more practical approach is then introduced
using the LAMMPS software package, provide installation and simu-
lation instructions for modeling the Lennard-Jones system using this
package. We also demonstrate how to visualize and analyse the results
of a simulation. Finally, we also demonstrate how LAMMPS can be used
to model more realistic systems, such as water, biological and mineral
systems. (These finals parts are interesting, but not central to exposition
of this book, and can be skipped on a first reading).

3.1 Atomic modeling basics

How do we determine the motion of a system of atoms? This is really a

problem in quantum mechanics — the motion and interaction of atoms,
including the full motion of the electrons, needs to be determined from
the basic equations of quantum mechanics. This is fully possible for small
systems with present technology. However, to address the behavior of
macroscopic systems, we need thousand, millions, or billions of atoms
to get representative results from our calculations. However, if we want
to model thousands to millions of atoms we need a different and more
computationally efficient approach.
The use of the classical approximation to describe the motion of a
system of atoms is, fortunately, able to capture many of the features
of macroscopic systems. In this approximation, we use Newton’s laws
of motion and a well-chosen description of the forces between atoms to
find the motion of the atoms. The forces are described by the potential
energies of the atoms given their positions. The potential energy functions
can, in principle, be determined from quantum mechanical calculations
because the forces and the potential energies depend on the states of
the electrons — it is the electrons that form the bonds between atoms
and are the origin of the forces between atoms. However, in the classical
3.1 Atomic modeling basics 33

approximation we parametrize this problem: We construct simplified

models for the interactions that provide the most important features of
the interactions.

3.1.1 Lennard-Jones potential

These parametrized models can be simple or complicated. They can
include only pair interactions, three-particle interactions, or even many-
particle interactions. Here, we will primarily use simple models, since
the statistical and thermal effects we are interested in do not depend
strongly on the details of the system. One of the simplest models for
atom-atom interactions is a representation of the interactions between
nobel-gas atoms, such as between two Argon atoms. For the interaction
between two noble-gas atoms we have two main contributions:
• There is an attractive force due to a dipole-dipole interaction. The
potential for this interactions is proportional to (1/r)6 , where r is
the distance between the atoms. This interaction is called the van
der Waals interaction and we call the corresponding force the van der
Waals force.
• There is a repulsive force which is a quantum mechanical effect due to
the possibility of overlapping electron orbitals as the two atoms are
pushed together. We use a power-law of the form (1/r)n to represent
this interaction. It is common to choose n = 12, which gives a good
approximation for the behavior of Argon.

The combined model is called the Lennard-Jones potential:

 12  6 !
σ σ
U (r) = 4 − . (3.1)
r r

Here,  is a characteristic energy, which is specific for the atoms we

are modeling, and σ is a characteristic length.

The Lennard-Jones potential and the corresponding force F (r) is

illustrated in Fig. 2.3. We see that the Lennard-Jones potential reaches
its minimum when
34 3 Getting started with Molecular Dynamics modeling

d
F (r) = − U (r) = 0 , (3.2)
dr
which occurs at
  r
F (r) = 24 0 (σ/r)6 − 2 (σ/r)12 = 0 ⇒ = 21/6 . (3.3)
σ
We will use this potential to model the behavior of an Argon system.
However, the Lennard-Jones potential is often used not only as a model
for a noble gas, but as a fundamental model that reproduces behavior that
is representative for systems with many particles. Indeed, Lennard-Jones
models are often used as base building blocks in many interatomic poten-
tials, such as for the interaction between water molecules and methane
and many other systems where the interactions between molecules or
between molecules and atoms is simplified (coarse grained) into a single,
simple potential. Using the Lennard-Jones model you can model 102 to
106 atoms on your laptop and we can model 1010 -1011 atoms on large
supercomputers. However, if you are adventurous you may also model
other systems, such as water, graphene, or complex biological systems
using this or other potentials as we demonstrate in Sect. 3.5

3.1.2 Initial conditions

An atomic (molecular) dynamics simulation starts from an initial config-
uration of atoms and determines the trajectories of all the atoms. The
initial condition for such a simulation consists of all the positions, ri (t0 )
and velocities vi (t0 ) at the initial time t0 . In order to model a realistic
system, it is important to choose the initial configuration with some care.
In particular, since most potentials such as the Lennard-Jones potential
increase very rapidly as the interatomic distance r goes to zero, it is
important not to place the atoms too close to each other. We therefore
often place the atoms regularly in space, on a lattice, with initial random
velocities.
We generate a lattice by first constructing a unit cell and then copying
this unit cell to each position of a lattice to form a regular pattern of
unit cells. (The unit cell may contain more than one atom). Here, we will
use cubic unit cells. For a cubic unit cell of length b with only one atom
in each unit cell, we can place the atom at (0, 0, 0) in the unit cell and
generate a cubic lattice with distances b between the atoms by using this
cubic unit cell to build a lattice. This is illustrated for a two-dimensional
system in Fig. 3.1. Such a lattice is called a simple cubic lattice.
3.1 Atomic modeling basics 35

Fig. 3.1 a Illustration of a (a) (b)

cell(1,4) cell(2,4) cell(3,4) cell(4,4)
unit cell for a square lattice.
b A system consisting of
cell(1,3) cell(2,3) cell(3,3) cell(4,3)
4 × 4 unit cells, where each

L=4 b
of the cells are marked and
cell(1,2) cell(2,2) cell(3,2) cell(4,2)
indexed for illustration.

b b cell(1,1) cell(2,1) cell(3,1) cell(4,1)

b b

L=4 b

However, for a Lennard-Jones system we know (from other theoretical,

numerical and experimental studies) that the equilibrium crystal structure
is not a simple cubic lattice, but a face centered cubic lattice. This is
a cubic lattice, with additional atoms added on the center of each of
the faces of the cubes. The unit cell for a face centered cubic lattice is
illustrated in Fig. 3.2. We will use this as our basis for a simulation and
then select a lattice size b so that we get a given density of atoms. The
whole system will then consist of L × L × L cells, each of size b × b × b
and with 4 atoms in each cell.

Fig. 3.2 Left Illustration

of a unit cell for a face
centered cubic lattice. Unit
cell atoms illustrated in
blue and the base position
of other cells shown in red.
Right A system consisting
of 10 × 10 × 10 unit cells.

3.1.3 Boundary conditions

A typical molecular model of a liquid of Argon molecules is illustrated in
Fig. 3.3a. In this case, we have illustrated a small system of approximately
10 × 10 × 10 atom diameters in size. Below, you will learn how to set
up and simulate such systems on your laptop. Unfortunately, you will
not be able to model macroscopically large systems — neither on your
laptop nor on the largest machine in the world. A liter of gas at room
temperature typically contains about 1023 atoms, and this is simply
beyond practical computational capabilities.
But it is possible with a small system to gain some insights into how
very large, even infinite, systems behave? One of the problems with the
36 3 Getting started with Molecular Dynamics modeling

Fig. 3.3 a Visualization of a 10 × 10 × 10 simulation of an Argon liquid. b Visualization

of a 10 × 10 simulation of an Argon gas, showing the actual simulation area in the center
box, and 8 copies of the center box illustrating the principles of the periodic boundary
conditions.

10 × 10 × 10 system above is the external boundaries. But we can fool

the system into believing it is infinite by applying what we call periodic
boundary conditions. If the atoms on the left of the system do not see
emptiness to their left, but instead see the right hand side of the system,
as if the system is wrapped around a cylinder, the system will look like
it is inifinite. This is illustrated in Fig. 3.3b. This convention of periodic
boundary conditions is usually applied in all simulations in order to
avoid dealing with boundary conditions. (There may be some possibly
problematic aspects of periodic boundaries, but we will not address or
worry about these here).

3.1.4 Integrating the equations of motion

How do we determine the behavior of the system? We solve the equations
of motion. For molecular dynamics simulations we usually use an algo-
rithm called the Velocity-Verlet, which is approximately like the forward
Euler method, but it is very well suited for conservative forces. The
velocity is calculated at both time t and at intermediate times t + ∆t/2,
where ∆t is the time-step, whereas the forces are only calculated at the
full time-steps, t, t + ∆t, t + 2∆t etc. The most time-consuming part
of the calculation is the calculation of the forces. We therefore want to
limit the number of times we calculate the forces and still have as high
precision as possible in our calculations. The Velocity Verlet algorithm
ensures a good trade-off between precision in the integration algorithm
and the number of times forces are calculated.
3.2 Simple implementation of a molecular dynamics simulator 37

In the Velocity-Verlet algorithm the positions ri (t) and velocities vi (t)

of all the atoms, i = 1, . . . , N , are propagated forward in time according
to the algorithm:

vi (t + ∆t/2) = v( t) + Fi (t)/mi ∆t/2 (3.4)

ri (t + ∆t) = r( t) + vi (t + ∆t/2) (3.5)
Fi (t + ∆t) = −∇V (ri (t + ∆t)) (3.6)
vi (t + ∆t) = v( t + ∆t/2) + Fi (t + ∆t)/mi ∆t/2 , , (3.7)

This method has very good properties when it comes to energy conserva-
tion, and it does, of course, preserve momentum perfectly.

3.2 Simple implementation of a molecular dynamics

simulator

How can we implement the full simulation procedure in python? We

need to set up the initial configuration of atoms, integrate the motion
for a given number of time-steps, and then output the results to a file
that can be read by standard visualization programs. Here, we provide
a full implementation to show how a molecular dynamics simulation
is implemented. However, this is mainly for illustrational purposes so
that you can see how it is done. We will actually use a state-of-the-art
open-source simulation package to perform our simulations.

3.2.1 Non-dimensional equations of motion

However, all the quantities in a molecular dynamics simulations are
very small. It is therefore usual to introduce measurement units that
are adapted to the task. For the Lennard-Jones model we usually use
the intrinsic length and energy scale of the model as the basic units of
length and energy. This means that we measure lengths in units of σ and
energies in units of 0 . A vector r0i in the simulation is therefore related
to the real-world length ri through

ri = σ r0i ⇔ r0i = ri /σ . (3.8)

Similarly, we can introduce a Lennard-Jones time, τ = σ m/, where m

p

is the mass of the atoms, and the Lennard-Jones temperature T0 = /kB .

38 3 Getting started with Molecular Dynamics modeling

Using these notations, we can rewrite the equations of motion for the
Lennard-Jones system using the non-dimensional position and time,
r0i = ri /σ and t0 = t/τ :

d2 
6 12
 
= 24 (σ/r ) 2 (σ/r ) (3.9)
X
2
m ri 0 ij − ij r /r
ij ij ,
dt2 j

to become
d2 0 X  −6  
r = 24 r − 2r −12
r 0
/r 02
. (3.10)
d(t0 )2 i j
ij ij ij ij

Notice that this equation is general. All the specifics of the system is
now part of the characteristic length, time and energy scales σ, τ , and 0 .
For Argon σ = 0.3405µm, and  = 1.0318 · 10−2 eV, and for other atoms
you need to find the corresponding parameter values.

Quantity Equation Conversion factor Value for Argon

Length x0 = x/L0 L0 p=σ 0.3405µm
Time t0 = t/tau τ = σ m/ 2.1569 · 103 fs
Force F 0 = F/F0 F0 = mσ/τ 2 = /σ 3.0303 · 10−1
Energy E 0 = E/E0 E0 =  1.0318 · 10−2 eV
Temperature T 0 = T /T0 T0 = /kB 119.74 K

3.2.2 Initialization of system

First, we need to initialize the system by generating a lattice of atoms
with random velocities. We write a small script to generate a lattice and
write it to disk. The lattice consists of units cells of size b (measure in
units of σ, as are everything in the simulation). If a unit cell starts at r0 ,
then the 4 atoms in the units cell are at positions r0 , r0 + (b/2, b/2, 0),
r0 + (b2 , 0, b2 ), and r0 + (0, b/2, b/2). If a system consists of L × L × L
such cells it is simple to create the system: We simply loop through all
L3 positions of r0 and for each such position we add atoms at the four
positions relative to r0 . This is done using the following script, which is
explained in more detail below:
L = 5; % Lattice size
b = 2.0; % Size of unit cell (units of sigma)
v0 = 1.0; % Initial kinetic energy scale
N = 4*L^3; % Nr of atoms
r = zeros(N,3);
3.2 Simple implementation of a molecular dynamics simulator 39

v = zeros(N,3);
bvec = [0 0 0; b/2 b/2 0; b/2 0 b/2; 0 b/2 b/2];
ip = 0;
% Generate positions
for ix = 0:L-1
for iy = 0:L-1
for iz = 0:L-1
r0 = b*[ix iy iz]; % Unit cell base position
for k = 1:4
ip = ip + 1; % Add particle
r(ip,:) = r0 + bvec(k,:);
end
end
end
end
% Generate velocities
for i = 1:ip
v(i,:) = v0*randn(1,3);
end
% Output to file
writelammps(’mymdinit.lammpstrj’,L*b,L*b,L*b,r,v);

In addition, we need the following function to write the data to file:

function writelammps(filename,Lx,Ly,Lz,r,v)
%WRITELAMMPS Write data to lammps file
fp = fopen(filename,’w’);
s = size(r);
ip = s(1);
fprintf(fp,’ITEM: TIMESTEP\n’);
fprintf(fp,’0\n’);
fprintf(fp,’ITEM: NUMBER OF ATOMS\n’);
fprintf(fp,’%d\n’,ip); % Nr of atoms
fprintf(fp,’ITEM: BOX BOUNDS pp pp pp\n’);
fprintf(fp,’%f %f\n’,0.0,Lx); % box size, x
fprintf(fp,’%f %f\n’,0.0,Ly); % box size, y
fprintf(fp,’%f %f\n’,0.0,Lz); % box size, z
fprintf(fp,’ITEM: ATOMS id type x y z vx vy vz\n’);
for i = 1:ip
fprintf(fp,’%d %d %f %f %f %f %f %f \n’,...
i,1,r(i,:),v(i,:));
end
fclose(fp);
end

Notice that we use the vectors bk to describe the four positions relative
to the origin, r0 of each cell:
bvec = [0 0 0; b/2 b/2 0; b/2 0 b/2; 0 b/2 b/2];
...
r0 = b*[ix iy iz]; % Unit cell base position
for k = 1:4
ip = ip + 1; % Add particle
r(ip,:) = r0 + bvec(k,:);
40 3 Getting started with Molecular Dynamics modeling

end

Also notice that we use a variable ip to keep track of the particle index
for the atom we are currently adding to the system. This is simpler than
calculating the particle number each time. That is, we can simply use
ip = ip + 1; % Add particle

instead of the more elaborate and less transparent

ip = ix*4*L*L + iy*4*L + iz

to calculate the current particle number each time a new atom is added.
Finally, we add a random velocity vector, with a normal (Gaussian)
distribution of velocities with a standard deviation of v0 using the randm
function. This can be done using a loop or with a single, vectorized
command:
for i = 1:ip
v(i,:) = v0*randn(1,3);
end
#

or
v = v0*randn(ip,3);

Finally, the state is saved to the file mymdinit.lammpstrj. The resulting

file, which contains the complete atomic state of the system, looks like
this:
ITEM: TIMESTEP
0
ITEM: NUMBER OF ATOMS
500
ITEM: BOX BOUNDS pp pp pp
0.000000 10.000000
0.000000 10.000000
0.000000 10.000000
ITEM: ATOMS id type x y z vx vy vz
1 1 0.000000 0.000000 0.000000 -0.306633 -2.732455 1.612753
2 1 1.000000 1.000000 0.000000 0.099804 0.487968 -1.545902
3 1 1.000000 0.000000 1.000000 -0.500267 0.777696 0.028699
4 1 0.000000 1.000000 1.000000 -0.404407 -0.867741 0.626161
...

where we only have included the first four of 500 atoms. This file can
then be used as a starting point for a simulation. The output from a
simulation will have a similar format, providing the state of the atomic
system which we can then analyze in details.
3.2 Simple implementation of a molecular dynamics simulator 41

3.2.3 Integration of the equations of motion

Starting from the initial configuration, we integrate the equations of
motion to find the particle trajectories and output them to file for further
analysis. First, we write a short function to read the initial configuration
from file, and then we integrate and write results to a file.
The function readlammps1 used to read from a LAMMPS trajectory
file will be reused many times throughout this book, hence we strive to
make it sufficiently general and sophisticated. You can find the listing
and discussion of this program in the Appendix XXX Here, we simply
notice that we can call it as
[Lx,Ly,Lz,r,v]=readlammps(’mymdinit.lammpstrj’);

In order to read in the initial state. The following program integrates

the equation of motion:
[Lx,Ly,Lz,r,v]=readlammps(’mymdinit.lammpstrj’);
L = [Lx Ly Lz]; s = size(r); N = s(1);
t = 3.0; dt = 0.001; n = ceil(t/dt);
a = zeros(N,3); % Store calculated accelerations
for i = 1:n-1 % Loop over timesteps
a(:,:) = 0;
for i1 = 1:N
for i2 = i1+1:N
dr = r(i,i1,:) - r(i,i2,:);
for k = 1:3 % Periodic boundary conditions
if (dr(k)>L(k)/2) then
dr(k) = dr(k) - L(k);
end
if (dr(k)<-L(k)/2) then
dr(k) = dr(k) + L(k);
end
end
rr = dot(dr,dr);
aa = -24*(2*(1/rr)^6-(1/rr)^3)*dr/rr;
a(i1,:) = a(i1,:) + aa(1); % from i2 on i1
a(i2,:) = a(i2,:) - aa(2); % from i1 on i2
end
end
v(i+1,:,:) = v(i,:,:) + a*dt;
r(i+1,:,:) = r(i,:,:) + v(i+1,:,:)*dt;
% Periodic boundary conditions
for i1 = 1:N
for k = 1:3
if (r(i+1,i1,k)>L(k))
r(i+1,i1,k) = r(i+1,i1,k) - L(k);
end
if (r(i+1,i1,k)<0)
1
http://folk.uio.no/malthe/fys2160/readlammps.m
42 3 Getting started with Molecular Dynamics modeling

r(i+1,i1,k) = r(i+1,i1,k) + L(k);

end
end
end
end
writelammps(’mymddump.lammpstrj’,Lx,Ly,Lz,r,v);

Notice how the periodic boundary conditions are implemented. They

need to be included both when the relative distance ∆rij between two
atoms are calculated and when the positions of the atoms are updated
during the integration step.
The main idea here is to show you the structure and principles of a
molecular dynamics program, as well as how data is handled and stored
in files. This python program is not practically useful because python is
too slow, in particular for a program using many nested loops. We will
therefore not use this code, but instead use a professionally developed
open-source molecular dynamics integrator to find the time development
of the atomic system. The principles are the same, but the computational
efficiency is many orders of magnitude greater for a professional code.

3.3 Running a molecular dynamics simulation

There are several efficient packages that solves the equations of motion for
a molecular dynamics simulation. The packages allow us to model a wide
variety of systems, atoms and molecules, and are efficienty implemented
on various computing platforms, making use of modern hardware on
your laptop or desktop or state-of-the-art supercomputing fascilities. We
use a particular tool developed at Sandia National Laboratories called
LAMMPS2 .

3.3.1 Installation of LAMMPS

If you want to be able to reproduce the simulations performed here you
will need to install LAMMPS on your computer. This is very simple
if you have a Mac or an Ubuntu system — for a windows system you
will have to follow the installation instructions found at the web-site for
LAMMPS. You can find all the documentation of LAMMS here3 .
2
http://lammps.sandia.gov/
3
http://lammps.sandia.gov/
3.3 Running a molecular dynamics simulation 43

Mac installation. On a mac you should be able to install LAMMS using

Homebrew or Macports.
Using Homebrew (Homebrew4 ) LAMMPS is installed with:
Terminal

brew tap homebrew/science

brew install lammps

If you want to use the parallell implementation of LAMMPS you can

install this version using
Terminal

brew tap homebrew/science

brew install lammps --with-mpi

Using MacPorts (MacPorts5 ) LAMMPS is installed with:

Terminal

port install lammps

Ubuntu installation. You can install a recent version of LAMMPS with:

Terminal

sudo apt-get install lammps

Python interface installation. In addition you should install the module

Pizza.py which we use to read simulation data into Python, and you
need a Python installation that includes pylab. I recommend Anaconda6 ,
but the Enthought version also works fine. Download Pizza.py7 and
follow the installation instructions.

3.3.2 Starting a simluation in LAMMPS

If you have successfully installed LAMMPS, you are ready to start your
first molecular dynamics simulations. The LAMMPS simulator reads
its instructions on how to run a simulation from an input file with a
4
http://brew.sh/
5
https://www.macports.org/
6
http://continuum.io/downloads
7
http://pizza.sandia.gov/
44 3 Getting started with Molecular Dynamics modeling

specific syntax. Here, we will set up a two-dimensional simulation of a

Lennard-Jones system using the file in.myfirstmd:
# 2d Lennard-Jones gas
units lj
dimension 2
boundary p p p
atom_style atomic

lattice hex 0.75

region simbox block 0 20 0 10 -0.1 0.1
create_box 1 simbox
create_atoms 1 box

mass 1 1.0
velocity all create 2.5 87287

pair_style lj/cut 2.5

pair_coeff 1 1 1.0 1.0 2.5

neighbor 0.3 bin

neigh_modify every 20 delay 0 check no

fix 1 all nve

dump 1 all custom 10 dump.lammpstrj id type x y z vx vy vz

thermo 100
run 5000

The simulation is run from the command line in a Terminal. Notice that
the file in.myfirstmd must be in your current directory. I suggest creat-
ing a new directory for each simulation, copying the file in.myfirstmd
into the directory and modifying the file to set up your simulation, before
starting the simulation with:
Terminal

lammps < in.myfirstmd

This simulation should only take a few seconds. It produces output in

the Terminal and two files: dump.lammpstrj, which contains all the data
from the simulation, and log.lammps, which contains a copy of what was
output to your terminal. Fig. 3.4 illustrates the positions of the atoms
initially and at the end of the simulation.

The input file in.myfirstmd consists of a series of commands to be

interpreted by LAMMPS. Here, we look at what these do in detail.
3.3 Running a molecular dynamics simulation 45

Fig. 3.4 (Left) Illustration of the initial hexagonal lattice generated in the simulation.
(Right) Illustration of the final positions of the atoms after 5000 timesteps.

(You can skip this at first reading, and return when you wonder
what the parameters actually do).
# 2d Lennard-Jones gas

This line starts with a # and is a comment that is ignored by the

program.
units lj
dimension 2
boundary p p p
atom_style atomic

This block describes general features of the simulation:

The units lj command selects Lennard-Jones units, which were
introduced in Sect. 3.2.1. This means that lengths are measured
in units of σ, energies in units of 0 , time in units of τ = σ m/,
p

and temperature in terms of T0 = /kB . For Argon σ = 0.3405µm,

and  = 1.0318 · 10−2 eV. Other atomic models will have other
parameters.
The dimension command specifies the dimensionality of the
simulation: 2 or 3. Here we run a 2d simulation.
The boundary command specifies boundary conditions to be
applied. Here we have periodic boundaries in the x-, y-, and z-
directions.
The atom_style command specifies the complexity of the descrip-
tion of each atom/particle. Here, we will use the simplest description,
atomic, which is used for noble gases and coarse-grained simulation
models.
46 3 Getting started with Molecular Dynamics modeling

lattice hex 0.75

region simbox block 0 20 0 10 -0.1 0.1
create_box 1 simbox
create_atoms 1 box

This block sets up the dimensions of the 20 × 10 simulation box

and fills the box with atoms with a given packing fraction.
The lattice command generates a lattice of points. This does,
surprisingly enough, not actually generate any atoms, it only gener-
ates a set of positions in space where atoms will be generated when
we generate atoms. The type hex specifies a two-dimensional lattice
of hexagonal shape, so that each atom has six nearest neighbors.
And the number 0.75 is called the scale and is the reduced density,
ρ0 , when we have chosen LJ units for the simulation. (Notice that
the scale is interpreted differently if we do not use LJ units, see the
LAMMPS documentation for more information).
The region command defines a region which is a block extending
over 0 < x < 20, 0 < y < 10, −0.1 < z < 0.1 We give this region
the name simbox.
The create_box command now actually creates the simulation
box based on the spatial region we called simbox. The simulation
box will only contain 1 (one) type of atoms, hence the number 1.
The create_atoms finally fills the simulation box we have defined
using the lattice we have defined with atoms of type 1.
mass 1 1.0
velocity all create 2.5 87287

This block defines the material properties of the atoms and defines
their initial velocities.
The mass command defines that atoms of type 1 will have a mass
of 1.0 relative to the mass of the Lennard-Jones model. This means
that all atoms have mass 1 in the Lennard-Jones units. This means
that the masses of all the atoms are the same as the mass m used
in the non-dimensionalization of the Lennard-Jones model.
The velocity command generates random velocities (using a
Gaussian distribution) so that the initial temperature for all atom
types in the system is 2.5 in the dimensionless Lennard-Jones units.
The last, strange integer number 87287 is the seed for the random
number generator used to generate the random numbers. As long as
you do not change the seed number you will always generate same
initial distribution of velocities for this simulation.
3.3 Running a molecular dynamics simulation 47

pair_style lj/cut 2.5

pair_coeff 1 1 1.0 1.0 2.5

This block specifies the potential between the atoms.

The pair_style command specifies the we want to use a Lennard-
Jones potential with a cut-off that is of length 2.5. What does this
mean? It means that since the Lennard-Jones potential falls so
quickly to zero as the distance between the atoms increase, we will
approximate interaction to be zero when the atoms are further than
2.5 away from each other (measured in Lennard-Jones units, that
is in units of σ). The simulator ensures that both the potential
and the force is continuous across the transition. There are many
other types of force fields that you may use — take a look at the
documentation of LAMMPS for ideas and examples.
The pair_coeff command specifies the parameters of the
Lennard-Jones model. The two first numbers, 1 1, specifies that we
describe the interaction of atoms of type 1 with atoms of type 1.
And the parameters of the Lennard-Jones model are 1.0 1.0 2.5.
This means that The interaction between an atom of type 1 with an
atom of type 1 has a σ-value corresponding 1.0 times the the general
σ-value (hence the first number 1.0), and a 0 -value corresponding
to 1.0 times the overall -value (hence the second number 1.0). The
cut-off for this interaction is 2.5 — the same value as we specified
above.
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no

This block contains information about how the simulator is to calcu-

late the interaction between the atoms using list of neighbors that
are updated at regular intervals. You do not need to change these
parameters, but changing them will typically not have any effects
on the simulation results, only on the efficiency of the simulations.
fix 1 all nve

This one-line block specifies what type of simulation we are per-

forming on the atoms. This is done by one or more fix commands
that can be applied to regions of atoms. Here, the fix, which we
call 1 (you can choose numbers or names for identity), is applied to
all particles and specifies that the simulation is run at constant
nve, that is, at constant number of particles (n), constant volume
48 3 Getting started with Molecular Dynamics modeling

(v, meaning that the simulation box does not change during the
simulation), and constant energy (e). You may be surprised by the
constant energy part. Does the integration algorithm ensure that
the energy is constant. Yes, it does. However, there can be cases
where we want to add energy to a particular part of the system, and
in that case the basic interation algorithm still conserves energy,
but we add additional terms that may change the total energy of
the system.
dump 1 all custom 10 dump.lammpstrj id type x y z vx vy vz
thermo 100
run 5000

This block specifies simulation control, inclusing output and the

number of time-steps to simulate.
The dump command tells the simulator to output the state. The
1 is the name we give this dump — it could also have been given a
name such as mydump. We specify that all atoms are to be output
using a custom output format, with output every 10 time-steps
to the file dump.lammpstrj, and the ‘id type x y z vx vy vz’ list
specifies what is output per atom.
The thermo command specifies that output to the Terminal and
to the log file, log.lammps, is provided every 100 timesteps.
The run command starts the simulation and specifies that it will
run for 5000 timesteps.

3.3.3 Visualizing the results of a simulation

It is good practice to look at the results of the simulation. Use for example
VMD8 or Ovito9 to visualize the results. Here, let us demonstrate the
use of VMD. First, open VMD. Then open the File -> New Molecule
menu item. You find the dump.lammpstrj file that were generated by
your simulation run, and press load. The whole time sequence of your
simulation is now loaded in VMD. However, the default visualization
mode is usually not that clear. You fix that by going to the main menu
again, and select Graphics -> Representations... In the window that
opens you change the Drawing method to CPK and you select Bond Radis
8
http://www.ks.uiuc.edu/Research/vmd/
9
http://www.ovito.org/
3.4 Analyzing the results of a simulation 49

to be 0.0. Now, your visualization should show the time dynamics of an

ensemble of atoms. Enjoy!

3.4 Analyzing the results of a simulation

The results from the simulations can be analyzed using built-in tools
from LAMMPS. We demonstrate these tools by measuring the number
of particle on the left side of the box as a function of time, by extracting
that data from the simulation files.
We have illustrated the text-format output files from LAMMPS above.
The file may contain data from may timesteps, t. For each timestep,
there are a few initial lines describing the system, followed by a list of
properties for each atom. We have chosen the custom output form
dump 1 all custom 10 dump.lammpstrj id type x y z vx vy vz

Each line of atom properties contains data of the form id type x y z

vx vy vz. This means that we will get access to both the atom position
and the atom velocity for each of the atoms if we read the data in this
file. Our strategy will then be to use the atom position xi and compare it
with Lx /2, where Lx is the dimensions of the simulation box, measured
using the same units as used for the positions xi of the atoms. How can
we read the data from the LAMMPS dump files. We could read our own
input functions, but it is simpler to use already developed code, which is
distributed with the LAMMPS distribution.

3.4.1 Matlab implementation

For matlab, we use the tools found in the tools/matlab directory, which
was part of the lammps installation. (If you cannot find these after
installing LAMMS, you can always download LAMMS again as a tarball,
extract the tarball, and find the programs in the resulting directory).
You need to copy the file readdump_all.m and readdump_one.m to your
current working directory — the directory where you ran the simulation
and where the file dump.lammpstrj is located.
First, we read data from all the timesteps into matlab:
data = readdump_all(’dump.lammpstrj’);

We need the time values ti at each of the timesteps and the number of
time-steps, nt:
50 3 Getting started with Molecular Dynamics modeling

t = data.timestep;
nt = length(t);

We set up an array, nleft, to store n(t), the number of atoms on the

left side as a function of time:
nleft = zeros(nt,1);

The size of the simulation box is stored in the variable box, and we store
Lx /2 in the variable halfsize:
box = data.x_bound;
halfsize = 0.5*box(:,2);

Now, we loop through all the timesteps. For each timestep we extract
all the x-positions for all the atoms in a list xit. Next, we find a list
of all the atoms that are on the left side, that is, all the atoms with
an x-position smaller than Lx /2. This is done by the find command.
Finally, we count how many elements are in this list. This is the number
of atoms that are on the left side of the box. We store this number in the
array nleft for this timestep, and reloop to address the next timestep:
for it = 1:nt
xit = data.atom_data(:,3,it);
jj = find(xit<halfsize(it));
nleft(it) = length(jj);
end

The complete program reads:

%start1
data = readdump_all(’dump.lammpstrj’);
t = data.timestep;
nt = length(t);
nleft = zeros(nt,1);
box = data.x_bound;
halfsize = 0.5*box(:,2);
for it = 1:nt
xit = data.atom_data(:,3,it);
jj = find(xit<halfsize(it));
nleft(it) = length(jj);
end
plot(t,nleft), xlabel(’t’), ylabel(’n’)

A short and simple program to handle a complicated set of data. This

should provide you with a good basis for measuring various properties of
the simulated system using Python.
3.4 Analyzing the results of a simulation 51

3.4.2 Python implementation

For Python we use the tool Pizza.py. You should then simply start your
python session by calling pizza.py initially. If we assume that you have
installed the program in /pizza, you can start your ipython session
using pizza.py by typing
Terminal

ipython -i ~/pizza/src/pizza.py ~/pizza/src/pizza.py

in your terminal. The you are ready to analyze data from the simulation.
First, we read data from all the timesteps into Python:
from pylab import *
data = dump("dump.lammpstrj") # Read all timesteps

We need the time values ti at each of the timesteps and the number of
time-steps, nt:
t = data.time()
nt = size(t)

We set up an array, nleft, to store n(t), the number of atoms on the

left side as a function of time:
nleft = zeros(n,float); # Store number of particles

The size of the simulation box is found in the variable box, and we store
Lx /2 in the variable halfsize:
tmp_time,box,atoms,bonds,tris,lines = data.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir

Now, we loop through all the timesteps. For each timestep we extract
all the x-positions for all the atoms in a list xit. Next, we find a list
of all the atoms that are on the left side, that is, all the atoms with
an x-position smaller than Lx /2. This is done by the find command.
Finally, we count how many elements are in this list. This is the number
of atoms that are on the left side of the box. We store this number in the
array nleft for this timestep, and reloop to address the next timestep:
#Get information about simulation box
tmp_time,box,atoms,bonds,tris,lines = data.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir
for it in range(nt):
xit = array(data.vecs(it,"x"))
jj = find(xit<halfsize)
nleft[it] = size(jj)
52 3 Getting started with Molecular Dynamics modeling

The complete program reads:

#start1
from pylab import *
data = dump("dump.lammpstrj") # Read all timesteps
t = data.time()
nt = size(t)
nleft = zeros(n,float); # Store number of particles
#Get information about simulation box
tmp_time,box,atoms,bonds,tris,lines = data.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir
for it in range(nt):
xit = array(data.vecs(it,"x"))
jj = find(xit<halfsize)
nleft[it] = size(jj)
plot(t,nleft), xlabel(’t’), ylabel(’N_p’), show()

A short and simple program to handle a complicated set of data. This

should provide you with a good basis for measuring various properties of
the simulated system using Python.

3.4.3 Results
The resulting plot is shown in fig. 3.5. If you wonder why the number of
atoms on the left is varying, how it is varying, and how to describe how
it is varying — this indeed in the topic of this book so just read on!

Fig. 3.5 Plot of n(t), the

210
number of atoms on the
left side of the simulation 205
box, as a function of time,
200
n

where the time is measured

in the number of timesteps 195
in the molecular dynamics
190
simulation. 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
t

3.5 Advanced models

Molecular dynamics simulation packages such as LAMMPS are profes-

sional tools that are used for research and development. Here, I provide
a few examples of more advanced use of LAMMPS that may inspire you
to try the tool also on your own.
3.5 Advanced models 53

3.5.1 Coarsening
Out first example is an extension of the first two-dimensional simulation
you performed above. What happens if we start the system with a
homogeneous distribution of atoms, but with a low initial energy? In
addition, we keep the average kinetic energy in the system approximately
constant. (This corresponds, as you will learn later, approximately to
keeping the temperature in the system constant):
units lj
dimension 2
boundary p p p
atom_style atomic

lattice hex 0.5

region simbox block 0 80 0 40 -0.1 0.1
create_box 1 simbox
create_atoms 1 box

mass 1 1.0
velocity all create 0.05 87287

pair_style lj/cut 2.5

pair_coeff 1 1 1.0 1.0 2.5

neighbor 0.3 bin

neigh_modify every 20 delay 0 check no

fix 1 all nvt temp 0.25 0.25 1.0

dump 1 all atom 1000 dump.lammpstrj
thermo 100
run 50000

The resulting behavior shown in Fig. 3.6 is a phase separation: The system
separates into a liquid and a gas phase, demonstrating a phenomenon
called spinoidal decomposition. We will discuss the behavior of such
systems using both molecular dynamics models and algorithmic models
later.

Fig. 3.6 Snapshots from a simulation of a Lennard-Jones liquid, with a low initial density.
A slowening coarsening is observed.
54 3 Getting started with Molecular Dynamics modeling

3.5.2 Hemoglobin in water

3.5.3 Lipid bilayers
3.5.4 Fracture of Silicon crystal
3.5.5 Water permeation in a carbon nanotube
3.6 Summary

• Molecular dynamics simulations model how atomic systems propagate

forward in time by integrating the equations of motion based on a
given interaction potential between the atoms.
• We have introduced a commonly used interaction potential, the
Lennard-Jones potential: V (r) = 4 (σ/r)−12 − (σ/r)−6 , where the

behavior is determined by the energy scale 0 and the length scale σ.

• We run molecular dynamics simulations using custom code, public or
commercial codes, which include efficient integrators.
• A typical simulation models a system with constant energy, volume,
and number of particles.
• Molecular dynamics simulations are routinely used as part of research
and development tasks across disciplines, and are important tools
for applications such as the development of new materials, chemical
catalysis optimization and drug development.

3.7 Exercises

3.7.1 Problems
Exercise 3.1: Evolution of density
We will here study a 10 × 10 × 10 system of Argon atoms, as done above.
Initiate the system with all particles in the left 1/3 of the system. We
call the number of particles in the left, center and middle thirds of the
system n1 (t), n2 (t), and n3 (t) respectively.
a) How do you expect n1 , n2 , and n3 to develop in time?
b) Measure n1 (t), n2 (t), and n3 (t) and compare with your predictions.
3.7 Exercises 55

Exercise 3.2: Temperature variation

a) Measure the maximum and minimum temperature, Tmax and Tmin
respectively for a simulation of L = 10 × 10 × 10 atoms, and calculate
∆T = Tmax − Tmin .
b) Measure ∆T for three different system size, L = 4, L = 8, and L = 16.
Comment on the results.

Exercise 3.3: Coarsening

Start a LAMMPS simulation with the following init file:
# 3d Lennard-Jones melt
units lj
atom_style atomic
lattice fcc 0.2
region box block 0 10 0 10 0 10
create_box 1 box
create_atoms 1 box
mass 1 1.0
velocity all create 0.1 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nvt temp 0.05 0.05 1.0
dump id all atom 50 dump.lammpstrj
thermo 100
run 10000

(This simulation is performed at constant temperature and not at constant

energy). Visualize the time development of the system and comment on
what you see.
 Probability and Statistics
4

Abstract This chapter will introduce you to some of the most common
and useful tools from statistics, tools that you will need throughout
your career. We will learn about statistics motivated by an introductory
example — trying to understand the data you generated by running
a molecular dynamics simulations. This is a good starting point, since
statistics is about describing real data. The data from a realistic molecular
dynamics simulation can be quite overwhelming. The data-sets become
extremely big, so we need effective tools to pick out the most important
information. Here, we introduce the concept of probabilities, the basic
properties of probabilities, such as the rule of addition and the rule of
multiplication. We introduce probability distributions and densities. We
introduce the expected value and the variance and their estimators, the
average and the standard deviation. We introduce the biomial, normal and
exponential distribution. We demonstrate that the distribution of a sum
of independent variables usually are normally distributed, independently
of the details of the distribution of each event in the sum. Finally, we
use these tools to develop a simple theory for the number of particles on
the left side of a gas.
Statistics provides us with the tools to describe real data, whereas
probability theory provides us with the theoretical underpinning for
statistics. Here, we will start with a practical approach to probability
theory, based on the idea of frequencies of event, and use this to develop
the fundamental laws of probabilities. This approach suits a physicists
approach well and is also the most direct approach from a computational
perspective. Again, we will introduce both theory and the computational

57
58 4 Probability and Statistics

tools needed to test the theory and make accurate measurements also
for large data sets.
Probability theory is a classic field that has occupied many of the
great mathematicians and physicists throughout history. It has been
developed along with gambling, but this is just an unfortunate association.
Probability theory is the tool we need to address a world of real data
and unknown processes, and the tool we need to describe the behavior
of systems with many particles, where the physical laws no longer are
absolute, but only represent the overwhelmingly most probable outcome
of an experiment or an observation. However, we will start our venture
into statistical and probability motivated by observations and real data
— in our case the data from measuring the number of atoms on the left
side of a box filled with a gas.

4.1 Motivating example — an ideal gas

A molecular dynamics simulations of a gas produces fluctuations, because

the atoms are moving approximately randomly around. Here, we will use
the data from this approximately random system as a model to analyze
and understand random systems.
We introduced the measure n(t) to characterize the number of atoms
on the left side of the box — you can find this data in the datafile
ndata.d1 .
We can generate the molecular dynamics data using the same ap-
proach as in Chap. 3, but we generate more data points to have better
statistics for our analysis. The data is generated by a molecular dynamics
simulation using LAMMPS to model a two-dimensional gas system using
the following LAMMPS input file:
# 2d Lennard-Jones gas
units lj
dimension 2
atom_style atomic
lattice hex 0.10
region box block 0 20 0 10 -0.1 0.1
create_box 1 box
create_atoms 1 box
mass 1 1.0
velocity all create 2.5 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
1
http://folk.uio.no/malthe/fys2160/ndata.d
4.1 Motivating example — an ideal gas 59

neighbor 0.3 bin

neigh_modify every 20 delay 0 check no
fix 1 all nve
dump 1 all custom 10 gasstat01.lammpstrj id type x y z vx vy vz
thermo 100
run 50000

which is run by typing

Terminal

lammps < in.gasstatistics01

We then use the script we developed in Sect. 3.4 to extract n(t), the
number of atoms in the left half of the system (x < Lx /2) as a function
of time t:
from pylab import *
data = dump("gasstat01.lammpstrj") # Read output states
t = data.time()
nt = size(t)
nleft = zeros(nt,float) # Store number of particles
# Get information about simulation box
tmp_time,box,atoms,bonds,tris,lines = data.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir
for it in range(nt):
xit = array(data.vecs(it,"x"))
jj = find(xit<halfsize)
numx = size(jj)
nleft[it] = numx
plot(t,nleft), xlabel(’t’), ylabel(’n’), show()
np.savetxt(’ndata.d’,(t,nleft))

The result is n(t) for a sequence of timesteps ti , as illustrated in Fig. 4.1

for N = 210 atoms in the box. The resulting sequence ti , ni is also written
to the file ndata.d using the command dlmwrite.
Now, we can analyze the particles in the gase by reading and analyzing
the data file ndata.d. The file is loaded using
import numpy as np
t,n = np.loadtxt(’ndata.d’)

How can we understand what this data represents and what it tells
us about the gas? We will develop several measurement techniques to
characterize the data, and we will develop a model that may explain the
data and our analysis of the data. Such a model may not reproduce all
the features of the data exactly, but it should represent the main features
in a good way. What do we mean by that?
60 4 Probability and Statistics

230

220

210

200
n
190

180

170
0 1 2 3 4 5 0 200 400
t #104 N (n)
Fig. 4.1 Plot of the number of atoms on the left half (n) as a function of time (t) for 50000
timesteps for two runs with different random seeds for the initial velocity distribution.

We notice that the data has some randomness to it. If we did another
simulation, but with a different random seed for the initial velocities, we
would expect the behavior to be different — at least not identical — but
also to retain some of the features. This is illustrated in Fig. 4.1, where
two curves n(t) are illustrated. The curves have similar features, but are
clearly not identical.
What general features of n(t) should a model predict or explain? Since
the curves are not identical, our model will clearly not reproduce n(t) in
detail, but instead it should reproduce some of its general properties. For
example, we could characterize n(t) by how often it visits a particular
value of n: We count the number of times, N (x), when there are x atoms
on the left side, that is, how many times n(t) = x in a given time interval.
This is done automatically by a histogram:
histogram(m)

(We will explain this method in detail below). A plot of N (x) is shown
in Fig. 4.1, in the same figure as the data-set. This gives us a first
insight into the frequency of occurrence of the various values of n. And
we immediately observe that not all n-values occur equally frequently:
values close to n = 200 occur more frequently that values further away
from n = 200. Fortunately, there is a whole field of science dedicated to
studying frequencies of occurrence and histograms of frequencies — the
field of probability theory and statistics. This chapter therefore contains
a brief introduction to statistics and probability theory, before we develop
a model for the molecular dynamics data.
4.2 Probabilities 61

4.2 Probabilities

4.2.1 Statistical experiments

The sequence ni of atoms on the left side of the gas appears to be a
sequence of random numbers2 . In order to develop the intuition and
tools needed to describe such sequences, we will here introduce random
number and sequences of random numbers.
In statistics, we introduce the term statistical experiment to de-
scribe an event with a possible random outcome:

A statistical experiment is a trial with a number of possible

outcomes or events. The result of an experiment is called an out-
come or an event and the set of all possible outcomes is called the
sample space.

Typically, we will consider statistical experiments that are uncorre-

lated, meaning that if we know the result of the previous experiment, this
will not help us know the result of the next experiment. (However, many
physical systems, such as the gas, are correlated, because the state of
the system may not have changed completely between two subsequent
measurements). For example, throwing a fair die or flipping a fair coin
are uncorrelated experiments. The outcome of such an experiment is the
number, mi , on the die.

4.2.2 Generating a sequence of random numbers

How can we generate a random outcome from a statistical experiment —
or a sequence of outcomes? An analogue method is to throw a die and
record the number shown each time. Numerically, we use a pseudo-random
number generator3 .
We generate a random number between 1 and 6 in python by:
2
There are also important differences between measuring n(ti ) and a sequence of random
numbers such as from throwing a dice: For the measurements subsequent elements may
be correlated: If we know the value at i, the system may not have changed significantly
to the measurement i + 1. (Depending on the time between measurements). However, for
a sequence of dice throws we expect the event to the uncorrelated.
3
The random number generator only produces pseudo-random numbers based on
a deterministic algorithm that generates a sequence of apparently random numbers.
However, for all our purposes these numbers appear to be random.
62 4 Probability and Statistics

randint(0,6)

(Here, the first argument is the smallest random integer and the second
argument is the largest random integer, and an optional third argument is
the number of outcomes to produce). And we can use the same command
to generate a sequence of 4 such experiments by:
randint(0,6,4)+1

array([1, 4, 4, 2])

The result is a sequence, ni , i = 1, . . . , M of M outcomes from a numerical

stastistical experiment. How can we understand this experiment by
analyzing the outcomes?

4.2.3 Measuring the frequency of occurrence

Our first approach to characterize the results from the experiment is to
measure the frequency of occurrence of a particular outcome: We count
how many times a given value x occurs in the sequence of numbers, ni ,
by generating a histogram. A histogram records how many times, N (x)
a value x occurs in the interval from xj to xj+1 = xj + ∆x, where ∆x
is the bin width of the histogram. python has automatic functions for
finding the histogram for a data set. We generate 1000 random numbers
between 1 and 8 and find the histogram using:
n = randint(0,8,1000)+1
[Nx,edges] = histogram(n,8,range=(0.5,8.5))
print Nx

array([127, 131, 128, 135, 116, 140, 127, 96])

print edges

[ 0.5 1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5]

(Notice the range used in order to ensure there are 8 bins of size 1 each).
Here, the edges are the xj values giving the edges of the bins used. We
can find the centers of the bins and plot the results using:
x = (edges[1:]+edges[:-1])/2
print x
4.2 Probabilities 63

[ 1. 2. 3. 4. 5. 6. 7. 8.]

plot(x,Nx)

We can compare this with a traditional bar-type histogram plot using

hold(’on’)
hist(n,8,range=(0.5,8.5))
hold(’off’)
show()

The resulting plot is shown in Fig. 4.2. We see that the frequencies are
approximately, but not exactly, the same for all the outcomes. How can
we interpret the frequency of occurrence?

Fig. 4.2 Plot of the his- 150

togram of 1000 outcomes

from an experiment where 100
N (x)

we choose a random num-

ber between 1 and 8. 50

0
0 1 2 3 4 5 6 7 8 9
x

4.2.4 Frequencies and probabilities

When we analyze a statistical experiment, or data from experiments or
simulations, we will assume that an experiment reflects an underlying
process which has given properties. We estimate these properties using
various estimators. For a sequence of outcomes ni , such as from our
numerical experiment above, we can measure the frequency F (x) which
is defined as

We observe a sequence of M outcomes, ni , i = 1, . . . , M , from

a statistical experiment. The frequency F (x) is defined as how
many times, Nx , we have measured x in the sequence — that is the
number of times ni = x — divided by the number of elements, M ,
in the sequence:
Nx
F (x) = . (4.1)
M
64 4 Probability and Statistics

The frequency is an estimator for an underlying property of the process

— the probability of an outcome. We can define the probability as the
frequency when we do infinitely many experiments:

The probability, P (n = x), of an outcome x is the limit of the

frequency of the outcome when we perform infinitely many experi-
ments:
Nx
P (n = x) = lim F (x) = lim . (4.2)
M →∞ M →∞ M

This limit is not a mathematical limit as known from calculus, but

instead a limit of inifintely many experiments. We call this the
frequency definition of probability.

This is not the only possible definition of a probability. In statistics you

may meet other definitions. But it is a practical and useful definition
for our purposes. Also, it reflects the important differene between the
underlying properties of a process, given here by the probability for an
outcome, and the observation of outcomes of that process, the frequency.

4.2.5 Example: Probability of a dice

We throw a fair dice with six sides. What is the probability to throw a 1?
Numerical experiments. First, we perform a numerical statistical ex-
periment to gain intuition. We generate M throws of a die with a result
between 1 and 6 and plot the frequency, F (x) = N (x)/M , for experiments
with increasing values of M :
from pylab import *
Mval = array([10,100,1000,10000,100000])
nM = len(Mval)
for im in range(nM):
M = Mval[im]
m = randint(0,6,M)+1
[Nx,edges] = histogram(m,6,range=(0.5,6.5))
Fx = Nx/M
x = (edges[1:]+edges[:-1])/2
plot(x,Fx,’-o’)
xlabel(’x’), ylabel(’F(x)’), show()

The results are shown in Fig. 4.3.

4.2 Probabilities 65

Fig. 4.3 Plot of the fre- 0.3

M=10
quency of occurrences, M=100
0.25
F (x) for x = 1, 2, 3, 4, 5, 6 M=1000
M=10000

F (x)
for M = 10, 100, 1000, and 0.2

10000 throws.
0.15

0.1
1 1.5 2 2.5 3 3.5 4 4.5 5 5.5 6
x

We see that the frequencies, F (x) = N (x)/M converges as M in-

creases. We expect the frequencies to converge towards the underlying
probabilities, P (x). What are the values of F (x) for M = 10000?
print Nx

array([0,1613, 0.1677, 0.1638, 0.1694, 0.1718, 0,1660])

Both these results, and the histogram plots in Fig. 4.3, indicate that the
probability is a constant, the same, for all possible outcomes x. This is
what we would expect from a fair dice. Otherwise, we would know that
the dice preferred a particular outcome, and it would not be fair.
Theretical argument. We can also devise a theoretical argument for the
value of the probability P (x) for a throw of the die. From the definition
of probability, the probability to get i is
Ni
P (i) = , (4.3)
M
where M is the number of throws and i may be 1,2,3,4,5 or 6 — valid
in the limit when M becomes large. From this definition we see that
probabilities are normalized:
P
Ni M
P (i) = = =1, (4.4)
X
i

i
M M

where Ni = M because in each throw one and only one of the 6

P

possible outcomes are possible. This gives us the normalization property

of probabilities:
P (i) = 1 , (4.5)
X

when any two outcomes i cannot occur at the same time and the sum is
over all possible outcomes.
We can use this property to calculate the probability to throw a 1.
This follows from two characteristics of the experiments. First, the dice
66 4 Probability and Statistics

is fair, which means that P (i) are the same for all i, P (i) = const = p.
Second, we use the normalization condition:
6 6 6
1
P (i) = P =p =p·6=1 ⇒ p= (4.6)
X X X
,
i=1 i=1 i=1
6

This example uses a standard set of tricks that you may want to remember:
If all outcomes are equally probable we can assume they all have the
same probability p, and the probabilities are normalized.
We can also illustrate another basic principle of probabilities that also
serves as a useful trick: The probability of something not happening can
be found from the normalization condition. For example, what is the
probability of not throwing a one in a single throw?
Here, we use a trick and split all possible outcomes into P (one) and
P (not one). Since one of these two outcomes must occur, they must be
normalized, so that

P (one) + P (not one) = 1 , (4.7)

where we know that P (one) = p = 1/6. We can therefore find P (not one)
from
1 5
P (not one) = 1 − P (one) = 1 − p = 1 − = , (4.8)
6 6

4.2.6 Properties of probabilities

From this example, we have learned several new properties of probabilities
that are general:

Normalization: Probabilities are normalized so that the sum of

the probabilities of all the possible outcomes is 1:

P (x) = 1 , (4.9)
X

when any two outcomes x cannot occur at the same time and the sum
is over all possible outcomes x. This is called the normalization
condition of probabilities.
4.2 Probabilities 67

This can be proven from the frequency definition and that the total
number of observations, x N (x) must be the same as the total number
P

of experiments, M :
X N (x) 1 X M
P (x) = = N (x) = =1. (4.10)
X

x x M M x M

Inversion rule: We can always divide all the possible outcomes

of an experiment into two parts, that an outcome A occurs and
that the outcome A does not occurs. Either of these outcomes must
occur, and they cannot occur at the same time. The normalization
condition therefore gives:

P (A) + P (not A) = 1 ⇒ P (A) = 1 − P (not A) . (4.11)

Sets of outcomes. Notice that outcomes from an experiment can be

grouped in various ways. For example, when we throw a six-sided dice,
we can group the set of all possible outcomes into two set: set A = 1,2,
and set B = 3,4,5,6. These two sets are independent , meaning that
set A cannot occur at the same time as set B, and they span the total
sample space , the set of all possible outcomes. The normalization
rule can therefore be applied to sets A and B: P (A) + P (B) = 1. (It is
customary to write the probability for a set A as P (A)).
Sets do not need to be independent. For example, we could define
three sets A = 1,2,3, B = 3,4 and C = 4,5,6. These sets span the sample
space, but they are not independent. We can therefore not apply the
normalization rule to these sets: P (A) + P (B) + P (C) 6= 1.
Random variable. If the outcome of an experiment is a number, such
as for the throw of a die, we call the outcome a random variable, and
usually denote it with a upper case symbol for the random variable and
a lower case sumbol for the outcome of an experiment. For example, for
the throw of a die, we would use the symbol N for the number shown
in the die, and the symbol n for a particle outcome. The probability to
observe the outcome n when we measure the random variable N can
therefore be written as the probability for N to be equal to n: P (N = n).
Rule of addition. For two independent sets of outcomes, A and B, the
probability that A or B occurs is given as the sum of their probabilities:
68 4 Probability and Statistics

Addition rule: For two independent sets A and B, the probability

for A or B is
P (A or B) = P (A) + P (B) . (4.12)

This rule can be proven using the frequency definition of probabilities:

If we perform M experiments, the number of results in set A is N (A)
and the number of results in set B is N (B). The number of results in A
or B is therefore N (A) + N (B) and the probability for A or B is therefore
P (A or B) = (N (A) + N (B))/M = P (A) + P (B). We can extend this
rule also to cases where A and B not are independent:

P (A or B) = P (A) + P (B) − P (A and B) . (4.13)

Where the last term is zero if A and B are independent, because in that
case they cannot both occur at the same time.

4.2.7 Probability for simultaneous outcomes

What would happen if we threw a die two times? How can we find
the probability to throw a 1 in the first throw and a 6 in the second
throw? This corresponds to asking questions about the outcomes from
two experiments. We could write the probability for both of these events
to occur at the same time as:

P (n1 = 1 and n2 = 6) , (4.14)

We could call the event that we get a 1 in the first throw A and the event
that we get a 6 in the second throw B. We can then write the probability
for both to occur as P (A and B). This is usually written at P (A ∩ B),
using the intersection sign ∩ because the outcome is the intersection of
A and B, the set of outcomes that include both A and B.
Numerical approach. First, we estimate the probability for (A and B)
numerically: We draw two random numbers, n1 and n2 , many times, and
count how many times, N , we get a n1 = 1 and n2 = 6 at the same time.
First, we generate the numbers n1 and n2 :
n1 = randint(0,6,1000)+1
n2 = randint(0,6,1000)+1

We can find all the values that are equal to 1 or 6 using the equal to
operator
4.2 Probabilities 69

i1 = 1*(n1==1)
i2 = 1*(n2==6)

Where we have multiplied with one to ensure the result is an integer and
not True/False. Then we find how many times n1 = 1 and n2 = 1 at
the same time by summing i1 and i2. Where this sum is equal to 2,
both n1 = 1 and n2 = 6. We then simply have to count how many times
the sum is equal to 2 to find N , the number of times n1 = 1 and n2 = 6
at the same time, and the frequency, F = N/M , of this:
m = 1*(i1+i2)==2
N = sum(m)
F = N/1000.0
print p

0.034

The result of this experiment is therefore p ' F = N/M = 0.034.

Theoretical approach. How can we develop a theoretical answer for
this problem? There are several approaches we can follow. We can use
the same approach as we used numerically above: We can look at all
the possible outcomes from the experiment which consists of throwing
a dice two times. How many outcomes are there in total? The first die
has 6 possible outcomes, and the second die has 6 outcomes. In total,
there are 6 × 6 = 36 different outcomes. (Notice in particular, that the
outcome where the first die is a 2 and the second is a 4 is different from
the outcome where the first die is a 4 and the second die is a 2. We have
to count both these). All these outcomes are equally likely, and will have
the same probability, if the die is fair. This means that the probability p
for any such outcome is found from the normalization rule:
36
p = p 36 = 1 ⇒ p = 1/36 . (4.15)
X

i=1

The probability for a particular outcome, n1 = 1 and n2 = 6 is the

probability for one of these 36 outcomes, and hence the probability for
this is 1/36.
This approach is very robust. We can easily use the same approach to
find the probability for the first die to be equal to 1 or 2 and the second
die to be equal to 6. Can you find the argument and the answer?
Alternative theoretical approach. Notice that we could have used an
alternative approach: In order to get a 1 in the first throw and then a
6 in the second throw, we see that there are 6 possibilities for the first
70 4 Probability and Statistics

throw, and only 1 of these give a 1. That is, the probability to get a 1
in the first throw is p1 = 1/6. Then, in the second throw, there are six
possible outcomes, but only one of them give a 6. Thus the probability
to get a 6 in the second throw is p2 = 1/6. What is the probability to
get both, that is a 1 in the first throw and a 6 in the second throw. The
total number of outcomes is 6 × 6 and the number of outcomes that give
n1 = 1 and n2 = 6 is 1 × 1. Hence, the probability for both is
1×1 11
p= = = p1 p2 . (4.16)
6×6 66
This indicates that the probability for (A and B) is the product of the
probability for each of the events — given that the events cannot occur
at the same time. (If they could occur at the same time, that is, that
they are not independent, then we could not find the total number of
outcomes by multiplying the number of outcomes in throw 1 and the
number of outcomes in throw 2).
Rule of multiplication. This rule is indeed a general law in statistics,
which we call the rule of multiplication:

Rule of multiplication: For two independent events A and B, the

probability to observe A and B at the same time is the product of
the probability to observe A and the probability to observe B:

P (A ∩ B) = P (A) P (B) , (4.17)

4.2.8 Example: Throwing two dice

Let us answer various questions for the results of two fair dice.
What is the probability to throw 1,2 or 3 in the first and 4,5 or 6 in the
second throw?
We can use the rule of multiplication to find this. The probability
to throw 1,2 or 3 in the first throw can be found from the addition
rule. The outcomes 1,2 or 3 cannot occur at the same time. Hence the
probability to observe 1,2 or 3 is the sum of the probability for each,
p1 = P (1) + P (2) + P (3) = p + p + p = 1/6 + 1/6 + 1/6 = 3/6. Similarly,
the probability to throw 4,5 or 6 in the second throw is p2 = 3/6.
The rule of multiplication then gives that the probability for both is
p1 p2 = (3/6) (3/6) = 9/36 = 1/4.
4.3 Expected value and Variance 71

What is the probability that the sum of the dice is 7?

We can use the rule of addition to find this. We need to find all
the possible outcomes that add to 7. Each of the outcomes have the
same probability (1/36) and they cannot occur at the same time. What
outcomes add to seven? We write the outcome as two numbers where
(1, 2) means that the first throw is a 1 and the second is a 2, whereas
(2, 1) means that the first throw is a 2 and the second throw is a 1. The
outcomes that add to 7 are (1, 6), (2, 5), (3, 4), (4, 3), (5, 2), and (6, 1),
six outcomes in total. The probability that the sum is 7 is therefore
P (n1 + n2 = 7) = 6 (1/36) = 1/6.
What is the probability that at least one of the dice show a 1?
This problem can be solved in several ways. We could count the number
of outcomes (n1 , n2 ) where at least one of the numbers is a 1: There are
11 such outcomes. This means that probability for at least one 1 is 12/36.
Can you think of a way to use the rule of multiplication? (Be careful
to ensure that the events you are multiplying cannot occur at the same
time!)

4.3 Expected value and Variance

We have now found that we can describe a random process by the proba-
bility for a given outcome. However, there are even simpler descriptions
that can be measured directly from the sequence of measurements, the
average and the standard deviation:

4.3.1 Average
For a sequence of outcomes ni , i = 1, . . . , M , the average of the
outcome is defined as the arithmetic average, n̄, of the outcomes:
M
1 X
n̄ = ni , (4.18)
M i=1

4.3.2 Expected value

The average is an estimator for an underlying property of the process,
which we call the expected value of the random variable N . We can see the
72 4 Probability and Statistics

relation between the estimator and the underlying property by replacing

the sum in the average by a sum over all possible outcomes, x, and the
number of times each outcome x occurs, Nx :
M
ni X Nx x X  Nx 
n̄ = = = x= F (x) x , (4.19)
X X

i=1
M x M x M x

In the limit of infinitely many measurements, the frequency F (x) → P (x),

and we can replace the sum with a sum over probabilities

E(N ) = lim F (x) x = P (x) x , (4.20)

X X
M →∞
x x

where we have introduced the notation E(N ) for this asymptotic value
of the average. This is indeed what we will use as the definition of the
expected value:

Expected value of a random variable: The expected value

E(N ) of a random variable N is defined as:

E(N ) = P (n) n , (4.21)

X

where P (n) is the probability to observe outcome n.

We will also use the notation hN i = E(N ) for the expected
value.

The best way to estimate the expected value is to use the average as an
estimator for the expected value. It can be shown that the average may
be considered a best possible estimator for the expected value — using
a specific definition of the word best. (You will learn more about what
characterizes good estimators and how to find them in basic courses in
statistics, and in a textbook such as ).

4.3.3 The expected value is linear

The expected value is linear. This means that the expected value of
Y = aX + b is

E(Y ) = E(aX + b) = a E(X) + b . (4.22)

4.3 Expected value and Variance 73

We can prove this using the definition of the expected value in (4.21):

E(Y ) = E(aX + b) = P (y) y = P (x)(ax + b) (4.23)

X X

y x

=a P (x) x + n P (x) = aE(X) + b . (4.24)

X X

x x

4.3.4 Expected value of a function g(N ) of a random variable

N
We will often be interested in the expected value of simple functions of
random variables, such as N 2 or sin(N ). We can find the expected value
from the definition:
We can calculate the average of the outcome n directly, but we can
also calculate the average of a function of n. For example, we may be
interested in the average of n2 or sin(n). In this case, we define the
expected value as:

E(g(N )) = hg(N )i = g(N (x)) P (x). (4.25)

X

4.3.5 Example: Expected value of a die throw

Let us use this to find the expected value for the throw of a die. The
result of a throw is the random variable N , and the expected value is
therefore hN i = E(N ). We find hN i from the probability P (n) to observe
the value n:
6 6 6
1
hN i = E(N ) = P (n) n = pn =
X X X
n
n=1 n=1
6 n=1 (4.26)
1 21 7
= (1 + 2 + 3 + 4 + 5 + 6) = = .
6 6 2
Notice that this is an exact result that characterizes the underlying
process — it is not a measurement.
For a sequence of observations, nj , j = 1, . . . , M , we can estimate the
expected value of N using the average:
M
1 X
n̄ = ni , (4.27)
M j=1
74 4 Probability and Statistics

where the sum is over all the measurements M . For example, we can use
python to estimate the average of the sequence of die throws found in
Sect. 4.2.5 using using the formula in (4.27):
myavg = sum(m)/len(m)
print myavg

3.5044

The average is implemented as the standard function average in python:

myavg2 = average(nd)
print myavg2

3.5044

Notice that this estimated value is close, but not exactly equal to, the
theoretical value.

4.3.6 Variance and Standard deviation

In addition to the expected value, we may be interested in understanding
how an outcome varies around the expected value. Does the outcomes
typically occur very close to or far away from the expected value. We
could characterize this by the expected value of the deviation from the
expected value of the random variable N :

E (N − E(N )) = P (x) (x − E(N )) = P (x) x − P (x)E(N

(4.28)
)
X X X

x x x
| {z }
=E(N )

= E(N ) − E(N ) P (x) = E(N ) − E(N ) = 0 . (4.29)

X

x
| {z }
=1

Ok, so this did not work, since the result is always zero. Instead, we may
use the absolute value of the deviation:

E (|N − E(N )|) = P (x) |x − E(N )| . (4.30)

X

This does not become zero. However, it is more usual to characterize the
deviation by the expected value of the square of the deviation:
 
E (N − E(N ))2 . (4.31)
4.3 Expected value and Variance 75

We can simplify the expression by expanding the paranthesis:

   
E (N − E(N ))2 = E N 2 − 2N E(N ) + (E(N ))2
= E(N 2 ) − 2E(N )E(N ) + (E(N ))2 (4.32)
2
= E(N 2 ) − (E(N )) .

The average of N 2 is called the variance of the random variable N :

The variance of a random variable N is defined as the expected

value of N 2 :
Var(N ) = E(N 2 ) . (4.33)
Here, we will use the term deviation for the deviation from the
expected value:
 
E (N − E(N ))2 = σ 2 = E(N 2 ) − (E(N ))2 . (4.34)

For a random variable with zero expected value, the deviation and the
variance is the same.

4.3.7 Standard deviation

We estimate the deviation, which can be used to find the variance, using
the standard deviation:

Standard deviation: We measure the standard deviation, σ̄, of a

sequence of M outcomes ni using:
v
M
1 X
u
σ̄ = t (ni − n̄)2 . (4.35)
u
M − 1 i=1

where n̄ is the average, which also is estimated from the data.

(There is a subtle statistical reason that we divide by M − 1 and not by

M in this sum if we use the estimated value n̄ instead of the exact value
hN i in this sum.)
76 4 Probability and Statistics

4.3.8 Example: Variance and standard deviation of a die

throw
We found that for a single die the probability is P (i) = p = 1/6 for
i = 1, . . . , 6. We can find the deviation for a single throw of a die, N ,
through

hN 2 i − (hN i)2
6 6
!2
= P (i)i − P (i)i
X X
2

i=1 i=1
(4.36)
1 7
 2
= (12 + 22 + 32 + 42 + 52 + 62 ) −
6 2
91 49 35
= − = ,
6 4 12
This is again an exact result. We can estimate the standard deviation
directly from the 10000 dice throws directly using the formula in (4.32):
myavg = average(m)
mystd = sqrt(sum(m*m)/(len(m)-1.0)-myavg*myavg)
print mystd

1.7072142923487961

Or you can use the built-in function std directly

std(m)

1.7072142923487961

4.4 Toward a theory for n(t) for the MD model

Now, let us use what we have learned so far to develop a theory that
can describe the probability P (n) to observe a particular value n in the
molecular dynamics simulation of the gas/liquid system.

4.4.1 One atom model

Let us start by simplifying the system. First to one atom. In that case we
guess that the atom has equal probability to be in the left or in the right
4.4 Toward a theory for n(t) for the MD model 77

hand half, since the system is symmetric and we have no reason to prefer
the left to the right side. Hence we would expect there only to be two
possible outcomes for n, n = 0 and n = 1 with equal probability. This is
just like flipping a coin (throwing a 2-sided dice). From our discussions
above, we know that the probability for a fair coin to show the number n
corresponds to the probability of a fair, two-sided die to show the number
n:
P (n) = 1/2, n = 0, 1 , (4.37)
where we have already seen that this theory fits well with a numerical
simulation. However, this model is too simple to predict n(t) for a many-
particle gas. Let us instead address a two-atom system.

4.4.2 Two-atom model

We make the model slightly more advanced by looking at two independent
atoms, 1 and 2. We will assume that each atom is described by the random
variables n1 and n2 , which is 1 if the atom is on the left side and 0 if
the atom is on the right side. We assume that these two values are
independent and that their probabibilities for each of the atoms are the
same as for a single atom in (4.37).
Using this notation, the total number of atoms on the left side is
n = n1 + n2 . How can we find the probability for the possible values
of n? We address this by a numerical experiment and by a theoretical
argument:
Numerical approach. We study the distribution of n by performing
a numerical statistical experiment. We generate two sets of random
numbers, n1 (i) and n2 (i), for i = 1, . . . , M , and add them to get n. We
can then find the probabilities for various values of n using the same
method as we did above. First, we generate 2 arrays n1 and n2 each with
M elements. We do this by generating a variable with M × 2 random
numbers so that the first column equals n1 and the second column equals
n2 :
M = 1000
m = randint(0,2,(M,2))
print m[0:5,:]

[[0 1]
[1 0]
[0 0]
78 4 Probability and Statistics

[1 1]
[1 0]]

Where we have shown the first 5 elements for n1 and n2 . We find n by

adding the columns corresponding to n1 and n2 . That is, we want to
generate a new array with M elements, where each element corresponds
to the sum along each row in the m-matrix. This is done by the sum
command, but we must specify that the sum should be done along the
second dimension of the array, so that we sum along the rows:
n = sum(m,axis=1)
print n[0:5]

[1 1 0 2 1]

We can now find and plot the histogram:

M = 10000
N = 2
m = randint(0,2,(M,N))
n = sum(m,axis=1)
hist(n), show()

The resulting plot is shown Fig. 4.4.

Fig. 4.4 Plot of the his- 6000

togram for 10000 realiza- 5000

tions of the sum of two coin 4000

tosses: n = n1 + n2 , where
Ni

3000

n1 = 0, 1 and n2 = 0, 1. 2000

1000

0
0 1 2
i

Theoretical approach. We can find the probability P (n) from a theo-

retical argument: There are four possible outcomes of (n1 , n2 ), but only
three possible outcomes of n = n1 + n2 , as illustrated in the following
table:
n1 n2 n = n1 + n2
0 0 0
1 0 1
0 1 1
1 1 2

If all the 4 outcomes of (n1 , n2 ) are equally likely, with probability

p = 1/4, then the probability for an outcome n is given as the number,
4.4 Toward a theory for n(t) for the MD model 79

g(n) of (n1 , n2 )-states that give the same n-value, multiplied with the
probability, p, per state, where p = 1/M , and M = 22 is the total number
of (n1 , n2 )-states.
1
P (n) = g(n)p = M g(n) , (4.38)
2
where the multiplicity g(n) is read from the table above, and are
tabulated in the following table:

n g(n) P (n)
0 1 1/4
1 2 2/4
2 1 1/4

We can compare these results directly with the histogram in Fig. 4.4.
The histogram records how many experiments, Nn , that resulted in
outcome n, when the total number of experiments were M = 10000.
The probability P (n) is approximately equal to Nn /M , hence we must
compare Nn = P (n) M with the values in the histogram, as done with
the following code:
nn = array([0 1 2])
Pn = array([0.25, 0.5, 0.25])
Nn = Pn*M
plot(nn,Nn)

The resulting correspondence is excellent, as seen in Fig. 4.4.

4.4.3 Many-atom system

How can we generalize this theoretical approach to many atoms? For the
two-atom system we counted the number of possible outcomes, that is,
all the possible (n1 , n2 ) values: (0, 0), (1, 0), (0, 1), (1, 1). There are four
outcomes in total. We call such an outcome a microstate. There are four
microstates. Each such microstate is equally probable — just like when
we threw two dice. The possible states for n are n = 0, n = 1, and n = 2.
We call these states macrostates. The macrostates do not have the
same probability, because there may be a different number of microstates
in each macrostate. To find the probability of a macrostate, we have to
count how the number of microstates in a particular macrostate.
For two atoms there are four microstates in total, each with probability
1/4. The macrostate n = 0 can only result from one microstate, n = 1
from two, and n = 2 from one.
80 4 Probability and Statistics

This method can easily be generalized to any number of atoms. For

N atoms, each microstate can be described as a sequence of N numbers,
(n1 , n2 , . . . , nN ), for example (1, 0, 0, 1, 0, 1, 1, 0, 0, 0, . . . , 1), and n will be
the sum, n = i ni , of the values in the sequence. To find the probability
P

of a macrostate, we must count how many microstates results in a

particular macrostate. To do this, we use a clever trick: We realize that
the value of n only depends on how many of the ni in the microstate are
1. It does not matter where they are in the sequence. We can therefore
find all microstates that give a value n by finding out how many ways we
can place n ones in a sequence of N numbers. (The remaining numbers
in the sequence will be zeros).
We have now mapped the problem onto a well know problem in
combinatorics with a simple solution4 . We have n ones (and N − n zeros).
Let us decide where to place the ones. The first one can be placed in N
positions in the sequence, the second in N − 1 positions and so on until
we reach N − n + 1. This results in N !/(N − n)! different ways to place
the ones.
However, using this method we have counted many microstates several
times, because the ones are identical and therefore the order in which
they are placed into the sequence is inconsequential. We can show this
in the two-atom system. With the proposed counting method, we count
one state where we first place a 1 at the first position (n1 = 1), and
then we place a one at the second position (n2 = 1). This is the state
(1, 1). However, we also count a state where we first place a 1 at the
second position (n2 = 1) and then we place a one at the first position
(n1 = 1), giving the state (1, 1). But these two states are identical! And
here we have counted these states twice. We must therefore correct for
this by dividing by the number of times we have counted the same
state. For a sequence with n ones, such as for this sequence with 4 ones:
(1, 0, 1, 1, 0, 1), we must divide by all the possible ways we could have
generated this state, which corresponds to all the possible ways we can
organize n numbers, which is n!. This means that the total number of
microstates is
N!
!
N
Ω(n, N ) = = , (4.39)
(N − n)! n! n
where we have introduced the multiplicity Ω(n, N ) and the usual notation
used for this combinatorical result.
4
This is a common approach in physics that you will meet many times in your career
as a physicist.
4.4 Toward a theory for n(t) for the MD model 81

The total number of microstates is 2N , and all these microstates are

equally likely, so that the probabilty for n is

1 N! 1
!
N
P (n) = = . (4.40)
n 2N n!(N − n)! 2N

4.4.4 Comparing theory and simulation

We have now developed a theoretical description of the system, and the
theory provides a prediction for the distribution of n — the number of
atoms in the left half of the system — given the assumption that the
position of each atom is independent of all the position of all the other
atoms.
Reading data from simulations. Let us compare the theory directly
with the measurements. First, we compare the simulated result from 5001
i-values, ni , (from a simulation of 50000 time-steps in in.gasstatistics015 )
and N = 400 atoms. The resulting trajectory gasstat01.lammpstrj6 is
analyzed by the script gasplotnt01.py7 . You can find the data in the
file ndata01.d8 , which was generated by the script. We read this data,
extract the n-values, and plot n(t)
from pylab import *
t,n=loadtxt(’ndata01.d’);
plot(t,n), show()

The resulting plot of n(t) is shown in the top of Fig. 4.5.

Estimating the probability distribution P (n). We estimate the proba-
bility distribution P (n) for the n using the histogram function: We count
the number of times, Nn , that n(t) = n, and then divide by the total
number of observations, M (called nt in the code):
Nn
P (n) ' . (4.41)
M
The results depend on the number of bins used for the histogram. We
can display the histogram with 10 and 20 bins using
hist(n,bins=10)
5
http://folk.uio.no/malthe/fys2160/in.gasstatistics01
6
http://folk.uio.no/malthe/fys2160/gasstat01.lammpstrj
7
http://folk.uio.no/malthe/fys2160/gasplotnt01.py
8
http://folk.uio.no/malthe/fys2160/ndata01.d
82 4 Probability and Statistics

show()
hist(n,bins=20)
show()

The resulting histograms are shown in Fig. 4.5. Notice that the number
of counts in each bin goes down when the number of bins is increased.
These histograms only measure the number of outcomes in each bin,
where the bin may span over a size ∆n corresponding to many n-values.
However, we would like to estimate P (n) as Nn /M , where Nn is the
number of outcomes that are equal to n. This can be done either by
ensuring that the bin width is n, or we need to divide by the bin size
in order to find the estimate for the probability P (n) across the bin:
P (n) ' N (n, n + ∆n)/(M ∆n). Why do we need to divide by ∆n? If we
count N (n, n + ∆n), we have really counted

N (n, n + ∆n) = N (n) + N (n + 1) + N (n + 2) + . . . + N (n + ∆n) . (4.42)

If we now divide by M , the total number of outcomes, we get

N (n, n + ∆n) N (n) N (n + 1) N (n + 2) N (n + ∆n)
= + + + ... +
M M M M M

(4.43)
' P (n) + P (n + 1) + P (n + 2) + . . . + P (n + ∆n)
(4.44)
1
= ∆n (P (n) + P (n + 1) + P (n + 2) + . . . + P (n + ∆n))
∆n
(4.45)
= ∆n P (n + ∆n/2) . (4.46)

We therefore need to divide by ∆n to find the average probability over

the interval from n to n + ∆n, and we need to find the centers of the
bins, n + ∆n/2, in order to plot the data in this point. We can calculate
the probabilities and the bin centers directly from the output from the
histogram-function:
Nn,edges=histogram(n,bins=10)
ncenter = 0.5*(edges[1:]+edges[:-1])
dn = edges[1:]-edges[:-1]
plot(ncenter,Nn/dn/nt)
4.4 Toward a theory for n(t) for the MD model 83

240 1500
220
1000

N (n )
n (t) 200
500
180
160 0
0 1 2 3 4 5 160 180 200 220 240
t 4 n
x 10
240

220
n (t)

200

180

160
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
t x 10
5

2000 2000
N (n )

N (n )
1000 1000

0 0
160 180 200 220 240 160 180 200 220 240
n n
Fig. 4.5 Plot of n(t) from the simulation and ni from 5001 random microstates from
50,000 timesteps (top left) and 500,000 timesteps (middle). (Top right) Histogram of
n(t) for 50,000 timesteps with 10 and 20 bins. (Bottom left) Histogram of n(t) for 50,000
timesteps (blue) and for a simulation with 5001 random outcomes using the binomial
model (red). (Bottom right) Histogram of n(t) for 500,000 timesteps (blue) and for a
simulation with 5001 random outcomes using the binomial model (red).

Comparison with theory — numerical estimates. We now have the

tools needed to compare the observed results from the molecular dynamics
simulations with our simple theory for n(t). The number of observations
from the molecular dynamics simulations is 5001. To compare with the
observed results, we generate set of possible values of n for N = 400
atoms using the theory we have developed so far: Each of the N atoms
can either be on the left side (Xi = 1) or on the right side (Xi = 0) with
equal probability, and we count the total number of atoms, n, on the left
side
N
n= (4.47)
X
Xi ,
i=1

We generate 5001 such values randomly and compare the two histograms:
ni = randint(0,2,(5001,400))
nt = sum(ni,axis=1)
84 4 Probability and Statistics

hist(nt), hist(n), show()

The resulting plot is shown in Fig. 4.5. Hmmm. This did not look
like a very good fit. What has gone wrong? It may be that the initial
correlations in the data were too strong, so that not all the atoms really
are active and can change places between each measurement from the
simulation. Let us increase the time interval between measurements of ni ,
but keep the number of observations the same. We rerun with the input
file in.gasstatistics029 resulting in gasstat02.lammpstrj10 , producing the
data file ndata02.d11 , which we plot using
from pylab import *
t,n=loadtxt(’ndata02.d’);
ni = floor(rand(5001,400)*2)
nt = sum(ni,axis=1)
hist(nt), hist(n), show()

The resulting sequence n(t) of n-values is shown in Fig. 4.5. We see that
the correlations in the n(t) signal are now less prominent, although there
are still some correlations present. However, the histograms in the bottom
right part of Fig. 4.5 now show much better correspondence, indicating
that the theory we have developed provides a good explanation of the
behavior. Indeed, the correspondence between observations and theory
is surprisingly good given the simplicity of the model. We may therefore
conclude that the model captures the most important features of the
behavior of n(t).
Comparison with theory — calculated probabilities. Now, let us also
compare directly with the probabilities we have calculated for n:
N!
P (n) = 2−N , (4.48)
n!(N − n)!
We use the scipy.stats.binom.pmf function to calculate P (n) as a
function of n. (This uses the scipy12 package). In order to compare
directly with the histogram, we notice that P (n) = Nn /M , and therefore
Nn = M P (n). We also choose many bins (301) to ensure that each value
of n falls into one and only one bin, so that P (n) = Nn /M , where Nn is
the number of outcomes in the bin that includes n. This is done by the
following script:
9
http://folk.uio.no/malthe/fys2160/in.gasstatistics02
10
http://folk.uio.no/malthe/fys2160/gasstat02.lammpstrj
11
http://folk.uio.no/malthe/fys2160/ndata02.d
12
http://www.scipy.org
4.5 The binomial distriubtion 85

import scipy, scipy.stats

M = 5001;
Nn,edges=histogram(n,bins=10)
ncenter = 0.5*(edges[1:]+edges[:-1])
dn = edges[1:]-edges[:-1]
xn = scipy.linspace(0,400,401)
pmf = scipy.stats.binom.pmf(xn,400,0.5)
plot(ncenter,Nn/dn/M,xn,pmf), show()

The resulting plot is shown in Fig. 4.6. The correspondence is not spec-
tacular, but decent, showing that the theory we have developed gives
a reasonably good description of n(t) from the molecular dynamics
simulations.

250

200

150
N (n )

100

50

0
160 170 180 190 200 210 220 230 240
n

Fig. 4.6 Plot of the histogram for n(t) from the simulation and ni from a binomial
distribution.

4.5 The binomial distriubtion

The binomial distribution fits well to the observed data for the molecular
simulation. Indeed, the binomial distribution provides a good description
for any sum of independent events, such as the sum of several dice, the
results of many flips of a coin, or the behavior of the number of atoms in
a part of a gas.
In general, the binomial distribution describes the case where we have
a sequence of N identical, independent trials with discrete outcomes Xi ,
i = 1, . . . , N – often called a Bernoulli process. The simplest case is when
Xi is binary, such as either 0 or 1. We have looked at a fair coin, but
even when the coin is not fair, such as if the case if we have probability
p for Xi = 1 and q = 1 − p for Xi = 0, the results are described by the
binomial distribution.
86 4 Probability and Statistics

Binomial distribution: For N independent random variables, Xi ,

with the same distribution of outcomes, the sum Z = N
i=1 Xi , is
P

described by a binomial distribution, P (z):

!
N
P (Z = z) = pz q N −z , (4.49)
z

This formula reproduces our results from above for a fair dice with
p = q = 1 − p = 1/2.

4.5.1 Example: Coin toss

We can use the binomial distribution to answer questions about coin
tosses. Let us assume we flip a fair coin, so that p = q = 1/2.
(a). If we flip the coin N times, what is the probability that all the flips
are ones?
Getting all the flips to be ones corresponds to n = N . We can use the
formula to find the probability for this:
N!
P (N, N ) = 2−N = 2−N . (4.50)
N !(N − N )!
(b). If we flip the coin N = 5 times, what is the probability to get
exactly 3 heads? Again, we can apply the formula directly, now with
n = 3 heads and N = 5 flips:
5! 5
P (N = 5, n = 3) = 2−5 = . (4.51)
3!(5 − 3)! 16

4.5.2 Properties of the binomial distribution

We can gain intuition about the binomial distribution by studying nu-
merical experiments of N = 10, 20, 40, and 80 trials when Xi is 0 or 1
with equal probability, and then measuring Z = i Xi . This is done by
P

generating a set of M random sequences of N steps:

(
0 (1 − p)
Xi,j i = 1, . . . , N , j = 1, . . . , M , (4.52)
1 p
4.5 The binomial distriubtion 87

and
N
zj = Xi,j , j = 1, . . . , M , (4.53)
X

i=1

This gives M outcomes zj , and we can then collect statistics about these
outcomes. We generate M outcomes zj of length N in the array z using
xi = randint(0,2,(M,N))
z = sum(xi,axis=1)

Again, we use histograms to estimate P (z) ' Nz /M , where Nz is how

many times we observed z in M outcomes:
from pylab import *
Nvalues = array([10,20,40,80]) # Number of flips
M = 100000 # Number of samples
for ival in range(0,size(Nvalues)):
print ival
N = Nvalues[ival]
ni = randint(0,2,(M,N))
nt = sum(ni,axis=1)
y,binEdges=histogram(nt,bins=100)
bincenters = 0.5*(binEdges[1:]+binEdges[:-1])
j = find(y>0)
plot(bincenters[j],y[j]/(1.0*M),’-o’)
xlabel(’n’)
ylabel(’P(n)’)
show()

We have used 100 bins. This is more than the number of possible values,
which span from 0 to 80 for the case where N = 80 and over a smaller
range when N is smaller. This was done to ensure that each bin only
contains one possible outcome. Now, if we use 100 bins for the case when
N = 20, there are only 21 possible outcomes, and many of the bins will
be empty. We therefore only plot the values in the bins that contain at
least one outcome. This is done by the find function, which returns an
array with indicies for all the element where the histogram is non-zero.
The resulting plots of the estimated probabilities for N = 10, 20, 40, 80
are shown in Fig. 4.7.

4.5.3 Expected value and variance of the binomial

distribution
From the plots we see that both the average value and the width of P (z; N )
increases with N . Let us see how this compares with the theoretical values
for the expected value and the variance of the binomial distribution. We
88 4 Probability and Statistics

0.25 0.25
N = 10
N = 20
0.2 N = 40 0.2
N = 80

0.15 0.15

P (n )

P (n )
0.1 0.1

0.05 0.05

0 0
0 20 40 60 80 −20 −10 0 10 20
n (n − ⟨n ⟩)

0.25 0.4

0.2
0.3

σ · P (n )
0.15
P (n )

0.2
0.1

0.1
0.05

0 0
−5 0 5 −5 0 5
( n − ⟨ n ⟩ ) /σ ( n − ⟨ n ⟩ ) /σ
Fig. 4.7 Plot of the P (n) as a function of n for N = 10, 20, 40, and 80.

can calculate the theoretical expected value of the binomial probability

distribution using the definition of the expected value:
N N
!
N
hni = nP (N, n) = npn q N −n
X X

n=0 n=0
n
N
!
d N (4.54)
=p pn q N −n
X
dp n=0 n
d
=p (p + q)N = pN (p + q)N −1 = N p ,
dp
Similarly, you can use the same approach to show that the deviation is:

h(n − hni)2 i = N pq , (4.55)

We test these theoretical predictions directly on the simulated results.

First, let us measure the deviation from the average, n − hni = n − N p,
and find the histogram as a function of the average. As we see in Fig. 4.7
4.5 The binomial distriubtion 89

the distribution now appears symmetric around n − N p = 0, which

validates the theory for the average.
What about the width√of the distribution? The theoretical prediction
is that the width is σ = N pq. We can test this by rescaling the n-scale
√
with σ by plotting P (n) as a function of (n − hni)/σ where σ = N pq.
The resulting plot is shown in Fig. 4.7. The widths in these figures are
correct, but the heights are now different. Experimentation shows that
we can rescale the height also by multiplying with σ as shown in Fig. 4.7.
Can we understand how we arrived at this particular way of rescal-
ing the distributions? Yes – this we can understand by addressing the
continuum limit of the distribution.

4.5.4 Continuum limit of the binomial distribution

The binomial distribution is valid for discrete values z. However, as N
becomes large, the range of possible outcomes for the sum becomes wide
and we may ask what the limit of the distribution is for large N ?
In this case, we need an approximate expression for N ! when N is
large. This is given by Stirling’s approximation, which is
√
ln N ! ' ln 2πN + N (ln N − 1) , (4.56)

The second term contains the factor N , which typically will be much
larger than the ln N term in the first factor. We will therefore often only
include the second N ln N − N term, but here we will also include the
smaller first factor.
Here, we will prove that when N becomes large, the binomial distri-
bution approaches that of a Gaussian or a Normal distribution
N! 1 −
((N/2)−n)2
P (N, n) = 2−N ' √ e 2(N/4) . (4.57)
n!(N − n)! 2πσ 2
We will provide you with two ways to approach this derivation, using
purely analytical techniques and using a combination of analytical and
symbolic methods.

4.5.5 Continuum limit — Analytical derivation

We will study the binomial distribution when n is close to its average
value hni = N p = N/2. To simplify the notation we will here use hni = n̄.
90 4 Probability and Statistics

We introduce the relative deviation from the average h, defined as

n − n̄
h= , (4.58)
n̄
where we will look at the case when n is close to its average so that h is
small. We solve for n:

n̄h = n − n̄ ⇒ n = n̄ + n̄h = n̄ (1 + h) . (4.59)

Similarly, we can express N − n in terms of n̄ and h:

N − n = N − n̄ (1 + h) = 2n̄ − n̄ − n̄h = n̄ (1 − h) . (4.60)

We can use these terms to simplify the binomial distribution

N! N!
P (N, n) = 2−N = 2−N . (4.61)
n!(N − n)! (n̄ [1 + h)]! [n̄ (1 − h)]!
Let us now look at ln P (N, n) and gradually introduce Stirling’s formula
for the faculty functions

ln P (N, n) = ln (2πN )1/2 + N ln N − N (4.62)

1/2
− ln (2πn) − n ln n + n (4.63)
− ln (2π(N − n))1/2 − (N − n) ln(N − n) + (N − n) (4.64)
− N ln 2 , (4.65)

where we notice that the terms

− N + n + (N − n) = 0 , (4.66)

cancel. As you do more of these approximations by youself, you will

notice that this is always the case when you apply Stirling’s formula to
the binomial formula.
Let us look at the terms that include the 2π factor, and introduce
n = n̄(1 + h) and (N − n) = n̄(1 − h). Notice that we have still not made
any other approximations than to apply Stirling’s formula.
4.5 The binomial distriubtion 91

ln (2πN )1/2 − ln (2πn)1/2 − ln (2π(N − n))1/2 (4.67)

1 1 1
= ln (2πN ) − ln (2πn̄ (1 + h)) − ln (2πn̄ (1 − h)) (4.68)
2 2 2
1 2πN
= ln (4.69)
2 2πn̄ (1 + h) 2πn̄ (1 − h)
1 N
= ln . (4.70)
2 2πn̄ (1 − h2 )
2

Now we assume that h  1 and we insert that n̄ = N p = N/2, getting

1 N 1 N 1 N
ln ' ln = ln , (4.71)
2 2πn̄ (1 − h )
2 2 2 2πn̄2 2 2π(N/2)2

where we will not insert σ 2 = N pq = N/4:

1 1 1
ln = ln √ . (4.72)
2 2π(N/4) 2πσ 2
We have then been able to simplify ln P (N, n) significantly:
 1
ln P (N, n) = ln 2πσ 2 + N ln N − n ln n − (N − n) ln(N − n) − N ln 2 .
2

(4.73)
Now, let us simplify further by introducing n = n̄(1 − h) and (N − n) =
n̄(1 + h). Let us first simplify the −n ln N − (N − n) ln(N − n) terms:

− n ln n − (N − n) ln(N − n) (4.74)
= −n̄(1 + h) ln n̄(1 + h) − n̄(1 − h) ln n̄(1 − h) (4.75)
= −n̄(1 + h) (ln n̄ + ln(1 + h)) − n̄(1 − h) (ln n̄ + ln(1 − h)) (4.76)
= −2n̄ ln n̄ − n̄(1 + h) ln(1 + h) − n̄(1 − h) ln(1 − h) (4.77)
N N
= −2 ln − n̄(1 + h) ln(1 + h) − n̄(1 − h) ln(1 − h) (4.78)
2 2
= −N ln N + N ln 2 − n̄(1 + h) ln(1 + h) − n̄(1 − h) ln(1 − h) .
(4.79)

Here, we see that the terms −N ln N + N ln 2 cancel the terms N ln N −

N ln 2 in ln P (N, n). We are left with the last two terms. Now, we
assume that h  1, and we can therefore use that ln(1 + h) ' h and
ln(1 − h) ' −h, getting:
92 4 Probability and Statistics

− n̄(1 + h) ln(1 + h) − n̄(1 − h) ln(1 − h) (4.80)

' −n̄(1 + h)(h) − n̄(1 − h)(−h) (4.81)
= −2n̄h2 . (4.82)

We can now put all this together into ln P (N, n), getting
1
ln P (N, n) = ln √ − 2n̄h2 , (4.83)
2πσ 2

where we insert h = (n − n̄)/n̄ in the last term:

(n − n̄)2
2
n − n̄

− 2n̄h2 = −2n̄ = −2 . (4.84)
n̄ n̄
We insert for n̄ = N p = (N/2) = 2(N/4) = 2σ 2 :

1 (n − n̄)2
ln P (N, n) = ln √ −2 , (4.85)
2πσ 2 2σ 2

If we now insert N/4 = N pq = σ 2 , we see that the result of all of these

calculations are that:
1 2
1 1
e− 2 ( σ ) , (4.86)
− s s2 1 s 2
P (N, n) = √ e 2(N/4) = √ e− 2σ2 = √
2πσ 2 2πσ 2 2πσ 2
where
N
s= − n = n̄ − n . (4.87)
2
We can therefore rewrite the result in an even more general form:

1
e− 2 ( σ ) .
1 n−n̄ 2
P (N, n) = √ (4.88)
2πσ 2
We now use this result to understand the scaling we found to work in
fig. 4.7. We see that if we rewrite the equation as
1
P (N, n)σ = √ e− 2 ( σ ) ,
1 n−n 2
(4.89)
2π
we would expect all the data to fall onto a common, universal curve –
given as the Gaussian distribution – as we indeed observe in Fig. 4.7.
4.5 The binomial distriubtion 93

4.5.6 Continuum limit — Symbolic derivation

Also when we use symbolic tools, we need both to have a good mathemat-
ical intuition, and we often also need to rewrite and simplify expressions
ourselves. In particular, we often need to interpret limits, such as the limit
of large N , ourselves using our mathematical experience and intuition.
First, we realize that the general expression for P (N, n) is difficult
to work with. It is simpler to work with some variable s, which is close
to zero, instead of the variable n which is close to N/2, since we know
from experience that this makes the algebra simpler when we derive a
Taylor expansion around s = 0. We therefore introduce s = (N/2) − n,
and rewrite the binomial probability as
N!
P (N, n) = 2−N . (4.90)
(N/2 − u)! (N/2 + u)!
Also, we know from experience that with all these factors, it may be
simpler to work with the logarithm of the probability, ln P (N, n). We
introduce these quantities symbolically
from sympy import *
u = symbols(’u’)
N = symbols(’N’)
P = factorial(N)/(factorial((N/2)-u)*factorial((N/2)+u))*2**N
lnP = log(P)

We are then ready to start symbolic manipulation. First, we would

like to Taylor expand lnP around u = 0. We start from the lowest orders
(second order):
TlnP = series(lnP,u,n=2)
print TlnP

log(2**N*factorial(N)/(factorial(N/2))**2) + O(u**2)

Ooops. There is no u-dependence! We know from experience that this

typically means that the first order terms cancel. We need higher order
terms:
TlnP = series(lnP,u,n=3)
print TlnP

log(2**N*factorial(N)/(factorial(N/2))**2) + ...
u**2*(-gamma(N/2 + 1)*polygamma(0, N/2 + 1)**2/factorial(N/2) - ...
gamma(N/2 + 1)*polygamma(1, N/2 + 1)/factorial(N/2) + ...
gamma(N/2 + 1)**2*polygamma(0, N/2 + 1)**2/(factorial(N/2))**2) + ...
O(u**3)
94 4 Probability and Statistics

A bit messy. Let us simplify:

simplify(TlnP)

log(2**N*factorial(N)/(factorial(N/2))**2) - ...
u**2*polygamma(1, N/2 + 1) + O(u**3)

That is more like it! Now, we use our mathematical insight to simplify
further. We realize that N is a large number and that N/2 + 1 ' N/2.
The polygamma is the derivative of the digamma function. We can look
up the digamma-function, Ψ (x), finding that it can be approximated as

Ψ (x) ' ln x − (1/2x) − (1/12x2 ) + . . . (4.91)

In the limit of large x we can use Ψ (x) ' ln x. The notation

polygamma(1,N/2+1) means the first (hence the number 1) derivative
of the digamma-function. The polygamma(1,N/2+1)-term is therefore
approximately d ln x/dx = 1/x. We insert x = N/2, getting

N! u2
ln P = ln(1/2)N + ln − . (4.92)
(N/2)! (N/2)! (N/2)
This expression already shows us how P depends on u:
u2
− (N/2)
P (N, u) = C(N ) e . (4.93)

Where the prefactor C(N ) is

N!
ln C(N ) = ln(1/2)N + ln = −N ln 2 + ln N ! − 2 ln(N/2)! .
(N/2)! (N/2)!
(4.94)
In order to sort out the√prefactor, C(N ), we need to use Stirling’s
approximation ln x! ' ln 2πx + x ln x − x, which is valid for large x.
We apply Stirling’s approximation to the prefactor:

ln C(N ) = −N ln 2 + ln N ! − 2 ln(N/2)!
√
' −N ln 2 + ln 2πN + N ln N − N (4.95)
 q 
− 2 ln 2π(N/2) + (N/2) ln(N/2) − (N/2) .

This is again well suited for symbolic simplification. Notice the use of
sym(2) to ensure that python does not insert an approximate value for
log(2):
4.5 The binomial distriubtion 95

lnC= -N*log(sym(2))+log(sqrt(2*pi*N))+N*log(N)-N- ...

2*(log(sqrt(2*pi*(N/2)))+(N/2)*log(N/2)-(N/2))
simplify(lnC)

-log(sqrt(N)) - log(pi)/2 + log(2)/2

Thispis as good as it gets. We see that ln C = −(1/2) ln(πN/2) =

− ln 2πN/4. We insert this prefactor into P (N, u) in (4.93), getting
1 −u2
P (N, u) = p e 2(N/4) . (4.96)
2πN/4

This result can be generalized by realizing that n̄ = N/2 and σ 2 = N/4.

We can therefore write u as u = n̄ − n, and P (N, n) as
1
e− 2 ( σ ) .
1 n̄−n 2
P (N, n) = √ (4.97)
2πσ 2
This is exactly the same result as we found above, but now derived with
a combination of analytical and symbolic tools.

4.5.7 Probabilities and probability densities

There is a small, but very important, distinction between the P (N, n)
we introduced for the bionomial distribution, and the P (N, n) we have
introduced now, because the new version is defined for all possible values
of n, not only for discrete ones. This means that the probability to
observe a specific value for n is really zero. We can only specify the
probability to observe n in some interval from n to n + dn:

P (n is between n and n + dn) = P (N, n)dn , (4.98)

We now realize that our notation is confusing, and we should clean it up.
Let us introduce the notation that the random variable Z is the result
of a binomial experiment. The probability for the observed value of Z, z,
to be in the range from n to n + dn is then written as

P (n < Z < n + dn) = fZ (n)dn , (4.99)

where we call fZ (z) the probability density for the random variable
Z.
Notice that it is the probability density fZ (z) that is normalized, but
instead of summing we need to calculate the integral:
96 4 Probability and Statistics

∞ ∞ 1
Z Z
1 2
fZ (z)dz = √ e− 2 u du = 1 , (4.100)
−∞ −∞ 2π
where normalization is ensured by the prefactor.
Notice that fZ (z) is a function of n and σ and not of N anymore.

4.6 Key Concepts in Combinatorics

Both in statistics and in physics we frequently need to count configu-

rations. In statistics we need to count all the possible outcomes of an
experiment and in statistical physics we need to count all the possible
states a system can be in. It may seem that each problem you meet
needs a special approach, but usually we can use a standard trick in
problem solving — we can map a new problem we are working on onto
a problem we already know the solution to. Indeed, this is one of the
most common techniques in physics, and what maps mathematics and
physics so powerful. There are not that many different problems out
there! However, in order to use such a technique, you need to know some
of the classic counting problems. Here, we will introduce you to three
classic problems arising when you select random elements from a bag
with k balls numbered from 1 to k.
Ordered sequence with replacement. First, we draw a sequence of
numbers by drawing a random ball from the bag, noting down the
number, placing the ball back into the bag, and repeating this process
N times. How many different such sequences, (n1 , n2 , n3 , . . . , nN ), are
there when ni = 1, 2, . . . , k? This corresponds to asking how possible
outcomes are there if we throw a dice 5 times, or how many possible
outcomes are there if we flip a coin 8 times. The result is a sequence of
numbers, (1, 6, 3, 5, 1), or (H, T, H, H, T, T, H, T ), and we ask how many
such (ordered) sequences are there?
For the first number, there are k possibilities. For each such possibility,
there are k possibilities for the second number, and so on, so that the
total number of possibilities is Ω = k k˛ . . . k (N times), or Ω = k N .
There are 65 possible outcomes if we throw 5 dice, and there are 28
possible outcomes if we flip a coin.
Ordered sequence without replacement. What if we perform the same
experiment, but we do not place a ball back into the bag after it has
been selected. This means that a given number now just can occur once
4.6 Key Concepts in Combinatorics 97

Fig. 4.8 Illustration of 5 4 5

an ordered sequence with
replacement.
5 4 5

5 2 5 2 5 2
4 4 4
1 3 1 3 1 3

in the sequence. We still draw the numbers in sequence and consider the
sequence (1, 5, 3) different from (5, 3, 1). How many such sequences are
there when there are N numbers in the sequence and k different balls
in the bag? This corresponds to asking how many ways can you place
students at the various chairs in class or how many ways can ways can
we select runners for the first, second and third place in a marathon —
in all these cases the position in the sequence matters.
How many possible configurations are there of such a sequence. For
the first number, there are k possible values. However, the value that
occured in the first number, cannot occur as the second number. There
are therefore only k − 1 values available for the second number, k − 2
for the third number, etc. The number of ways to select a sequence of N
such numbers is therefore Ω = k (k − 1) (k − 2) . . . (k − (N − 1)), which
we also can write as Ω = k!/(k − N )!.

Fig. 4.9 Illustration of an 5 4 2

ordered sequence without
replacement.
5 4 2

5 2 2 2
4 4
1 3 1 3 1 3

Unordered set without replacement. Now, what if we do not care

about the order in which we draw the elements? We still draw N balls
from the bag, but we do not care about the order in which they were
selected. This means that we only care about the set of drawn elements
and not the sequence. Hence if we draw (1, 5, 3) or if we draw (5, 3, 1) this
both corresponds to drawing the set {1, 3, 5}. How many such sets can
be drawn if we draw N elements from a bag with k different elements?
This corresponds to asking how many 5-cards hands can we draw from a
full deck of cards or how many sets of possible lotto numbers there are?
98 4 Probability and Statistics

How many possible such sets are there? We know that there are
k!/(k − N )! ordered sequences. But if we count all the ordered sequences,
we count both (1, 5, 3) and (5, 3, 1) as two occurrences, even though they
are part of the same set. We have therefore counted way too many sets.
How many different sequences are there that results in the same set?
This corresponds to the number of ways we can order the N numbers
(1, 3, 5), that is the number of ordered sequences of N elements, which is
N !. We can therefore find the number of sets by dividing the number of
sequences by N !, giving us Ω = k!/(N ! (k − N )!).

Fig. 4.10 Illustration of an

2
unordered sequence with 5 4
replacement.
5 4 2

5 2 2 2
4 4
1 3 1 3 1 3

4.7 Universal distributions

The result we have found now for the Binomial distribution is general.
Indeed, the sum of any set of independent, random variables will always
follow a normal distribution, independently of the underlying distribution
of ench individual part of the sum. This is a fascinating result with wide
ramifications: The behavior of the sum is independent of the behavior of
the individual components. The collective behavior is general and not
dependent on the details. This is a general feature in statistical systems
— that behaviors are universal and independent of the details — and a
central part of physics is to discover what types of universal exist and
what universality class a given process belongs to. Here, we will look at
two types of universal behaviors: The normal distribution and extreme
value statistics.

4.7.1 Central limit theorem

The result we found for the binomial distribution is very general — it
is a result of a general theorem called the Central Limit Theorem: For
4.8 Probability distributions 99

any sequence of independent, identically distributed random variables

Xi , the sum of the variables
N
Z= (4.101)
X
Xi ,
i=1

is described by a Gaussian or Normal distribution when N becomes

large as long as the standard deviation σx of each Xi if finite. The random
variable Z has an expected value µ = N Xi and a standard deviation
σ = N σx and its probabilty density is
1
e− 2 ( σ ) .
1 Z−µ 2
fZ (z) = √ (4.102)
2πσ
This result is valid for all possible values of the underlying variables Xi
as long as they are independent and have a finite standard deviation.

4.7.2 Extreme value statistics

The central limit theorem describes the behavior of the average value
of a sequence of identical variables, Xi . However, we can also ask other
questions, such as what is the minimal or the maximum value of the
sequence of N elements. This would for example be relevant if we were
interested in the stength of a chain — the strength would be determined
by the element with the smallest strength. What is the distribution of
such elements?
We could address this numerically. We could generate a set of elements
that are uniformly distributed between 0 and 1, find the minimum of
this, and then perform this experiment many times, and characterize the
resulting distribution.
ams 1: need to write this part

4.8 Probability distributions

4.8.1 The probability density and the cumulative distribution

The probability desity fZ (z) for a random variable Z gives the probability
for Z to be in the range from z to z + dz:

P (z < Z < z + dz) = fZ (z)dz . (4.103)

100 4 Probability and Statistics

This means that the probability for Z to be smaller than z, P (Z ≤ z) =

FZ (z) is given by the sum of all possible z values smaller than z:
Z z
P (Z ≤ z) = FZ (z) = fZ (z)dz . (4.104)
−∞

The function FZ (z) = P (Z ≤ z) is called the cumulative distribution

function of Z. We can find the probability density fZ (z) from the
derivative of the cumulative distribution function:
d
fZ (z) = FZ (z) . (4.105)
dz
The probability for Z to be larger than z is then

P (Z > z) = 1 − P (Z ≤ z) = 1 − FZ (z) . (4.106)

In many cases it can be simpler to first calculate either FZ (z) or 1−FZ (z)
and then find the probability density fZ (z) through derivation.

4.8.2 Uniform distribution

The uniform distribution describes a random process with values between
a and b so that the probability P (z, z + dz) = fZ (z)dz does not depend
on the value of z as long a z is between a and b.
This means that fZ (z) = c is a constant. We can find the constant
from the normalization condition of the density:
b 1
Z
fZ (z)dz = c(b − a) = 1 ⇒ c = . (4.107)
a b−a
This distribution:
 0

 z<a
fZ (z) = 1
a≥z≤b ,
b−a (4.108)
 0

b<z
is called the uniform distribution.
The expected value a random variable Z which is uniformly distributed
from a to b is:
b b z 1 (b2 − a2 ) 1
Z Z
hZi = zfZ (z)dz = dz = = (b + a) . (4.109)
a a b−a 2 b−a 2
4.8 Probability distributions 101

4.8.3 Exponential distribution

The distribution
1
fZ (z) = e−z/c , (4.110)
c
for positive z is called the exponential distribution. It is commonly
used to describe waiting time statistics.
Normalization. We check that the distribution is normalized:
∞ 1 −z/c ∞
Z Z
e dz = e−u du = 1 . (4.111)
0 c 0

Expected value. The expected value of Z is

∞ 1 −z/c ∞
Z Z
hZi = ze dz = c e−u du = c . (4.112)
0 c 0

Interpretation of the exponential distribution. We can understand the

origin of the exponential distribution as a waiting time distribution from
a simple example: A process occurs at a constant, but very low rate r,
so that the probability for an event to occur in a time interval from t
to t + dt is a constant r multiplied with dt, rdt. What is the probability
for the event to occur for the first time in the time interval from t to
t + dt? For this to happen, the even must not occur for the time t and
then occur in the time interval from t to t + dt. We can divide the time
t into small units ∆t. The probability for an event to occur in such a
small time interval is r∆t, and there are n = t/∆t such interval from 0
to t. The probability for no event to occur from 0 to t is then
n
r·t

n
P = (1 − r∆t) = 1 − → e−rt , (4.113)
n
when ∆t → 0 and n = (t/∆t) → ∞. The probability for the event to
occur for the first time in a time interval from t to t + dt is P , which is
the probability for no event to occur before t, multiplied by rdt, which
is the probability that the event occurs in the interval from t to t + dt.
This gives the probability for the event to occur for the first time in the
inverval from t to t + dt to be:

fT (t)dt = e−rt rdt . (4.114)

We can therefore interpret fT (t) as the probability density for the waiting
time T , and we may interpret the exponential distribution as a waiting
time distribution with a rate r or a characteristic time τ = 1/r.
102 4 Probability and Statistics

Comparison with numerical simulation. We can test this theory by

generating a simple data-set for waiting time statistics. Let us assume
that a light bulb has a probability p = 1/1000 to fail during an hour.
What is the distribution of the lifetime of such light bulbs?
Let us generate a numerical experiment to measure this, and then
compare the results of the numerical simulation with the theory. First,
how can we find the number of trials needed to observe the first event,
when the probability of an even in each trial is p? We could do this using
a loop, which continues until we have achieved success, and then count
how many attempts were needed:
n = 0
while (rand(1,1)>p);
n=n+1

This is not a very efficient algorithm in a vectorized language such

as python, but we have chosen to implement it like this because the
structure of the algorithm is very clear. We then need to repeat this
experiment nsamp times and collect the results in a variable tval. This
is implemented in the following program:
#start1
from pylab import *
p = 1.0/1000.0
nsamp = 10000
tval = zeros(nsamp,float)
for i in range(nsamp):
j = 0
while (rand(1)>p):
j = j + 1
tval[i] = j

Estimating the probability density. How to estimate the probability

density for T , the waiting time? We do this following the frequency
definition of probability: By counting how many events, Nt , are in an
interval from t to t + dt. The probability for T to be in the interval
from t to t + dt is then fT (t)dt = Nt /N , where N is the total number of
experiments. The number Nt are produced in a histogram, and we can
then find the probability density through
Nt
fT (t) = . (4.115)
N dt
Notice that the interval size, dt, appears in the denominator! This factor
is often forgotten by students, but it is essential in order to find the
probability density.
4.8 Probability distributions 103

First, let us find the histogram using bins that are 1 second in width.
We do this by choosing the number of bins used to correspond to the
largest value of T observed using
n,t=hist(tval,bins=max(tval))

where the variable n now contains the counts, Nt , and t contains points
representing the bins. Since the bins all have the same size, this is a
feature of the hist function, we can find the bin size dt from the first
two values of t:
dt = t[1]-t[0]

We can finally estimate the probability density using (4.115):

Pt = n/(nsamp*dt)

Notice that this is a vectorized command, which calculates the values for
Pt for all the values in n. Essentially, a command like this corresponds to
for i in range(len(n)):
Pt[i] = n[i]/(nsamp*dt)

We can then plot the calculated probability density and compare with
the theoretical values found above:

fT (t) = pe−pt , (4.116)

which is done by
theory = p*exp(-p*t)

which again is a vectorized command that calculates the values for theory
for all values in the t-array. The results are plotted by
plot(t,Pt,’o’,t,theory,’-’), show()

and the resulting distribution is shown to the left in Fig. 4.11.

Unfortunately, this plot is really messy! It is not easy to observe the
behavior because there are too many data points and too much noise.
What can we do to improve this? We can broaden the bins, so that there
are more points in each bin – we need to change the bin size dt. For
example, we can analyze the data using only 100 bins. However, we then
need to be very careful to remember to include the bin size dt when we
estimate the probability density. This was done using
n,t = hist(tval,bins=100)
dt = t[1]-t[0]
Pt = n/(nsamp*dt)
104 4 Probability and Statistics

−3 −3
x 10 x 10
2 1

0.8
1.5

0.6
fT (t )

fT (t )
1
0.4

0.5
0.2

0 0
0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000
t [ h] t[s]

−2 −3
10 10

−4
10
−4
10

fT (t )
fT (t )

−5
10
−6
10
−6
10

−8 −7
10 10
0 2000 4000 6000 8000 10000 0 2000 4000 6000 8000 10000
t [ h] t[s]

Fig. 4.11 Plot of the measured probability denstiy fT (t) for the waiting time distribution
for a process occuring with a probabity p = 1/1000 per hour. (Left) Bin size correspoding
to the smallest possible resolution in t. (Right) With 100 bins.

theory = p*exp(-t*p)
plot(t,Pt,’o’,t,theory,’-’)

And the resulting plots are shown to the right in Fig. 4.11. The results are
now much clearer. And we can now also see that the theory fits perfectly.
The data is indeed well described by the exponential distribution.

4.8.4 Poisson distribution

The Poisson distribution appears when we look at a process that occurs
at a constant rate, so that over a time interval T , the expected number
of events is λ. In this case, we can divide the time interval into N small
intervals ∆t, so that in each time interval the probability for the event
to occur is p = λ/N . As we divide the time interval into more and more
subintervals, the probability for an event to occur in a subinterval, p,
becomes smaller and smaller, and we can therefore assume that the
4.8 Probability distributions 105

probability for more than one event to occur in a subinterval is negligible.

What is then the distribution of n, the number of event occuring during
the time interval T ?
The number of events, n, occuring in N subintervals is the sum of the
number of events, ni , occuring in each subinterval i:
N
n= (4.117)
X
ni ,
i=1

where the probability for ni to be 1 is p = λ/N and the probability

for ni to be 0 is 1 − p. This means that n is described by a binomial
distribution:
N!
P (n) = pn (1 − p)N −n , (4.118)
n!(N − n)!)
where we can now insert p = λ/N :

N!
n  N −n
λ λ

P (n) = 1− (4.119)
n!(N − n)! N N
N!
N  n
λ λ

= n
λn 1 − 1− (4.120)
n!(N − n)!N N N
1 N!
N  n
λ λ

= λn 1 − 1− , (4.121)
n! (N − n)!N n N N
where we notice that when N → ∞:
N!
N n
λ λ
 
→1 ⇒ 1− → e−λ ⇒ 1− →1.
(N − n)!N x N N
(4.122)
This gives
λn e−λ
P (n) = , (4.123)
n!
for the probability for the number n of event in an interval with an
average number of events λ, where the events are rare and independent.
The Poisson distribution describes processes where there are a large
number of possible events (trials), but where the probability of each event
is low. The distribution describes how many events occur in a given time
interval. This can for example be used to describe the number of photons
detected by a telescope or a camera in astronomy, the number of cars
arriving at a traffic light, the decay of radiactive nuclei, or the number
of mutations on a strand of DNA.
106 4 Probability and Statistics

4.9 Summary

Statistical experiment. A statistical experiment is a trial with a

given set of possible outcomes.
Outcome. A result ni of a staticial experiment is called an outcome.
Probability. The frequency based definition of the probability pi for the
event that n = x is that P (n = x) = Nx /M , where Nx is the number
of times that n = x was observed in M events, in the limit when M
becomes large.
Random variable. A random variable Z describes the result of an
experiment. The outcome of a particular experiment is z.
Probability density. The probability for a set of discrete outcomes is
PZ (z). The probability for an outcome z in the range from z to z + dz
from a continous set of outcomes is fZ (z)dz, where fZ (z) is called the
probability density.
Expected value. The expected value of Z is hZi = z zPZ (z) for
P

a discrete set of outcomes and hZi = zfZ (z)dz for a continuous set
R

of outcomes. The expected value is estimated by the average, z̄ =

(1/M ) M i=1 zi .
P

Standard deviation. The variation in Z is characterized by the variance

σ 2 and the variation E((Z − E(Z))2 ). The variance is estimated from
the standard deviation σ 2 = (1/M ) (zi − hzi)2 or σ 2 = (1/(M −
P

1)) i (zi − z̄ 2 )2 .
P

4.10 Exercises

Exercise 4.1: Deck of cards

We will address various probabilities for drawing cards from a deck of
cards.
a) What is the probability to draw an ace of spades?
b) What is the probability to draw a two of diamonds?
c) What is the probability to draw a black card?
d) What is the probability to draw a spade?
4.10 Exercises 107

e) What is the probability to draw a card that is not a spade?

f) What is the probability to draw two spades from a full deck?
g) What is the probability to draw five cards from a full deck and get
four cards of equal value?

Exercise 4.2: Dice

We are using normal, fair, six-sided dice:
a) What is the probability to get a 1?
b) What is the probability to not get a 1?
c) If we throw two dice, what is the probability that both dice show 6?
d) If we throw two dice, what is the probability that the sum is 3?
e) If we throw two dice, what is the probability that the sum is j for all
j?
f) If we throw two dice two times, what is the probability to get two
six’es at least once?
g) If we throw four dice, what is the probability to get at least two
six’es?
h) If we throw two dice, what is the probability that both dice show the
same?
i) If we throw two dice, what is the probability that the sum is odd?
And even?
j) If we throw one dice six times, what is the probability that we get at
least one six?

Exercise 4.3: Casino in St. Petersburg

In the Casino in old St. Petersburg they had a special game. You paid a
fee to participate, and then you started to flip a coin, and you continued
flipping until you got heads. If heads came up for the first time after N
flips, you won 2N rubles. You can assume that the probability to flip a
head is p.
a) Find the probability P (N ) for the coin to show heads for the first
time at flip N .
108 4 Probability and Statistics

b) Show that the probability P (N ) is normalized.

c) Find the average number of flips needed to get the first heads. where
we have introduced q = (1 − p). We use a common trick to find such
sums, by realizing that the sum can be written as
∞ ∞ ∞
d n d X d 1 q q
nq n = q =q qn = q = = 2 ,
X X
q
n=0 n=0
dq dq n=0 dq 1 − q (1 − q)2 p
(4.124)
which we insert back to get
q 1−p
hN i = 1 + p 2
=1+ . (4.125)
p p
d) Find the average return for a player in this game. Show that the
return approaches infinity unless p is larger than 1/2.
e) If p = 0.6, what must the entry fee for the game be to ensure that
the Casino does not loose money?

Exercise 4.4: Coin-flip game

Two persons are playing a simple coin-flip game. They flip a coin in turn,
and whoever first gets a heads wins the game.
a) Write a program to model M = 10000 such games. Estimate the
probability for the person who starts flipping to win the game.
b) Find the probability for the first person to flip (the person who starts
flipping) to win the game.

Exercise 4.5: Waiting time statistics

Many problems in physics are about waiting — waiting for an event to
occur. For example, we often ask questions of the type — what is the
probability for an event X to happen for the first time after N attempts.
Concepts such as mean-free-path and mean-free-time are often associated
with such processes.
As an example of a waiting time distribution problem, let us ad-
dress electrons with a narrow distribution of kinetic energies around E
(E  kT ). These electrons are injected into a material, where they have
a certain probability to the scattered. The probabilitiy for scattering
4.10 Exercises 109

depends on the number of scattering centers the electron passes. This

gives a probability p∆x to be scattered after a short distance ∆x.
a) When the electron starts from x = 0, show that the probability, P (x),
for the electron to reach x without being scattered is e−px .
b) What is the probability for the electron to be scattered in the interval
from x to x + dx?
c) How far does the electron travel on average before it is scattered?
d) What is the probability for the electron to travel for a time t without
getting scattered?
e) What is the average time the electron travels before being scattered?
This is called the average-free-time between collisions.

Exercise 4.6: Poisson statistics

We assume that you are studying a rare event such as the chance of
getting the number 1000 if you spin a wheel of fortune with 1000 numbers
from 1 to 1000. To make our results more general we call the probability
to get 1000 p.
a) If you spin N times, what is the probability to get 1000 exactly one
time.
b) What is the general probability P (N, n) to get 1000 n times in N
spins. What is the average value of n, n̄?
c) Show that the expression can be approximated as

n̄n e−n̄
P (N, n) = , (4.126)
n!
when N approaches infinity and p  1.

Exercise 4.7: Gamma distribution

Let us address an unfair coin with a probability p for heads. We flip the
coin until we have obtained n heads, which concludes the experiment.
The result of the experiment is N , the number of flips.
a) Find the probability P (N, n) that we need to flip N times to get n
heads.
110 4 Probability and Statistics

b) Show that the probability is normalized.

c) Find an expression for the average N and the standard deviation of
N.

Exercise 4.8: Stirling’s formula

We often use Stirling’s formula as an approximation for the factorial.
However, the formula is only supposed to be a good approximation for
N  1. Let us see how good the approximation is for smaller N .
Stirling’s approximation gives
√ 
1 1
 
N ! = N N 2πN exp −N + +O , (4.127)
12N N2
or
1 1 1
 
ln N ! = ln (2πN ) + N (ln N − 1) + +O , (4.128)
2 12N N2
However, we often use simpler versions of the formula, such as
√ 1
S1 (N ) = N N e−N , S2 (N ) = 2πN S1 (N ) , S3 (N ) = e 12N S2 (N ) .
(4.129)
a) Write a program to compare the values for N = 1, 2, 4, 8, 16, 32, 64.
Plot the three resulting functions and the exact values.
b) Find the relative error from using N ! ' S3 (N ) for N = 1020 .
c) How large should N be for ln S1 (N ) to represent ln N ! with a relative
error of less than 10−6 ?

Exercise 4.9: Conducting strip

An experiment may result in two possible outcomes, heads with proba-
bility p and tails with probability q = 1 − p.
a) What is the probability that N independent flips all results in heads?
b) What is the probability for n heads and N − n tails in N flips?
4.10 Exercises 111

c) From the result above, find the average number of heads, n̄ and the
variance
(n −¯ ¯)n2 . (4.130)
Now, let us consider a strip consisting of N squares as illustrated in
fig. 4.12. The squares are electrically conductive with probability p. Two
conducting squares next to each other provide a conductive path.

1 2 N

1 2 N

1 2 N

Fig. 4.12 Illustration of strip of conducting patches.

d) What is the probability for the strip to be conducting? What is the

probability for one isolating square in the strip?
Now, we study two strips next to each other, that is, 2N squares.
We assume that a conducting path can be formed both by two squares
sharing a side and when touching at a corner.
e) What is the probability, Pc , for the double-strip to be conductive?
Let us assume that we have 2N squares, of which N +M are conductive
and differ from the others by a metallic surface. The 2N squares form a
double-strip of length N .
f) What is the total number of different configurations Ωtot ?
g) What is the total number of conducting configurations, Ωcond ?
h) If we define
Ωcond
Pcond = ; (4.131)
Ωtot
and introduce p = (N + M )/(2N ), will we get the same situation as
for Pc above or is the situation different? If this is the case, explain the
difference.
112 4 Probability and Statistics

Exercise 4.10: Velocity distribution in Argon gas

Let us use the molecular simulation approach to measure and characterize
the velocity distribution in a gas.
a) The file in.gasexer01013 can be used to simulate a two-dimensional gas.
The resulting output file is found in gasexer010.lammpstrj14 . Measure
and plot the distribution of the x-component of the velocities of the
atoms in the final time-step.
b) What is the average velocity v̄x and the standard deviation of the
velocities, σv ? Comment on the results.
c) Suggest a function P (vx ) that can fit the data and check how well it
fits the measured data.
q
d) Compare the distribution of vx and the distribution of v = vx2 + vy2 .
Comment on the results. (Difficult) Can you predict the functional form
of P (v)?
e) Run a sequence of simulations for N = 200, 400, 800, 1600 atoms and
find the standard deviation, σv , as a function of N . Find a reasonable
functional form for σv (N ). What happens when N becomes realistically
large, such as N = 1023 ?
f) Run a sequence of simulations for various values of the initial velocity
(energy) used in the simulations v0 = 1.5, 2.0, 2.5, 3.0, and 3.5. Find the
standard deviation σv as a function of K/N , the average kinetic energy
per particle. Comment on the result. What does this tell you about the
temperature-dependence of the velocity distribution?

13
http://folk.uio.no/malthe/fys2160/in-gasexer010
14
http://folk.uio.no/malthe/fys2160/gasexer010.lammpstrj
 A thermodynamical example
5

Abstract We have now seen how a system approaches equilibrium using

both molecular dynamics modeling and algorithmic modeling, but what
characterizes the equilibrium of a system? Here, we briefly introduce the
thermodynamic principles of equilibrium for an ideal gas. We measure
the ideal gas law in molecular simulations and develop a simplified theory
for the equation of state for the gas. We also quickly introduce a few key
concepts that will reappear with full force later in the book.
We start with a quick introduction to thermodynamcis with emphasis
on an example – the ideal gases. Here, we will focus of on a few key
concepts of the gas — in particular on the concept of temperature.
What do we know about temperature? Not much, yet, because we have
not introduced it rigorously. We do know that temperature is what we
measure with a thermometer. However, this is not that useful. We might
expect it to be related to energy. If all the atoms in a gas move faster —
so that the average or total kinetic energy increases — we may expect
the gas to have a higher temperature.
The typical definition of temperature is that temperature is the thing
that is equal when two systems are in thermal equilibrium. If a highly
agitated gas comes in contact with a less agitated gas, we expect the two
gases to becomes equally agitated over time. We expect the temperature
to become the same over time. And in the process the system with a lot
of kinetic energy will tranfer it to the system with less kinetic energy.
However, notice that even though temperature is related to energy, it
is not the same as energy. We can see that from two though experiments:

113
114 5 A thermodynamical example

• If we increase the velocity of all particles we do not expect the

temperature to increase.
• If we lift all the particles upwards in a gravitational field, increasing
their energy, we do not expect the temperature to increase.
Temperature is not directly releated to energy, but it rather related to
the distribution of energy. One of our goals for this course is to develop
a good understanding of temperature. Here, we will start from a simple
example — how is the energy in a gas related to other properties of the
gas?

5.1 Ideal gas

We start by addressing a few real systems in more detail. First, we discuss

the properties of an ideal gas, which is going to be our model system
for many, non-interacting particles. An ideal gas is a gas that has so low
density that there are no interactions between the particles. It is simply a
box filled with non-interacting particles (atoms or molecules). We assume
that we can model an ideal gas using molecular dynamics by looking at
a system with very low density. We start from such as simulation.

5.1.1 Ideal gas — Results from simulations

We can also use the molecular dynamics simulations to learn about the
ideal gas. We can, for example, determine how the pressure in the gas
depends on other properties of the system?
The pressure is a property we can measure in the gas and which may
vary or at least fluctuate with time — just as we have seen the number
of particles in the left half to vary. The volume and the total energy on
the other hand, are values we determine when we start the simulations,
and we typically do not allow them to vary throughout the simulation:
Total energy is conserved when the particles move in conservative force
fields, and we keep the size of the box, and hence the volume, constant.
If the gas is very dilute, all the particles are far away from each other
most of the time, and the interaction energies from the interatomic
potentials will be very small (since it decays rapidly with the distance
between atoms). We may therefore approximate the total energy by the
kinetic energy of the atoms instead. We will come back later to how to
5.1 Ideal gas 115

measure the pressure in a molecular dynamics simulation, but let us here

assume that it is measured from the average force on the walls of the
system. We introduce reflective walls in both the x and the y direction
to contain the system.
We can set up a two-dimensional simulation of a dilute gas, just as
we did before, and measure the volume V = Lx · Ly , the pressure P , as
measured by the simulation program, and the total kinetic energy
N
1  
K= + vyi (5.1)
X
2 2
mi vxi ,
i=1
2

which is also measured directly in the simulation program.

To characterize the behavior of a gas, we perform simulations and
record measurements for various values of the initial velocities of the
system — which determines the initial energy of the system — and for
various values of the volume V of the system. We do this by changing the
input file used by the simulation program. We then run the simulation,
and extract the volume from the actual size of the box used in the
simulation. We measure the pressure in the simulation and we measure
the total kinetic energy, K, in the final time-step of the simulation. This
is implemented in the program findeqstategas01.py1 :
from pylab import *
import re
pressarr = array([]) # Store pressures
volarr = array([]) # Store volumes
Karr = array([]) # Store kinetic energies
myvelocities = array([1.5,2.0,2.5,3.0,3.5])
myvolumes = array([0.010, 0.020, 0.040, 0.080])
for ivel in range(0,size(myvelocities)):
for ivol in range(0,size(myvolumes)):
# Change the word mydensity to myvolumes[ivol]
infile = open("in.gasstatistics30",’r’)
intext = infile.read()
infile.close()
replacestring = "%f" % (myvolumes[ivol])
intext2=intext.replace(’mydensity’,replacestring)
# Change the word myvelocity to myvelocities[ivel]
replacestring = "%f" % (myvelocities[ivel])
intext3=intext2.replace(’myvelocity’,replacestring)
infile = open("in.tmp",’w’)
infile.write(intext3)
infile.close()
# Run the simulator
print "Executing lammps < in.tmp"
1
http://folk.uio.no/malthe/fys2160/findeqstategas01.py
116 5 A thermodynamical example

os.system("lammps < in.tmp") # Run lammps

# Extract data from trajectory of simulation
d = dump("tmpdump.lammpstrj") # Read sim states
tmp_time,simbox,atoms,bonds,tris,lines = d.viz(0)
dx = simbox[3]-simbox[0]
dy = simbox[4]-simbox[1]
vol = dx*dy # Volume of box
t = d.time(), n = size(t)
# Calculate total kinetic energy of last timestep
vx = array(d.vecs(n-1,"vx"))
vy = array(d.vecs(n-1,"vy"))
K = 0.5*sum(vx*vx+vy*vy) # Sum of kinetic energy
# Read pressures calculated in simulation
l = logfile("log.lammps")
# Find pressure averaged over all timesteps
press = average(l.get("Press"))
# Store calculated values in arrays
pressarr = append(pressarr,press)
volarr = append(volarr,vol)
Karr = append(Karr,K)
# Plot the results
pvarr = pressarr*volarr
plot(Karr,pvarr,’o’),xlabel("K"),ylabel("PV"),show()

Notice the use of regular expressions to change the volume (really

density) and initial velocities in a default file, and write the resulting
text into a new file which is read by the simulation. Notice also how we
can run a program from the command line from python. This script runs
a sequence of 20 molecular dynamics simulations, and the result is shown
in Fig. 5.1.
The plot shows P V as a function of K. It is not easy to know how to
plot such data to find general relationships — often we have to make
guesses and test them out – but you might have guessed it by dimensional
analysis: We see that both P V and K have units of energy. However,
when we plot the data in this way, we see that P V is approximately
linearly dependent on K — indeed they are directly proportional with a
factor of one:
PV = K , (5.2)
We may rewrite K as N (K/N ), where N is the number of atoms in the
simulation. The kinetic energy per particle may be used as a measure
of how energetic the gas is, and we could define the this to be the
temperature, or proportional to the temperature with some factor of
proportionality kB to ensure that temperature has reasonable values for
the melting point of water and the boiling point of water, since these are
important scales for us. We therefore, for now, introduce the temperature
T through K/N = kB T , where kB is called Boltzmann’s constant:
5.1 Ideal gas 117

kB = 1.3806488(13) · 10−23 J/K = 8.6173324(78) · 10−5 eV/K . (5.3)

We can therefore rewrite the equation of state for the two-dimensional

dilute gas as
P V = N kB T , (5.4)
This form of the result is general and true also in one and three dimensions.
Indeed, what we have found here is the ideal gas law, or the equation
of state for the ideal gas. An ideal gas is a gas that is so dilute there
are no collisions (no interactions) between the atoms.

Fig. 5.1 Plot of the his- 1600

togram for n(t) from the

1400
simulation and ni from a
binomial distribution. 1200
PV

1000

800

600
600 700 800 900 1000 1100 1200 1300 1400 1500
K

5.1.2 Ideal gas — Results from theory

We have now observed the ideal gas law in our simulations. Can we also
develop a theoretical understanding and a theoretical argument for the
ideal gas law? We can do that by making a simplified, microscopic model.
Let us start by addressing the motion of a single atom bouncing in
a volume V = L3 . How can we find the pressure P on the walls of the
container?
We can find the force on the wall from the change in momentum of
the particle as it bounces off the wall. We assume that each bounce is
conservative, and that the only change in momentum is in the direction
normal to the wall. The velocity in the direction normal to the wall is
reversed in each collision: vx → −vx . We can use this assumption to find
the force acting on the particle – and from that we can find the reaction
force which is acting on the wall. The force on the wall is FX = m∆vx /∆t,
where ∆t is the time interval we are averaging over. The typical time
between two collisions corresponds to the time between each time the
particle hits the wall. This is twice the time it takes for a particle to
travel from one side and to the other side of the box. We use this as the
118 5 A thermodynamical example

typical time between collisions: ∆t = 2L/vx . In addition we know that

∆vx = 2vx , since the partilces reverses its velocity in each collision. This
gives
F −m(−2vx ) mvx2 mvx2
P = = = = . (5.5)
A A2L/vx AL V
Similarly, for the other particles we find:

P V = N mvx2 . (5.6)

We recognize this from the ideal gas law, which we could measure, which
gives P V = N kT . Therefore we have related temperature to the velocity
of a particle:
1 1
kT = . (5.7)
2 mvx2
We find similar results in the y and z direction, so that in total we have
3
K = kT . (5.8)
2

5.1.3 Ideal gas — equipartition principle

The result we found above:
1
hEk,x i = kT , (5.9)
2
is more general, as we will prove later, and it is valid not only for
the kinetic energy, but for all terms in the energy that are quadratic
in position or velocity. We will call this principle the equipartition
principle:

Equipartition principle: In equilibrium, every (quadratic) term

in the sum for the energy of a system has an energy which is
(1/2)kT

ams 2: We may here discuss vibration and rotation.

This simple principle can be applied to provide insight into the two
fundamental model systems for gases and solids:
Simplfied model for a gas — ideal gas. An ideal gas where there
are no interactions between the atoms/molecules. This means that the
5.2 Summary 119

particles only have kinetic energy and no (internal) potential energy due
to interparticle interactions. The total number of quadradic terms in the
energy sum is therefore 3N — there are three quadratic terms for each
particles. The equipartition principle therefore tells us that
3
U = N kT , (5.10)
2
If there are more degrees of freedom, such as from rotation or vibrations
of molecules, these must also be included by including f , the number of
degrees of freedom into the expression, and the expression for the energy
of the gas becomes
f
U = N kT , (5.11)
2
where f is:
• f = 3 if the only degree of freedom per particle are motion along the
three orthogonal directions.
• f = 5 if we include rotation in a diatomic gas. (Two new degrees of
freedom to describe rotation).
• f = 7 if we also include vibration in a diatomic gas. (Two additional
degrees of freedom to describe vibration).

Simplified model for a solid — ideal solid. In the simplest model for a
solid, each particle in the solid can move in three directions. A solid with
N atoms therefore have 6 degrees of freedom for each atom, three degrees
of freedom for their positions relative to their equililibrium positions and
three degrees of freedom for their velocities. This gives: 3

U = 3N kT , (5.12)

for a solid.

Clickers - Equipartition

5.2 Summary

• An ideal gas is described by the ideal gas law P V = N kT .

• Each degree of (quadratic) freedom for a system contributes an amount
(1/2)kT to the total energy in equilibrium.
120 5 A thermodynamical example

5.3 Exercises
 Micro- and Macro-states
6

We have seen that a gas develops towards a state where the number of
particles is uniform throughout the gas. We measured the number of
particles on the left side of the gas, and found that the system naturally
evolves towards a state with a uniform distribution. While the number of
particles in one half of the gas is fluctuating, the fluctuations becomes very
small compared with the number of particles, and the number of particles
increases. The behavior we observed for the gas is not a special case, but
a very general behavior in all systems with many particles — the system
will develop towards an equilibrium state. So far, we have only analyze
the number of particles — or the density — in a part of a system. In
this chapter we will extend this analysis to address thermal fluctuations,
fluctuations in energy, mechanical fluctuations, fluctuations in volume,
and density fluctuations, fluctuations in the number of particles in a
system.
Our plan is to follow the same strategy as we introduced for the gas:
We will study an isolated system — in molecular dynamics the number
of particles, the total energy and the volume is constant, which means
that the system is completely isolated. To understand what happens
inside this isolated system we will divide the system in two, and study
how the energy, the volume or the number of particles are distributed
between the two halves, just like we have done for the gas system.
Already for the gas, we saw that the system evolved towards an
equilibrium state, which corresponded to the most probable value for n,
and that the development was irreversible when we started from a state
that was far from the equilibrium state. Here, we will develop a better

121
122 6 Micro- and Macro-states

understanding for irreversible processes and the associated fundamental

law, the second law of thermodynamics. We will also study the conditions
for the equilibrium state.
However, the most important new concepts introduced in this chapter
occurs in the coupling between the microscopic and the macroscopic. We
will demonstrate how we can use our microscopic models to determine the
macroscopic behavior of systems. We will be able to predict quantitative
features of macroscopic systems such as the equation of state for an ideal
gas or the heat capacity for a gas or a solid. This is made possible by a
coupling from the microscopic to the macroscopic — a major achievement
in physics. Using this coupling we will introduce the concepts of entropy,
temperature, pressure, and chemical potential, which are the essential
components of thermodynamics — the theory of the thermal behavior of
macroscopic objects.
We start from our description of a gas — a description of both the
microscopic and the macroscopic state.

6.1 States of the model gas

We described the gas on several levels: In the molecular dynamics model

we described the detailed dynamics of all the particles. But this was too
detailed. Instead, we introduced a simplified model where we described
the position of each particle using a binary number, ni , which shows if
the particles is on the left side (ni = 1) or not (ni = 0). We call such
a description a microscopic description, and we can describe the state
of the system by providing the set of ni values, (n1 , n2 , n3 , . . . , nN ). We
call this set of numbers a microstate for the gas.
Given the microstate, that is the set of ni for every particle i, we could
calculate the number of particles on the left side, n = i ni . We call the
P

value of n a macrostate of the system. We found that there are many

possible microstates that give the same result for n, that is, there are
many possible microstates for a given macrostate.

6.1.1 Micro- and macro-states for two dice

If we throw two six-sided dice, the set of possible outcomes is the set
of pairs or numbers, (1, 1), (1, 2), (1, 3) etc. representing the number on
dice 1 and dice 2 respectively. That is, the number of possible states
6.1 States of the model gas 123

of the two-dice system is given by (n1 , n2 ), where n1 = 1, 2, . . . , 6 and

n2 = 1, 2, . . . , 6. These are the microstates of the two-dice system. How
many microstates are there? This is problem from combinatorics and for
two independent dice, there are 6 × 6 = 36 such (ordered) two-number
states.
The macrostates of this system represent a group of microstates. There
are many possible ways to divide the total set of outcomes into various
groups — various macrostates. For example, we could introduce a set
of macrostates corresponding to the sum, n = n1 + n2 , of the two dice.
In this case, the macrostate n can vary from 2 to 12. Each unique
value of Z represents a macrostate. But there can be many microstates
corresponding to a macrostate.
We call the number of microstates that give the same macrostate the
multiplicity of the macrostate. For the two-dice system, all the microstates
— all the possible pairs of numbers (n1 , n2 ) are equally probable since
the dice are fair and the two throws are independent of each other. The
multiplicity of macrostate can therefore be used to find the probability
of a macrostate: The probability of the macrostate is the number of
microstates in the macrostate divided by the total number of microstates.
If we introduce the notation Ω(n) for the multiplicity, then the probability
for state n is
Ω(n) Ω(n)
P (n) = = P12 , (6.1)
ΩT OT n=2 Ω(n)
where we have used that the total number of microstates is the sum of
the number of microstates for each of the possible values of n. Fig. 6.1
illustrates the number of states for each of the macrostates, and you
should now be able to find the probability for each macrostate simply by
counting the number of microstates.

6.1.2 Definition of micro- and macro-states

A microstate is a microscopic description of the degrees of freedom
of the system on some level of precision. Typically, but not always,
we can specify the microstate by specifying the state of each of the
N elements in the system, (n1 , n2 , . . . , nN ).
124 6 Micro- and Macro-states

6
6 7 8 9 10 11 12

5 6 7 8 9 10 11
4
4 5 6 7 8 9 10

N (n)
n2
3 4 5 6 7 8 9
2
2 3 4 5 6 7 8

1 2 3 4 5 6 7
0
1 2 3 4 5 6 0 5 10 15
n1 n
Fig. 6.1 Left Illustration of the microstates of two dice. The microstates that are part of
the same macrostate n = n1 + n2 are given the same color. Right Plot of the histogram
of the number of microstates for each macrostate n for two dice.

A macrostate is a set of microstates. Usually, the macrostate is

specified by a number which can be computed directly from the
microstate. We call this number the macrovariable. Each microstate
belongs to one and only one macrostate. The set of all possible
macrostates therefore span all the possible microstates of the sys-
tem.

The multiplicity Ω(N, z) of a macrostate described by the

macrovariable Z, is the number of microstates in the macrostate
characterized by the value z.

6.1.3 Multiplicity of the macrostate of the gas

We have already studied a particular macrostate of the ideal gas: The
number of particles, n, on the left side of a box. For each possible value
of n we found that there were an enormous amount of microstates. In
order to make our concepts and the notation very clear, we introduce a
more consistent notation: In our theoretical description of the system,
the microstates was a sequence of numbers describing the position of
each atom. Each atom may be on the left side (ni = 1) or on the right
side (ni = 0) for i = 1, . . . , N . The total number of atoms on the left
side was
6.1 States of the model gas 125

N
n= (6.2)
X
ni ,
i=1

We found that the probability distribution for n was given by the binomial
distribution
P (N, n) = Ω(N, n)2−N , (6.3)
where the multiplicity Ω(N, n) represents the number of microstates
that give the value n for n. We call this the number of microstates in
macrostate n.
Sharpness of the multiplicity. The multiplicity for the number of par-
ticles on the left side of an ideal gas was given as
!
N
Ω(N, n) = . (6.4)
n

We also found that this probability, and hence also the multiplicity, was
very sharp around its average value. Indeed, the average value n̄ and the
standard deviation σn are given as
s
N N
n̄ = , σn = . (6.5)
2 4
From this result we see that the standard deviation becomes very small
compared with the average:
√
σn N
= = N −1/2 , (6.6)
n̄ N
which means that for a litre of gas at room temperature and atmosphere
pressure, where N is of the order of 1023 we see that the standard deviation
is 10−10 of the average value, which is well beyond the resolution of most
measurement methods. This means that for all practical purposes the
number of particles n is a constant.
Most probable macrostate. For the ideal gas system, we expect the
system to cycle through all possible microstates: They are all equally
probable. However, we also found that the initial state where all the
atoms were on the left hand side did not appear spontaneously. Why? In
principle, all values of n are possible. In practice, there are most states
near the average value of n. So many more states that if we move a few
standard deviations away from the average, which is 10−11 of the average
value, the probability to find such a state becomes negligible. This is the
126 6 Micro- and Macro-states

reason why we find the return to the intial state comic — it is extremely
unlike to happen.
The development of the system away from n = N , where we started the
simulation previously, towards n = N/2 is therefore effectively irreversible,
because there are so many more states near n̄ = N/2 than near n = N or
n = 0. This also means that the system develops towards a macrostate
where the number of particles per volume (the number density) is the
same on each side (if the two sides are equally large). The system develops
towards a homogeneous density.
Fluctuations in particle number. When we study the number of par-
ticles on the left side of the gas, we are addressing the variations, of
the fluctuations, in the number of particles inside the system. Alterna-
tively, we could say that there are fluctuations in the number of particles
per unit volume, since the volume does not change, that is, there are
fluctuations in the density of the gas.
These fluctuations are related to transport of particles: If the number
of particles on the left side changes by ∆n: n(t + ∆t) = n(t) − ∆n,
the number on the right hand side increases by the same amount: The
particles ∆n are transported from the left hand side to the right hand
side. This can occur because the systems are open to particle transport:
There is no wall between the two halves.
Other fluctuations. Ok, we have a good description of the fluctuations
of the number of particles, or the density, inside an isolated gas. However,
we may be interested in variations in other macroscopic properties as
well. For example. if the two halves of the system can transfer energy
between each other, what are the fluctuations in energy in the two halves
of the system? However, if we want to address fluctuations in energy, we
would like to single out this fluctuation alone. We would therefore like to
have a system where the number of particles on each side of the system
is the same — identically the same — but where the total energy can be
distributed in many ways between the two parts of the isolated system.
This is the motivation to start working with other model system such as
solids, where the particles do not move around significantly, but energy
is free to flow throughout the system. We will therefore first address the
fluctuations of energy in a simplified solid — in the Einstein crystal.
6.2 Behavior of a solid — Observations 127

Fig. 6.2 Visualization of a solid of Argon atoms modelled using the Lennard-Jones
potential at two different times. The colors indicate the velocities in the x-direction.

6.2 Behavior of a solid — Observations

For a solid, the atoms are approximately fixed in space, but still vibrating.
For this system energy is free to flow, but particles will remain in place.
The solid is therefore well suited to study fluctuations and transport of
energy.
Running a molecular dynamics simulation of a solid. Let us first start
from a molecular model of a solid. We can address the behavior of a
solid using the same model we used for the Argon system, but with
smaller initial velocities and a smaller volume so that we ensure that the
system is in its solid phase. You find reasonable input parameters for a
simulation in in.solidstat021 :
# 2d Lennard-Jones solid
units lj
dimension 2
atom_style atomic
lattice hex 1.05
region box block 0 20 0 10 -0.1 0.1
create_box 1 box
create_atoms 1 box
mass 1 1.0
velocity all create 0.5 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nve
dump 1 all custom 100 solidstat02.lammpstrj id type x y z vx vy vz
thermo 100
run 500000
1
http://folk.uio.no/malthe/fys2160/in.solidstat02
128 6 Micro- and Macro-states

You can run the simulation using

lammps < in.solidstat02

The resulting atomic configuration is shown in Fig. 6.2. The atoms are
organized in a regular lattice, a triangular lattice, which corresponds
to the (equilibrium) crystal structure of the two-dimensional system.
(It should here be noted that a two-dimensional system is somewhat
unphysical, but it is simple to visualize. You can easily run all simulations
in 3d without changing any of the general aspects of the results).
Measuring energy in the solid. Based on these simulations, we can
measure the kinetic energy of the particles on the left side of the model.
This does not represent the total energy for a crystal, since the potential
energy part is non-negligible. But it does give a picture similar to what
we had for the gas simulations. We read the data from the simulations
into python, find all the particles on the left side, find the sum of the
kinetic energies of all these particles, and plot the result using the script
solidplotkt02.py2 :
from pylab import *
d = dump("solidstat02.lammpstrj") # Read output states
t = d.time()
n = size(t)
Ktot = zeros(n,float) # Kinetic energy
# Get information about simulation box
tmp_time,box,atoms,bonds,tris,lines = d.viz(0)
halfsize = 0.5*box[3] # Box size in x-dir
# Loop over all timesteps
for i in range(n):
x = array(d.vecs(t[i],"x"))
vx = array(d.vecs(t[i],"vx"))
vy = array(d.vecs(t[i],"vy"))
# Find list of all atoms in left half
jj = find(x<halfsize)
k = sum(0.5*(vx[jj]*vx[jj]+vy[jj]*vy[jj]))
Ktot[i] = k
plot(t,Ktot),xlabel(’t’),ylabel(’n(t)’),show()

Fluctuations in the kinetic energy. The kinetic energy K(t) as a func-

tion of time t is shown in Fig. 6.3. This plot looks similar what we saw
for n(t) for the gas — there are fluctuations around a clear average
value. We can characterize the size of the fluctuations by looking at the
probability distribution P (K) of the kinetic energy of the left-hand half
of the system, which is also shown in Fig. 6.3. This plot suggests that
2
http://folk.uio.no/malthe/fys2160/solidplotkt02.py
6.2 Behavior of a solid — Observations 129

the fluctuations also may by described by a Gaussian distribution. We

will look more closely at this as we develop a theory for the behavior in
the following, when we introduce the ideal crystal model for the behavior
of a solid.

65 400

60
300
55

N (K)
K(t)

50 200

45
100
40

35 0
0 1 2 3 4 5 30 40 50 60 70
t 5 K
#10
Fig. 6.3 Left Plot of the kinetic energy of the left-half of the Argon solid. Right Histogram
of the kinetic energy K for the Argon system.

Fig. 6.4 Plot of K(t) for 700

in.solidstat10
added velocity (Black) and 600 in.solidstat11

for the control simulation 500

K(t)

with no change (Red).

400

300

200
0 100 200 300 400 500 600 700 800 900 1000
t

Towards equilibrium of the solid. We can also use this simulation

framework to address how the system approaches equilibrium. We start
a simulation where the left half of the system have a different average
initial kinetic energy than the right half of the system. This is done by
the input file in.solidstat103 :
# 2d Lennard-Jones gas
units lj
dimension 2
atom_style atomic
lattice hex 1.05
region box block 0 40 0 20 -0.5 0.5
create_box 1 box
create_atoms 1 box
3
http://folk.uio.no/malthe/fys2160/in.solidstat10
130 6 Micro- and Macro-states

region 2 block 0 20 0 20 -0.5 0.05

group leftatoms region 2
mass 1 1.0
velocity all create 0.5 87287
velocity leftatoms create 1.5 282772
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nve
# Define output to file
dump 1 all custom 1 ss10.lammpstrj id type x y z vx vy vz
thermo 10
run 1000

The resulting plot of K(t) is shown in Fig. 6.4. Again, we see that the
system approaches a stationary state, corresponding to an equilibrium
state, after a short transient. The system has a clear direction of time.
What characterizes the equilibrium state? To answer this we need to
develop a theory for the behavior of the solid.

6.3 Behavior of a solid — Theory

Let us now build a simplified model of the solid, inspired by the model for
the ideal gas. Fig. 6.5 illustrates atomic arrangements in a solid, liquid
and gas system. In the simulated solid, and in a real crystal where all
the atoms are spaced regularly on a lattice, the atoms are fixed into a
position on a lattice through the interactions with the other atoms in the
crystal. The lattice configuration is a stable configuration as long as none
of the atoms gain enough energy to break out of the local potential well
they are situated in. A reasonable description of such a system would be
that each atom vibrates around the minimum in an energy landscape,
where the potential energy is due to the interactions with the other atoms.
For simplicity, we can assume that the atoms do not directly interact,
they only interact through their average potential field, but the potential
does not vary as the atom vibrates. In this simplified picture, which is
our theoretical approximation to the system, each atom in the lattice is
affected by a local potential, and we can describe the vibrations around
the minimum of this potential by the lowest-order approximation, as
illustrated in Fig. 6.5, where each atom is affected by a set of springs,
and therefore acts as if it affected by a spring force. This corresponds to
the most basic vibration system — the harmonic oscillator.
6.3 Behavior of a solid — Theory 131

Solid Liquid Gas

Model solid Model gas

Fig. 6.5 Snapshots from simulations of a solid, a liquid, and a gas using the molecular
dynamics model. Illustrations of the corresponding model systems.

6.3.1 Einstein crystal model

Our model for an ideal crystal therefore contains the following compo-
nents:
• The atoms in the crystal do no interact directly
• Each atom acts as a harmonic oscillator in a three-dimensional poten-
tial
• The total system consist of N such atoms
• The atoms in the system share the total energy in the system
This model is called the Ideal crystal or the Einstein crystal. This
model is going to be our basic model for a solid, and it is going to be
equally versatile as the ideal gas description of a gas.
Harmonic oscillator system. For a three-dimension system, each atom
consists of three independent oscillators in the x, y, and z directions so
that a system with N oscillators really only consists of N/3 atoms. Each
harmonic oscillator is subject to a potential
1
V (x) = kx2 . (6.7)
2
132 6 Micro- and Macro-states

where k is the “spring constant” and x is the deviation from the equilib-
rium position.
Quantized harmonic oscillator. From quantum mechanics we know
that the energies of such a system is quantized, with possible values

 = hν n . (6.8)

where n = 0, 1, 2, .. can only take integer values.

Energy in a system of harmonic oscillators. An Einstein crystal is a
system consisting of N such independent oscillators, where each oscillator
has the same ν and therefore the same energy levels. What are the possible
states of such a system of N oscillators? It depends on the total energy
E available to the system. We measure the energy in units of hν = ,
and we often use the symbol q for this dimensionless energy. Notice that
the number q can only take integer values.
Four-oscillator system. Let us address a simplified system with N = 4
and q = 2. What are the possible states of the whole system under
these conditions? We describe the states using a simple illustration where
we have drawn the possible energy levels for the N = 4 oscillators in
Fig. 6.6. There are generally two possibilities: (case A) One oscillator
may be in energy level 2, and the other in energy level 0; or (case B)
two oscillators may be in energy level 1 and the others in energy level
0. But how many possible ways can this occur in? We can describe the
possible configurations by a sequence (n1 , n2 , n3 , n4 ) where ni = 0, 1, 2, . . .
describes the state of oscillator i. For case A, the oscillator in energy level
2 can be placed in N = 4 possible places: (2,0,0,0), (0,2,0,0), (0,0,2,0),
and (0,0,0,2). There are therefore 4 possible states of the system with one
oscillator in energy level 2 and the rest in energy level 0. For case B, the
two oscillators in level 1 can be placed in 6 possible configurations. You
may be tempted to use the following argument to count the number of
possible configurations: We can choose the first oscillator from 4 positions
and the second from 3 positions, hence the total number of possibilities
are 4 · 3 = 12. However, this would be wrong, since we have then counted
all the actual configurations twice. If we place the first energy unit in
oscillator one and the second in oscillator two, we get the state (1,1,0,0),
but this is the same state we would get if we place the first energy
unit in oscillator two and the second in oscillator one: (1,1,0,0). The
total number of possible states are therefore only 4 · 3/2 = 6: (1,1,0,0),
(1,0,0,1),(0,0,1,1),(1,0,1,0),(0,1,0,1), and (0,1,1,0).
6.3 Behavior of a solid — Theory 133

q q
3 3
2 2
1 1
0 0
i=1 i=2 i=3 i=4 i=1 i=2 i=3 i=4

n1=0 n2=2 n3=0 n4=0 n1=0 n2=1 n3=0 n4=1

(0,2,0,0) (0,1,0,1)

| OO | | | O | | O
Fig. 6.6 Illustration of the Einstein crystal model.

Energy level diagram. It is customary to illustrate the states in an

energy level diagram. Fig. 6.6 shows an illustration of the state (0,2,0,0)
and (0,1,0,1). Another way to think about what states are possible is to
think of q as the total number of energy units and N as the total number
of boxes they can be placed in. The two states (0,2,0,0) and (0,1,0,1) is
also illustrated in this way in Fig. 6.6 – by the number of energy units
shown as balls in the boxes.
Microstates in the Einstein crystal. We call each sequence
(n1 , n2 , . . . , nN ) a microstate of the Einstein crystal. The energy
level or box representation in Fig. 6.6 therefore provides a basis for
counting the number of possible microstates for a given set of oscillators
N and a given total energy q of the system. However, there is an
even better way to represent the states that makes counting simpler,
illustrated in the bottom of Fig. 6.6. We can represent the microstate
by a sequence of energy units, 0, and walls, 1. If we only include the
internal walls (the outer walls of the boxes cannot change position) we
can list all possible microstates by listing all possible orderings of N − 1
walls and q energy units. The two states (0,2,0,0) and (0,1,0,1) then get
134 6 Micro- and Macro-states

the following representation 10011 and 10110. Now, we have mapped

the problem onto a problem we already know the answer to. (This is
a very common method in physics — to find some smart way to map
an unsolved problem onto a well-known and solved problem). We know
how to count how many possible configurations we can have of q zeroes
and N − 1 ones: This corresponds to the number of ways we can place q
zeros in q + N − 1 positions. The number of microstates for the Einstein
crystal with q energy units and N oscillators is therefore given as

(N − 1 + q)!
!
N −1+q
Ω(N, q) = = , (6.9)
q q!(N − 1)!

We call this the multiplicity of the Einstein crystal with N oscillators

and energy q.

Question
Clickers: Einstein crystal 1

6.3.2 Two Einstein crystals in contact

Now we know how to count microstates. But with only a single Einstein
crystal, the macrostate, the energy, q, in this case, is not that interesting.
In the numerical simulations, we saw how the energy was fluctuating
between two halves in a crystal. Can we apply the theory we have
developed of the microstates in a model crystal to address this behavior?
Fig. 6.7 illustrates the kinetic energy in the molecular dynamics system.
We divide the system into two parts, or two subsets of the oscillators, A
and B. For example, we may choose part A to be the left half and part
B as the right half as illustrated. We can now measure the energy, qA ,
directly, and calculate the energy in system B, qB = q − qA . How can we
make a model of the exchange between system A and B?
In our simulations, the two parts of the solids were connected. In the
model, there are no direct interactions between the oscillators. However,
all the oscillators in a crystal share the total energy: If one oscillator in a
crystal gets more energy, another oscillator gets less. Now, let us model
systems A and B as two separate Einstein crystals with NA and NB
oscillators and qA and qB units of energy in system A and B respectively.
Given NA , qA we have found how to calculate the multiplicity of system
6.3 Behavior of a solid — Theory 135

A, and similarly for system B. However, we will also assume that the
whole system (A and B together) is isolated so that the total energy, the
total volume and the total number of particles is constant — this was
also the underlying assumption for the molecular dynamics simulations.
Multiplicity of macrostate qA , qB = q − qA . The total system therefore
consists of two Einstein crystals sharing the total energy q so that
q = qA + qB . The macrostate of this system is the energy qA in system A.
This is similar to the gas system, where the macrostate was the number
of particles on the left side. Our plan is now to find the probability of
a macrostate by counting the number of microstates for macrostate qA
and divide by the total number of microstates. But how can we find the
multiplicity of macrostate qA of the whole system?

Fig. 6.7 Illustration of

two parts A and B of an
Einstein crystal.

A B

qA + qB = q

NA + NB = N

First, we notice that of the system is in macrostate qA for system A, it

must be in macrostate qB = q − qA in system B. It is therefore sufficient
to provide qA to describe the macrostate of the whole system. We already
know how to find the number of microstates for Einstein crystals A and B
respectively. What about the combined system? We can now use another
assumption about the system: System A and system B are independent
of each other. This means that if I know the microstate of system A,
which must be consistent with an energy qA , I cannot predict what the
microstate of system B is, except that it must be consistent with the
energy qB . This means that for each microstate in A, system B can be in
any of its microstates. This means that the total number of microstates
136 6 Micro- and Macro-states

is the number in system A multiplied by the number in system B. If

we call the multiplicity of system A, ΩA (NA , qA ), and the multiplicity
of system B, ΩB (NB , qB ), then the multiplicity of the whole system in
macrostate qA is Ω = ΩA (NA , qA ) ΩB (NB , qB ).

Question
Clickers - Einstein crystal 2

We can calculate the multiplicities ΩA , ΩB and Ω for all the possible

values of qA numerically. For example, if NA = NB = 5, and q = 10,
the the possible values of qA are 0, 1, 2, . . . , 10. The multiplicities are
calculated and plotted in the following script.
# Einstein solid NA - NB multiplicity
from pylab import *
from scipy.misc import comb
NA = 100
NB = 100
q = 200
N = NA + NB
# Find multiplicity of all macrostates
omegaA = zeros(q+1)
qAvalue = zeros(q+1)
omegaB = zeros(q+1)
omegaTOT = zeros(q+1)
# Loop through all macrostates and find multiplicity
for istate in range(0,q):
qA = istate
qAvalue[istate] = qA
omegaA[istate] = comb(qA+NA-1,qA)
qB = q - qA
omegaB[istate] = comb(qB+NB-1,qB);
omegaTOT[istate] = omegaA[istate]*omegaB[istate];
plot(qAvalue,omegaTOT,’-o’)
xlabel(’q’),ylabel(’W’)
show()
#
#matrix = [omegaA omegaB omegaTOT]

The resulting number of microstates are show in Fig. 6.8 and in the
following table. We notice that the the multiplicity Ω = ΩA ΩB has a
maximum for qA = q/2 = 5. But how can we use the multiplicities to
find the probability of a macrostate?
6.3 Behavior of a solid — Theory 137

Fig. 6.8 Plot of Ωtot as 2

#104
a function of qA for an
two Einstein crystals A 1.5

and B in contact with

+
1
NA = NB = 5 and q =
qA + qB = 10. 0.5

0
0 1 2 3 4 5 6 7 8 9 10
qA

qA qB ΩA ΩB Ωtot
0 10 1 1001 1001
1 9 5 715 3575
2 8 15 495 7425
3 7 35 330 11550
4 6 70 210 14700
5 5 126 126 15876
6 4 210 70 14700
7 3 330 35 11550
8 2 495 15 7425
9 1 715 5 3575
10 0 1001 1 1001

Fundamental assumption of statistical mechanics. To find the prob-

ability of a macrostate, we need more information about the microstates.
Are the microstates equally probable, as we assumed for a dice, or are
there some microstates that are more probable than others? We need
another assumption about the microstates — we need the fundamental
assumption of statistical mechanics:

Fundamental assumption of statistical mechanics: In an iso-

lated system in thermal equilibrium, all accessible microstates are
equally probable.

We cannot prove this assumption here, but we can understand where

it comes from: At the microscopic level we expect any process that takes
the system from state X to Y to be reversible, so that the probability for
the reverse process is the same. This assumption, or aspect of physics,
is called the principle of detailed balance. Another idea is that
the system, as it is evolving in time like in our simulation, will span
all possible microstates if you just wait long enough. For now, we will
simply assume that the fundamental assumption is correct, and that all
microstates have the same probability.
138 6 Micro- and Macro-states

Probabilities of the macrostate qA , qB = q − qA . Now we can find the

probability of a macrostate by simply summing the probabilities of all
the microstates in the macrostates. How many microstates are there in
a macrostate? This is the multiplicity ΩA ΩB of the macrostate. What
is the probability of a microstate? It is p = 1/Ω, where Ω is the total
number of microstates. And what is the total number of microstates?
This is the number of microstates for each possible value of qA , which
also corresponds to the number of microstates in the whole system, which
the multiplicity of an Einstein crystal with N oscillators and q units of
energy:
q
Ω= ΩA (NA , qA ) ΩB (NB , qB ) = Ω(N, q) . (6.10)
X
|{z}
qA =0 q−qA

The probability for macrostate qA is therefore

ΩA (NA , qA ) ΩB (NB , q − qA )
P (qA ) = Pq . (6.11)
qA =0 ΩA (NA , qA ) ΩB (NB , q − qA )

The probability of macrostate qA is proportional to ΩA ΩB , we can

therefore also interpret the plot of ΩA ΩB in Fig. 6.8 as a plot of the
probabilities. We observe that the macrostates are not equally probable.
For example, the qA = 0 macrostate is clearly much more unlikely
than qA /NA = qB /NB . How can we use such a result to interpret the
development of an Einstein crystal with time? Let us address this by
introducing a simplified model for the dynamics of an Einstein model,
and by a theoretical argument based on the probabilities of macrostates

6.3.3 Time development of the Einstein crystal — Monte

Carlo model
While the molecular dynamics simulation of a crystal lattice, illustrated
in Fig. 6.3, shows the time development of the system, the Einstein
crystal does not include any description of how the system develops
in time. The Einstein crystal only describes the system in terms of
the microstate, (n1 , n2 , . . . , nN ), of the system. If the system is in one
particular microstate, we do not have any physical laws that tells us how
the system can develop to another microstate. To include dynamics, we
need to add further assumptions to the model. When we defined the
Einstein crystal we assumed that there were no interactions between the
individual oscillators, whereas for the molecular dynamics simulations,
6.3 Behavior of a solid — Theory 139

the atoms had both specific positions in the crystal and they interacted
with their neighbors. How could we extend the Einstein crystal model
to include both simplified interactions and the relative positions of the
atoms, so that we can model energy flow without including the full
dynamics.
Global dynamics in the Einstein crystal. We can make a small mod-
ification to the Einstein crystal model to include some of these effects.
The system consists of two parts, A and B, so that each atom/oscillator
belongs to either A or B. As a first approximation, we introduce simplified
interaction between the oscillators, by allowing energy to be randomly
moved from one oscillator to another oscillator, while conserving the
total energy. This means that we go from one microstate s1 to another
microstate s2 with the same energy. Hence both these microstates are
possible microstates for the system, and they are both equally probable,
since all microstates are equally probable.
We introduce the following algorithm to imitate the random transmis-
sion of energy in the system:
• Select an oscillator (particle) at random, i1 . Let us attempt energy
transport from this oscillator.
• Select another oscillator (particle) at random, i2 . This oscillator may
receive energy from n1 .
• We transfer one unit of energy from i1 to i2 , if oscillator i1 has at
least one unit of energy.
We can implement this algorithm using the following steps: (i) Generate
an initial state of the Einstein crystal with energy qA in part A and
energy qB in part B, (ii) Perform one step of the algorithm, (iii) Measure
the new energy qA (and qB = q − qA ), (iv) Continue with step (ii).
We generate the initial state by placing each of the qA energy units in
part A at random oscillators in part A and similarly for part B. This is
implemented in the following program, which also plots both the state
and the energy qA of the system:
NA = 100, NB = 100 # Nr of oscillators
qA = 300, qB = 0 # Initial energy
q = qA + qB # Total energy
N = NA + NB # Total oscillators
state = zeros(N,float) # Microstate of the system
# state[0:NA-1] is part A, state[NA:NA+NB-1] is part B
ion() # Interactive plotting ON
# Generate initial, random state
placeA = randint(0,NA,(qA,1))
140 6 Micro- and Macro-states

for ip in range(len(placeA)):
i = placeA[ip], state[i] = state[i] + 1
placeB = randint(0,NB,(qB,1))
for ip in range(len(placeB)):
i = placeB[ip], state[i] = state[i] + 1
# Simulate state development
nstep = 100000
EA = zeros(nstep,float), EB = zeros(nstep,float)
for istep in range(nstep):
i1 = randint(0,N) # Select rand osc. i1
if (state[i1]>0): # Does it have any energy ?
i2 = randint(0,N) # Select rand osc. i2
state[i2] = state[i2] + 1 # Transfer
state[i1] = state[i1] - 1 # energy
# Output and display result dynamically
subplot(2,1,1)
plot(r_[0:NA-1],state[0:NA-1],’b’,r_[0:NB-1]+NA,state[NA:NA+NB-1],’r’)
xlabel(’i’); ylabel(’n_i’); draw()
subplot(2,1,2) # Avg energy in each system
EA[istep] = sum(state[0:NA-1])/NA, EB[istep] = q - EA[istep]
plot(r_[0:istep],EA[0:istep],’-r’,r_[0:istep],EB[0:istep],’-b’);
xlabel(’t’); ylabel(’q_A/N_A , q_B/N_B’), draw()

Fig. 6.9 shows the resulting dynamics. In this case all the energy starts on
the left hand side and is gradually transferred to an uniform distribution
— as expected.

10 3

5 2.5
ni

q A /N A , q B /N B

2
0
0 20 40 60 80 100 120 140 160 180 200
i 1.5
10 1

5 0.5
ni

0
0 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
0 20 40 60 80 100 120 140 160 180 200
t
i
10
200
5
ni

150
0
0 20 40 60 80 100 120 140 160 180 200
100
i
QA

20

10 50
ni

0 0
0 20 40 60 80 100 120 140 160 180 200 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000
i t

Fig. 6.9 Illustration of the time development of two Einstein crystals in thermal contact.

Local dynamics in the Einstein crystal. We could add another level

of complexity to the model, by introducing local interactions. This is
demonstrated in Sect. 6.10.
Sampling the space of all microstates. The model we have introduced
here is not a fundamental model for the dynamics of the model. We
6.3 Behavior of a solid — Theory 141

have introduced an algorithmic, stochastic model for the process, but

not a physical model. We cannot therefore interpret the progression of
such a model directly as time. However, the model does move through
a sequence of microstates, all of which are equally probably. Therefore,
if we study a long enough sequence we would expect the sequence of
microstates to sample all possible microstates. We get some insight into
whether we are far from equilibrium just be studying the development
of qA . From Fig. 6.9 we see that we initially are far from equilibrium,
and hence only have seen non-representative microstates, but after many
steps, the system approaches an equilibrium where qA appears to have
forgotten the details of the initial state. We will discuss such models and
how to sample the space of all possible states in Sect. 6.10.
Approaching the stationary state. Both of these models show that
the Einstein crystal will approach a stationary state with a uniform
distribution of energy per oscillator: qA /NA = qB /NB . This corresponds
to the macrostate with the maximum multiplicity. Why is that? How is
this observation of the dynamics explained by the probabilities of the
macrostates?

6.3.4 Time development of the Einstein crystal —

Theoretical argument
How can we interpret the time development of the Einstein crystal
using the probabilities of a macrostate in equilibrium? The probabilities
we have found an expression for are the probabilities of a macrostate
when the system have reached equilibrium. If the system starts with
all the energy in system A, qA = q, qB = 0, we know that this state
is very unlikely in equilibrium. If we just leave the system to itself, it
seems reasonable that it will develop from one microstate to another,
exploring the space of all possible microstates. (This is actually another
fundamental hypothesis called the ergodicity hypothesis). If we then
observe the system after some time, when it has forgotten about its
initial microstate, it will be in some random microstate. But since there
are so many more microstates in the most probable macrostates than in
the initial macrostate, it is much more likely that the system after a while
will be in the most probable macrostates. And it will be very unlikely
to find the system in the initial macrostate. The behavior therefore
corresponds to an irreversible processes. Not because it is impossible to
142 6 Micro- and Macro-states

go back to the initial macrostate, but just because it is very unlikely to

happen.
This also means that if the system starts with a much higher energy
per oscillator in system A than in system B, qA /NA  qB /NB , energy
will flow spontaneously from A to B, but not the other way around.
(Except for small fluctuations). This is an explanation of heat — the flow
of thermal energy from one side to another.
We have here essentially found the second law of thermodynamics:
the law of increased multiplicity. The system develops towards a state of
maximum (or increased) multiplicity. Simply because this is the most
probable behavior. And as the system size increases it becomes increas-
ingly more probable to be in the most probable macrostate and the
probability for all other states goes to zero. But to gain insights into
the probability of a macrostate for a large system, we need an improved
expression for the multiplicity of macrostate qA , an expression we can
analyze mathematically.

6.3.5 Approximate multiplicities using Stirling’s formula

Fig. 6.10 show the multiplicities of the macrostates qA for two systems
with NA = NB = 10, q = 200, and NA = NB = 50, q = 1000. In both
cases, the maximum of the multiplicity occurs at qA = qB = q/2, but
we see that for the system with larger N (and larger q) the multiplicity
is sharper relative to the position of the maximum. But how sharp is it
and how does the function look like around its peak? The multiplicity of
macrostate qA of the two-part Einstein crystal is Ω(qA ) = ΩA (qA ) ΩB (qB )
and the probability of the macrostate is P (qA ) = Ω(qA )/ΩT OT , where
the multiplicity ΩA (qA ) = (qA + NA − 1)!/(qA ! (NA − 1)!). Unfortunately,
it is difficult to discuss this expression because of the factorials. Instead,
we would like an approximative formula which we can address using the
tool-box of calculus. How can we find this approximative formula? We
will do this in two stages, first we will find an approximative expression
for the mulitiplicity using Stirling’s approximation, and then we will find
an approximative expression for the probability of a macrostate.
Theoretical argument: Approximation for the multiplicity. The mul-
tiplicity of an Einstein crystal is
(q + N − 1)!
Ω(N, q) = , (6.12)
q! (N − 1)!
6.3 Behavior of a solid — Theory 143

Fig. 6.10 Plot of N = 20, q = 200 N = 100, q = 1000

1 1
ΩA (qA ; q, N )ΩB (qB =
q − qA ; q, N ) as a 0.8 0.8

function of qA for a 0.6 0.6

+A +B

+A +B
NA = NB = 10, q = 200 b 0.4 0.4
NA = NB = 50, q = 1000.
0.2 0.2
Notice that the multiplicity
becomes sharper for larger 0
0 0.2 0.4 0.6 0.8 1
0
0 0.2 0.4 0.6 0.8 1
systems. qA =q qA =q

How can we approximate this expression? First, we employ the common

trick of working with the logarithm of the multiplicity. Life becomes
simpler when all the multiplications then become additions.

ln Ω(N, q) = ln(q + N − 1)! − ln q! − ln(N − 1)! , (6.13)

To simplify the factorials, we introduce Stirling’s approximation for the

factorial, which is good when the argument of the factorial is large:
1
ln x! ' x ln x − x + ln 2πx , (6.14)
2
which for large values of x can be further approximated to

ln x! ' x ln x − x . (6.15)

This is a good starting point to simplify the multiplicity. We can apply

Stirling’s approximation to each of the factorials in (6.12) if each of
the factorials are large. That is, if q and N are large. In the limit of
(very) large q and N we can also assume that N − 1 ' N . With these
approximations we get:

ln Ω(N, q) = ln(q + N − 1)! − ln q! − ln(N − 1)! (6.16)

' ln(q + N )! − ln q! − ln N ! (6.17)
' (q + N ) ln(q + N ) − (q + N ) − q ln q + q − N ln N + N
(6.18)
= (q + N ) ln(q + N ) − q ln q − N ln N , (6.19)

Now we assume that the number of units of energy is very large compared
to the number of oscillators: q  N , which corresponds to assuming that
we are in the classical limit where the quantum resolution into discrete
energy levels are not important. We can then simplify ln(q + N ) through:
144 6 Micro- and Macro-states

N N
    
ln(q + N ) = ln q 1 + = ln q + ln 1 + ' ln q , (6.20)
q q
where we have used the approximation ln(1 + x) ' 0, which is valid when
x  1. Plugging this back into the relation for ln Ω gets rid of the q ln q
terms, giving

ln Ω ' (q + N ) ln q − N ln N − q ln q (6.21)
q
= q ln q + N ln q − N ln N − q ln q = N ln . (6.22)
N
The multiplicity is then
N
q

Ω'e N ln(q/N )
= (q  N ) , (6.23)
N
This is the multiplicity of the Einstein crystal. We will see further
on that this is a very useful expression since we can use it to calculate
macroscopic properties of the Einstein crystal. Now, we will use this
formula to show that the multiplicity function is indeed very sharp.
Sharpness of the multiplicity function. Armed with this appoximation
for the multiplicity, we are ready to find an approximate expression
for the multiplicity and probability of a macrostate for an Einstein
crystal divided into two parts, A and B, that are in thermal contact. The
multiplicity for the two-part system is
NA  NB
qA qB

Ω = ΩA (NA , qA ) ΩB (NB , qB ) = (6.24)
NA NB
For simplicity, let us address the case where NA = NB . The multiplicity
is then  N  N
qA q qA qB N
 
ΩA ΩB = = . (6.25)
N N N2
We saw in Fig. 6.10 that the multiplicity is sharp around qA = q/2,
we therefore express both qA and qB as small deviations from q/2 by
introducing
q q
qA = + x , q B = − x , (6.26)
2 2
where x is a number much smaller than q (but it is still rather large).
We insert this expression back into the multiplicity, getting
6.3 Behavior of a solid — Theory 145
N N
qA qB q q
  
Ω = ΩA ΩB = =N −2N
+x −x (6.27)
N2 2 2
"  #N
2
q
=N −2N
−x 2
. (6.28)
2

Again, it is easier to work with the logarithm of this expression

"  #N
2
q
ln Ω = ln N −2N
+ ln −x 2
. (6.29)
2

We simplify the second part of this expression by pulling (q/2)2 outside

the paranthesis
#N
2x
"  "  2 !#
2 2
q q

ln −x 2
= N ln 1− (6.30)
2 2 q
2x
 2 2 !
q

= N ln + N ln 1 − . (6.31)
2 q

Where 2x/q  1 and we can therefore use the approximation ln(1+u) ' u
(when u  1):
#N
2x
" 
2  2 2
q q

ln −x 2
' N ln −N . (6.32)
2 2 q

The total multiplicity is therefore

 2
x√
N ln(q/2)2 −N (2x/q)2 −N (2x/q)2 −
Ω=N −2N
e e = Ωmax e , = Ωmax e q/(2 N )

(6.33)
This is the multiplicity for the macrostate qA = q/2 + x, and the probabil-
ity for this macrostate is simply the multiplicity multiplied by a constant.
The probability P (qA ) is therefore a similar function
 2
x√
−
P (qA ) = ΩA ΩB /ΩT OT = Pmax e q/(2 N )
, (6.34)

We have found that the multiplicity and the probability is a Gaussian

with a peak at x = 0, that is for qA = q/2. This function falls off rapidly
for both positive and negative x. How rapidly? The multiplicity has
√ fallen
to a value of 1/e when N (2x/q) = 1, that is, when x = q/(2 N ). It
2

might appear that this value of x is a large number, since q is a large

146 6 Micro- and Macro-states

number. However, for realistic systems N = 1022 , which means that the
multiplicity has fallen to 1/e after a deviation x = q/(2 1011 ), which is
very small compared with the value of q. If the full graph, that is with
the full range of possible qA values from 0 to q, spanned from here to the
Moon, the multiplicity of a system with would have fallen to 1/e over a
distance of 3.86 108 m/(2 1011 ) = 19mm. This also means that it is very
unlikely to observe a value of qA which is far away from q/2. The relative
variations of qA are extremely small in realistic systems — in practice
these fluctuations are not measureable at all.
We call the limit when there are no measureable fluctuations away
from the most likely macrostate the thermodynamic limit.
Comparison of exact and approximate results. We can test this ap-
proximation by comparing it with directly calculated values of the multi-
plicity. Fig. 6.11a shows plots of the multiplicity Ω(qA ; N, q)/Ωmax for
N = 10, 20, 40, 80 and q = 10 N . Fig, 6.11a shows direct plots of the
multiplicity rescaled by its maximum value. Otherwise the systems with
larger N and q would completely dominate the plot. The multiplicities
have a clear maximum value and decay approximately symmetrically
from each side of the maximum. The qA value that gives maximum
multiplicity increases as we increase N and q. We expect the maximum
to be at qA = q/2. Indeed, in Fig. 6.11b we see that if we plot Ωmax as a
function of qA − q/2, the distributions are centered around zero. However,
the widths of the distributions are also changing when we change q and
N . How wide is the distrubution compared to the average or typical
value of qA ? This is shown in Fig. 6.11, which shows Ωmax as a function
of (qA − q/2)/(q/2). Here, it is clear that the multiplicity becomes more
and more narrowly distributed as N and q increases — the multiplicity
is becoming sharper, and deviations from the average value for qA are
becoming less and less likely.
How good is the approximation we found in (6.33)? First, we replot
the multiplicities according to the
√ theory in (6.33): We plot Ω/Ωmax
as a function of (qA − (q/2))/(2 N ) in Fig. 6.11d. Yes! All the curves
now fall onto the same curve, corresponding to the Gaussian form in
(6.33). We call such a plot a data-collapse, and we often use such plots to
demonstrate/validate our theories. Now, we can even compare with the
 √ 
Gaussian curve, we plot exp (qA − q/2)/(2 N ) in the same plot in
Fig. 6.11d, and indeed the theoretical curve fits nicely with the observed
curve — the approximation we have developed seems sound.
6.4 The ideal gas — Theory 147

1 1
N=10
N=20
+ 0.5 0.5

+
N=40
N=80

0 0
0 500 1000 1500 2000 -1000 -500 0 500 1000
qA qA ! q=2
1 1

0.5 0.5
+

+
0 0
-1 -0.5 0 0.5 1 -100 -50 0 p 50 100
(qA ! q=2)=(q=2) (qA ! q=2)= N

√Ω(qA ; q, N )/Ωmax as a function of a qA , b qA − q/2, c (qA − q/2)/(q/2),

Fig. 6.11 Plot of
d (qA − q/2)/(2 N ).

6.4 The ideal gas — Theory

The main observations and conclusions we have made so far are valid
for most systems in thermal contact or for fluctuations in parts within
the system. The multiplicity function will typically be very sharp for
large systems, meaning that the system will only have a reasonable
probability to be in a very small fraction of the macrostates. Here, we
will demonstrate in detail that this is also the case for an ideal gas — a
simplified model of a gas.
Ideal gas. An ideal gas consists of N atoms (or molecules) in a box
of size L × L× as illustrated in Fig. 6.12. We assume that if the gas is
thin, that is if the particles typically are far from each other, they do
not interact much, except when they collide. As a first approximation,
we will assume that the particles do not interact at all. This is our
gas model, called an ideal gas, consisting of a set of particles moving
inside a box without interactions. However, we do assume an implicit
interaction — the atoms may exchange energy, but in such a way that
the total energy of the system is conserved. In addition to this, we will
encounter a two quantum mechanical features. First, if the particles are
Fermions, then two particles cannot be in the same energy state. We will
see what consequences this has further on. Second, the particle cannot
be discerned, and therefore we cannot discern two states where we only
148 6 Micro- and Macro-states

have exchanged two atoms. This will have consequences for how many
states there are for the system.

The ideal gas model consists of the following assumptions

• The gas consists of N particles
• The particles are confined to an L × L × L box of volume V
• The particles do not interact directly
• The total energy of the gas is E, but the distribution of energy
within the gas is not prescribed
• The particles are identical and cannot be discern from one another

To describe the ideal gas, we will start from a single particle, then
discuss two particles, before finally moving to a system of N particles.
Our goal is to find the multiplicity of the ideal gas and to show that the
multiplicity function indeed is sharp also for the ideal gas.

Molecular dynamics gas Ideal gas model

L L

L L

L L

Fig. 6.12 (Left) Illustration of a molecular dynamics simulation of a thin gas. Right
Illustration of the ideal gas model consisting of N non-interacting, identical particles in a
L × L × L box.

6.4.1 Microstates of a single particle

We start by addressing the multiplicity of a single particle in a three-
dimensional box. What are the possible states of a single particle? From
6.4 The ideal gas — Theory 149

quantum mechanics we know that the possible states of a particle in a

L × L × L box are quantized with energy levels

h2 h2  2 
( nx , nz , nz ) = n · n = n + n2
+ n2
, (6.35)
8mL2 8mL2 x y z

where m is the mass of the particle, L is the length of the box and
n = (nx , nz , nz ) is related to the momentum
h
p= n. (6.36)
2L
Each of the numbers nx , ny , nz represents state for the motion in the
x, y, and z-direction respectively, and they are all positive integers. We
can illustrate the possible states in a three-dimensional space with nx ,
ny , nz along the axes as illustrated in Fig. 6.13. Each point such as
(1, 1, 1) or (2, 3, 2) represent a state for system. In the n space there is a
constant density of states corresponding to one state per unit volume.
(Each 1 × 1 × 1 volume has one state). We also see that because the
energy depends on n2 , E = (h2 /8mL2 ) n2 , all states that have the same
distance to the origin in n-space have the same energy.

(a) nz (b)
ny

ny

n
nx nx

Fig. 6.13 Illustration of the states of a single particle in a three-dimensional box.

How can we use this to count the number of states in an ideal gas with
a given total energy E, but with only one particle, N = 1? In this case,
all the states with the same energy is on a the surface of sphere with
radius n in n-space. What is this radius? It is found from the relation
between n and E for the system consisting of a single particle in three
dimensions:
h2 2 2L √
E= n ⇒ n= 2mE (6.37)
8mL 2 h
150 6 Micro- and Macro-states

How can we count the corresponding number of states with this energy?
We need to find the surface area of the corresponding sphere in n-space,
remembering to include only 1/8 of this surface since we only address
positive n-values. This is simple for the 3d system. The surface area is
4πn2 , so that the number of states is:

1 1 8mL2 4πL2 4π
Ω1 = 4πn2 = 4π 2 E = E = 2 v 2/3 E . (6.38)
8 8 h h2 h
We notice that this result is a function of of E and V as it should, and
a function of N , but N = 1 here. How can we generalize this result to
more particles?

6.4.2 Microstates of two non-interacting particles

What happens if we have more than one particle, for example, two
particles? Let us address this in the simplest case — a one-dimensional
system. In this case each particle is described by a single quantum state
ni , n1 for particle 1 and n2 for particle 2, so that the state of the system
can be written as the pair (n1 , n2 ). When the total energy of the system
is E, the possible states of the system is therefore all the states so that
E = h2 /(8mL2 ) (n21 + n22 ) = E0 (n21 + n22 ), and we call this the multiplicity
Ω2 (E) of the two-particle system. This means that for a total energy of
E = 4E0 there are two possible states, (2, 0), and (0, 2), and therefore
Ω2 (4E0 ) = 2. Or is it? We have overlooked an important quantum
mechanical effect!
Indistinguishable particles. This result is true of the particles are dis-
tinguishable from each other. They are distinguishable if we can place
a label on each of them so that we can see that particle 1 is in state 2
whereas particle 2 is in state 0. However, for a gas of identical particles
(atoms, molecules), the particles are indistinguishable. We cannot discern
the state (2, 0) from the state (0, 2) because the molecules are indentical.
This means that we have counted too many states! We should not count
all the possible ways that we can interchange the two particles. In general
we know that for N particles, there are N ! possible ways to interchange
them. For two particles, there are 2! = 2 ways to interchange them. The
correct multiplicity is therefore Ω2 (4E0 ) = 2/2! = 1.
Fermions or Bosons. In addition, we should also reflect on whether
the particles are Fermions or Bosons. Why? Because if the particles are
6.4 The ideal gas — Theory 151

Fermions, there are additional restrictions. Two particles cannot be in the

same state. This means that the state (4, 4) is not a possible state for the
two-particle system. We will return to this case later. For now, we will
simply assume that we are in the limit when the energy is large, so that
there are many more possible states available than there are particles,
so that the number of restricted states for Fermions is negligible. These
considerations will be more explicit in Chap. 7.

6.4.3 Microstates of N non-interacting particles

Distinguishable particles. For N independent, distinguishable particles
all in a box of size V = L × L × L the energy of the system is given as:
3N
h2 X h2 X
E= N (n2
+ n2
+ n2
) = n2 . (6.39)
8mL2 i=1 xi yi zi
8mL2 i=1 i

Here, the sum is over 3N values of ni . This means that we write N

particles with 3 degrees of freedom as 3N particles with 1 degree of
freedom. This expression reproduces the result for N = 1. We see that
the energy E depends on the radius n in the 3N -dimensional n − space:
√
E = (h2 /(8mL2 )) n2 ⇒ n = (2L/h) 2mE (6.40)

We can use the same approach as for a single particle two find the number
of states with this energy. The density of states in the 3N -dimensional n-
space is still 1: There is one state for each 1×1×. . .×1 (3N times) volume.
4
We can therefore estimate the number of states by estimating the “area”
of a sphere with radius n in 3N dimesions. (We must also remember to
only include positive values for all the ni in n = (n1 , n2 , n3 , . . . , n3N ) ).
The general formula for the surface area of a d-dimensional sphere is:

2π d/2
A=   rd−1 . (6.41)
d
2 −1 !

We use this expression with d = 3N . In addition, we also need to ensure

that only the ni -values in the first “octant” (3N -tant) are used, that is
we must divide by the number of “quadrants” = 2d = 23N :
4
This is true if we do not include spin. For particles with spin 1/2 there will be two
states per such volume, one state with spin up and one state with spin down. We will
return to this effect in Chap. 10.
152 6 Micro- and Macro-states

1 2π 3N/2 2L
 3N −1
Ωdisting = (2mE)1/2 , (6.42)
23N
 
3N
−1 ! h
2

Indistinguishable particles. This the number of states of a system

of N distinguishable particles. However, an ideal gas consists of N
indistinguishable particles? How should the multiplicity be modified?
Based on our argument for two-particles above, we realize that we have
counted too many states. We must therefore divide by the number of
ways we can rearrange the N particles between the states, which is N !.
The multiplicity for a gas of indistinguishable particles is therefore

1 1 1 2π 3N/2 2L
 3N −1
Ω = Ωindisting = Ωdisting = (2mE)1/2 .
N! N ! 23N 3N − 1 !
 
h
2
(6.43)
We can simplify this expression by using that for very large N , 3N − 1 '
3N . However, in the first few examples, we will only be interested in the
V and E depedence of this expression, which is simple:

Ω = f (N )V N E 3N/2 . (6.44)

6.4.4 Interaction between two ideal gases

We have now found the multiplicity for an ideal gas of N atoms, volume
V , and energy E. We can use this expression to address the macrostate
of two ideal gases in equilibrium, or equivalently, to find the distribution
of energy inside an ideal gas.
Just like we did for the ideal crystal, we can divide the system into two
parts A and B as illustrated in Fig. 6.14. The whole system is isolated,
but the two parts are not thermally insulated. This means that the sum
of the energies in A and B is constant, EA + EB = E. We assume that
the two parts cannot change shape and that there is no transport of
particles from one part to the other. Thus, the number of particles and
the volumes of each of the two parts are constant and equal to NA , NB
and VA and VB . For simplicity we assume that the system is divided into
two equal parts: NA = NB = N/2 and VA = VB = V /2.
The question we want to answer is: What is the multiplicity of the
macrostate described by EA , Ω(EA )? In general, the multiplicity of
the combined system is the product of the multiplicities of each of the
systems:
6.4 The ideal gas — Theory 153

Fig. 6.14 An isolated ideal

gas is divided into two A ΔE B
equal-sized parts, A and B.
The two parts are separated
by a partition that allows
energy to pass through, but
it is immobile (so that the
volumes do not change),
and impermeable (so that
the number of particles on
each side is constant).
EA + EB = E
VA= VB= V/2
NA= NB= N/2

Ω(EA ; E, N, V ) = ΩA ΩB = f (NA )f (NB ) (VA VB )N (EA EB )N , (6.45)

The only variation occurs in EA and EB = E − EA . Just like we did for

the Einstein crystal, we can develop a theory for this variation for small
fluctuations in the energy. Let us look at a variation

EA = ĒA + ∆E , EB = ĒB − ∆E , (6.46)

where ĒA = ĒB = E/2 when NA = NB and VA = VB . The two systems

are then equal, and we would expect each system to have the same
equilibrium energy given by ĒA and ĒB . We insert EA = E/2 + ∆E and
EB = E/2 − ∆E into the multiplicity, getting
2 !N
E

Ω(∆E) = C − ∆E 2
, (6.47)
2

To simplify the algebra, we work on the logarithm of Ω(∆E), getting

2 !
E

ln Ω(∆E) = ln C + N ln − ∆E 2
2
2∆E
2 2 !
E
 
= ln C + N ln 1− (6.48)
2 E
2∆E
 2
= ln C + 2N ln(E/2) − N ,
E
which gives that
154 6 Micro- and Macro-states
 2
√
2 ∆E
− N ∆E − √
Ω(∆E) = Ωm e (E/2) = Ωm e (Ē/ N )
, (6.49)

This is Gaussian form for the multiplicity function, just like we found
for the Einstein crystal. This means that also for the ideal gas, the
multiplicity function is very sharp. Just as we found for the Einstein √
crystal, the width of the multiplicity function is proportional to N
while the average value is proportional to E which √ is proportional to N ,
hence the relative value of the width goes like 1/ N . For realistic values
of N , such as N = 1020 , this relative width becomes a very small number,
typically 10−10 , which means that we cannot discern the actual value of
the energy EA in the ideal gas from its average value E/2 unless we can
measure EA to a precision of 10−10 , which is practically impossible.
This means that it is very unlikely to find the system in a state that is
significantly different from E/2, where the multiplicity is at its maximum.
If the system started away from this value, for example by having much
more energy in part A than in part B, the system would evolve towards
the most probable macrostate, where EA = E/2. This is an example of
a general principle, the second law of thermodynamics.

6.5 Thermal equilibrium, Entropy and Temperature

We have found that for the two models system we have addressed, the
ideal gas and the ideal (Einstein) crystal, the most likely macrostate is
very sharp and any fluctuation away from the most likely macrostate is
extremely unlikely. If we start from a microstate that corresponds to a
macrostate that is away from the most likely macrostate the system will
develop toward the most likely macrostate, and therefore toward a state
with higher multiplicity — simply from the laws of probability.
This is what we call the second law of thermodynamics: The
multiplicity of a system increases. What characterizes the state it evolves
towards? The system evolves towards the macrostate with the largest
multiplicity, and this state corresponds to the stationary state or the
equilibrium state of the system.
Characteristics of the equilibrium state. What characterizes this equi-
librium state — the state with maximum multiplicity? Let us find the
maximum of the multiplicity for two Einstein crystals in contact. The
multiplicity of a macrostate with energy qA is
6.5 Thermal equilibrium, Entropy and Temperature 155

Ω(NA , qA )Ω(NB , qB ) = Ω(NA , qA )Ω(NB , q − qA ) (6.50)

where q = qA + qB is conserved. We can find the extremum of this

function by differentiation with qA :
d
Ω(NA , qA )Ω(NB , q − qA ) = 0 , (6.51)
d qA
dΩ(NA , qA ) dΩ(NB , q − qA )
Ω(NB , q − qA ) + Ω(NA , qA ) = 0 , (6.52)
d qA d qA
∂Ω(NA , qA ) ∂Ω(NB , qB )
Ω(NB , q − qA ) − Ω(NA , qA ) =0, (6.53)
∂ qA ∂ qB
∂Ω(NA , qA ) ∂Ω(NB , qB )
Ω(NB , qB ) = Ω(NA , qA ) , (6.54)
∂ qA ∂ qB
1 ∂Ω(NA , qA ) 1 ∂Ω(NB , qB )
= , (6.55)
Ω(NA , qA ) ∂ qA Ω(NB , qB ) ∂ qB
∂ ln Ω(NA , qA ) ∂ ln Ω(NB , qB )
= . (6.56)
∂ qA ∂ qB
∂ ln Ω(NA , qA ) ∂ ln Ω(NB , qB )
= . (6.57)
∂ qA ∂ qB
Definition of Entropy. This equation, (6.57), suggests an interesting
relation between the logarithms of the multiplicities for the qA -value that
corresponds to the maximum multiplicity, that is, for the macrostate
that is most probable. We introduce the term entropy to describe the
logarithm of the multiplicity of the system:

Entropy: The entropy of an isolated system with given N , V , and

E is given as
S = k ln Ω(N, V, E) , (6.58)
where k is called the Boltmann constant, k = 1.3806488(13) 10−23 J/K,
and Ω(N, V, E) is the multiplicity of the system.

Relation between a microscopic and a macroscopic description. The

definition of entropy relates the microscopic description of the world,
given in the form of the multiplicity function, with the macroscopic
description in the form of entropy. This definition of the entropy will
156 6 Micro- and Macro-states

allow us to calculate the macroscopic, thermodynamic properties of a

system from a microscopic theory of the system.
Entropy and the second law of thermodynamics. We have seen that
isolated systems will develop so that the multiplicity increases and they
will develop toward a state with maximal multiplicity. We can now
reformulate this law in terms of entropy: Isolated systems will develop
so that the entropy increases, and in equilibrium the system will be in a
macrostate with maximum entropy.
Properties of the Entropy. We see from the definition of the entropy
that it is additive:Since the entropy is defined as the logarithm of the
multiplicities, the entropy of two independent systems are the sums of
their individual entropies, because the total multiplicity is the product
of the individual multiplicities:

S = SA + SB when Ω = ΩA · ΩB . (6.59)

6.5.1 Equilibrium of the Einstein crystal

Let us see how the concept of entropy is related to the equilibrium of a
system through an example we already know in detail — the Einstein
crystal. For this system we know the multiplicity for a macrostate qA ,
where qA of the energy is in part A of the system and qB = q − qA is in
part B. The multiplicity for the macrostate qA is
(qA + NA − 1)! (qB + NB − 1)!
Ω(qA ) = Ω(NA , qA ) Ω(NB , qB ) = .
qA !(NA − 1)! (qB !(NB − 1)!
(6.60)
And the entropy of the system when it is in macrostate qA is therefore
(qA + NA − 1)! (qB + NB − 1)!
S(qA ) = ln Ω(qA ) = ln . (6.61)
qA !(NA − 1)! (qB !(NB − 1)!
We have already plotted the multiplicity of this system in Fig. 6.8, and
we found that it had a sharp maximum. Let us redo this, but instead
plot the Entropy S(qA ) of the system as a function of the macrostate qA .
This is done using the a similar program, but we now take the logarithm
of the multiplicity. The following prorgam find the multiplicity and the
6.5 Thermal equilibrium, Entropy and Temperature 157

entropy as a function of qA for two Einstein crystals in thermal contact

with NA = 300 and NB = 200 and q = 100 5
from pylab import *
from scipy.misc import comb
NA = 300
NB = 200
q = 200
multA = zeros(q+1,float)
multB = zeros(q+1,float)
mult = zeros(q+1,float)
N = NA + NB
qvalue = array(range(q+1))
for ik in range(len(qvalue)):
qA = qvalue[ik]
qB = q - qA
multA[ik] = comb(qA+NA-1,qA)
multB[ik] = comb(qB+NB-1,qB)
mult[ik] = multA[ik]*multB[ik]
SA = log(multA), SB = log(multB), STOT = SA + SB
plot(qvalue,SA,’-r’,qvalue,SB,’-b’,qvalue,STOT,’:k’)
xlabel(’q_A’), ylabel(’S’)

The resulting entropies of system A, SA (qA ), system B, SB (q − qA ), and

of the total system, ST OT (qA ) = SA (qA ) + SB (q − qB ), is plotted in
Fig. 6.15.

Fig. 6.15 Plot of the 600

entropies SA and SB
of two coupled Einstein 400
crystals, and plots of the
S

derivatives of the entropies 200

with respect to qA .
0
0 50 100 150 200
qA
6
dS/dq A

0
0 50 100 150 200
qA

Entropy and equilibrium. We can use the plot of ST OT (qA ) and SA (qA )
in Fig. 6.15 to better understand the condition for equilibrium. Equi-
librium occurs at the qA -value where the entropy (and therefore the
5
We have here chosen NA and NB not to be equal so that the position of the maximum
is not at the center of the figure. This makes the arguments clearer.
158 6 Micro- and Macro-states

multiplicity) is maximum. This point is easily recognized in Fig. 6.15a.

However, we can also find the equilibrium from the equilibrium condition
in (6.57):
∂ ln Ω(NA , qA ) ∂ ln Ω(NB , qB )
= , (6.62)
∂ qA ∂ qB
which we can we translate into a condition for the entropies
∂S(NA , qA ) ∂S(NB , qB )
= , (6.63)
∂ qA ∂ qB
that is, the derivatives of the entropies of the two subsystems are the
same when in equilibrium.
The derivatives of the entropies are shown in Fig. 6.15b. How can we
interpret the condition in (6.63)? We can use the plot of ∂SA /∂qA to
see what happens if system A is in a macrostate with a qA -value that
is smaller than the equilibrium value, such as the point marked with a
circle in Fig. 6.15b. In this case, we see from the plot of ∂SA /∂qA that
if we increase qA by one, SA increases and SB decreases. However, we
also see from the two curves that SA increases more than SB decreases,
because the slope of SA (qA ) is steeper than that of SB (qA ), that is

∂S(NA , qA ) ∂S(NB , qB )
> , (6.64)
∂ qA ∂ qB
in this point. A small increase in qA by ∆q: qA → qA + ∆q, therefore
results in an increase in the total entropy of the system
∂SA ∂SB ∂SA ∂SB
 
∆ST OT = ∆SA + ∆SB = ∆qA + ∆qB = − ∆q
∂qA ∂qB ∂qA ∂qB
(6.65)
since ∆qA = ∆q and ∆qB = −∆qA = −∆q since qB = q − qA and q is a
constant when the small amount ∆q of energy is transferred from part B
to part A. Since the entropy must increase as the system evolves towards
equilibrium, we see that a small transfer of energy from part B to part
A will occur when qA is smaller than the equilibrium value.
Test your understanding: See if you can formulate a similar argument when qA is
larger than the equilibrium value.

It is first when the two slopes are equal that the system is in equilibrium.
What does the slope of SA tell us? It tells us how much the entropy (or
multiplicity) changes when we add a small amount of energy. It tells us
which way the system will develop, because it will develop toward larger
6.5 Thermal equilibrium, Entropy and Temperature 159

entropy (multiplicity). For qA < q̄A the system will gain more entropy
(multiplicity) by increasing qA and decreasing qB . Since the system, most
likely, develops towards larger entropy (multiplicity) it will develop this
way. Not always, but most probably.

6.5.2 Temperature in the Einstein crystal

Now, we can interpret the slope, ∂SA /∂qA . This is what is equal when
two systems are in thermal equilibrium. Since we also know that the
temperature is equal in two systems in thermal equilibrium, it means
that this relation is related to the temperature. How is it related to
the temperature? The simplest assumption would be that it is a simple
function of temperature. (If we look at the dimensions, we would get a
strong hint, but this, does, of course depend on how we have chosen the
dimensions of kB , which means that this argument becomes circular.)
When the slope, ∂S/∂q, of the entropy as a function of energy is
larger in system A than in system B, ∂SA /∂qA > ∂SB /∂qB , energy
will spontaneously flow from system B to system A, since this will
lower the total energy. Since we also know that thermal energy tends
to flow from a system with high temperature to a system with low
temperature, we expect the temperature to be low when the slope is high.
One possible functional relationship with this behavior is to assume that
the temperature is inversely proportional to the slope. This will indeed
be how we define the temperature:

Temperature in an isolated system — a system with given

(N, V, E):
1 ∂S
 
= , (6.66)
T ∂E N,V
where we have included the N and V to show that we keep them
constant as we are changing E.

This is our definition of temperature. We will in the following see that

this is indeed consistent with our other possible definitions of temperature,
such as from the average kinetic energy per particle in an ideal gas.
Connection between the microscopic and the macroscopic. We now
have all the tools we need to connect the microscopic theories we have
160 6 Micro- and Macro-states

developed and the measurements we have done with macroscopic prop-

erties such as temperature, energy, and heat. This allows us to connect
between the microscopic and the macroscopic worlds. This is the coupling
between statistical mechanics and thermodynamics. Thermodynamics is
the rules and laws we use to describe thermal and mechanical properties
of macroscopic systems, whereas statistical mechanics is the theory that
allows us to calculate and prove thermodynamic principles, laws and
properties from a microscopical foundation.
We will in the following use this coupling to calculate the entropy
of a system, S(N, V, E) from a microscopic model: It may be from a
theoretical model where we can count all the states or from a numerical
simulation such as a molecular dynamics simulation. As soon as we have
the function S(N, V, E) we can start applying thermodynamics to this
function, to understand relations between energy, temperature, volume,
pressure, and the number of particles in our system. For this development,
for the application of thermodynamics, it is not important how we have
found S(N, V, E), we just need this function as a starting point for our
calculations. Let us therefore see how we can use the theories we have
developed to find entropies, temperatures and energies for the ideal gas
and the ideal crystal systems.
Microscopic.
• Multiplicity: Ω(N, V, E)
• An isolated system develops towards larger multiplicify
• In equilibrium, an isolated system is in the macrostate with maximum
multiplicity

Macroscopic.
• Entropy: S = kB ln Ω(N, V, E)
• An isolated system develops towards larger entropy
• In equilibrium, an isolated sytem is in the macrostate with maximum
entropy
• Temperature: (1/T ) = (∂S/∂E)N,V

Clickers: Thermal 01
6.5 Thermal equilibrium, Entropy and Temperature 161

6.5.3 Example: Entropy and energy of the Einstein crystal

We can now start connecting the microscopic and the macroscopic worlds.
We have developed methods to find the multiplicity of an ideal gas and
an Einstein crystal for a system with given N , V , and E. From this we
can calculate the entropy, and from the entropy we can calculate the
temperature.
The multiplicity of the Einstein crystal is
N
eq

Ω= , (6.67)
N
in the limit when q and N is large, and q  N . The internal energy, E,
is related to q by E = q, where  is the energy unit for the energy states
of the harmonic oscillator. The entropy is

S = k ln Ω = N k (ln q + ln(E/) − ln N ) , (6.68)

S = N k ln E − N k ln( N ) + N k . (6.69)
And the temperature is:
1 ∂S Nk
 
= = , (6.70)
T ∂E E
which gives
E = N kT . (6.71)
This is what we would expect from the equipartition theorem, since
N is the number of oscillators, and each oscillator has two degerees of
freedom, hence, in equilibrium the energy is kT per oscillator and N kT
in total.
From this we can also predict the heat capacity. The heat capacity
is the amount of thermal energy, heat, which we need to transfer to a
system in order to increase its temperature by one unit (Kelvin), which
is given as the derivative of the energy with respect to temperature:
∂E ∂
 
CV = = (N kT ) = N k . (6.72)
∂T N,V ∂T

We call this the heat capacity at constant volume, since we keep both the
volume and the number of particles constant when we take the partial
derivative.
162 6 Micro- and Macro-states

What we have found here now is a theoretical prediction of the heat

capacity of a solid. The prediction is that it only depends on the number of
oscillators, N , in the solid. This result can be compared with experimental
or simulational results to check its validity.

6.5.4 Example: Entropy and energy of the ideal gas

We can perform a similar calculation for the ideal gas. In this case the
multiplicity was a bit more complicated:

1 1 2π 3N/2 2L
 3N −1
Ω= (2mE)1/2 . (6.73)
N ! 23N 3N − 1 !
 
h
2

Here, we will assume that N is very large, so that N − 1 ' N , which

simplifies the expression to:

1 1 2π 3N/2 2L
 3N
Ω' (2mE)1/2 . (6.74)
N ! 23N 3N !
 
h
2

This is further simplified to

!3N/2
1 2πmEL2
Ω=2 . (6.75)
N! 3
! h2


2N

And the entropy is

S
= ln Ω
k
!3/2
3N 3N 3N 2πmEL2
 
= − (N ln N − N ) − ln − + N ln
2 2 2 h2
3 4πmE
3/2 !
V
    
= N − (−1) − − + ln + ln
2 N 3N h2
4πmE 5
3/2 !
V

= N ln + .
N 3N h2 2
(6.76)

This equation is called Sackur-Tetrode’s equation.

6.5 Thermal equilibrium, Entropy and Temperature 163

Sackur-Tetrode’s equation for the entropy of an ideal gas:

4πmE 5
3/2 !
V

S = N kB ln + . (6.77)
N 3N h2 2

We can use this to find the energy E of the ideal gas, by finding the
temperature:
1 ∂S
 
= . (6.78)
T ∂E N,V
Since N and V are constants in this derivative, we only need to include
the terms that include the energy:
3
S(N, V, E) = g(N, V ) + N k ln E 3/2 = g(N, V ) + N k ln E , (6.79)
2
where g(N, V ) contains all the factors that do not depend on E. It is
now easy to find the temperature:
1 ∂S 3 1
 
= = Nk , (6.80)
T ∂E N,V 2 E

which gives
3
E = N kT . (6.81)
2
Again, this corresponds to the equipartition principle. The ideal gas
has N atoms, and each atom has three degrees of freedom: classically
this corresponds to three independent directions for the velocities, and
quantum-mechanically this corresponds to the three independent quan-
tum number nx , ny , nz we used to characterize the states of the ideal gas.
Each degree of freedom contributes with kT /2, which gives 3N kT /2 for
the whole gas of N gas atoms.

6.5.5 Example: Entropy of expansion

We can use Sackur-Tetrode’s equation to gain more insight into the
behavior of the ideal gas. First, let us look at what happens if we allow
an ideal gas to expand without changing its energy or the number of
particles as illustrated in Fig. 6.16. We only change the volume. You can
think of an isolated box of size V2 , where the gas initially is confined
to a smaller part, V1 , of the box. Then an internal barrier is removed,
164 6 Micro- and Macro-states

and the gas is allowed to fill the entire volume, while the total energy
and the total number of particles is conserved since the whole system
is isolated. The change (increase, as always) in entropy in this case by
going from a volume V1 to a volume V2 is:
V2
∆S = N k (ln V2 − ln V1 ) = N k ln . (6.82)
V1
In this case, the expansion of the system was isoenergetic — done at
constant energy of the whole system. However, since for an ideal gas,
the energy and the temperature are related by E = (3/2)N kT , we see
that when the number of particles is constant, having a constant energy
corresponds to having a constant temperature. In this particular case,
the change in entropy we have found therefore also corresponds to an
isothermal expansion — an expasion done at constant temperature. (Iso
here means at the same or constant).
In this case, the process is clearly also irreversible because the entropy
of the isolated system increases during the processes. Indeed, we would
be very surprised if we placed the dividing wall back into the system, and
all the gas particles spontaneously concentrated in the initial volume, V1 ,
by itself. Such a process in an isolated system would be violating the
second law of thermodynamics, because entropy would decrease, and also
appears to violate our basic intuitions about valid or probable processes
in the world.

(a) (b)
V1 V2

Fig. 6.16 Illustration of an expansion of an ideal gas while the gas is in an isolated box.
The system is isolated, and the gas expands from V1 to V2 after the internal wall was
removed from the system.

6.5.6 Example: Entropy of mixing

What if we take two gases, A and B, each initially occupying a volume
V , and then allow the two gases to mix? Let us assume that the two
6.5 Thermal equilibrium, Entropy and Temperature 165

gases are inside a common, isolated system, so that their total number
of particles, energy and volumes are conserved. Then an internal barrier
between the two gases is removed, so that gas A can move into the volume
originally occupied by gas B and vice versa as illustrated in Fig. 6.17. If
the two gases are not interacting, and the two gases are not the same,
we can consider each process independently. First gas A expands into
twice its volume, leading to an entropy change
VA,2
∆SA = NA k ln , (6.83)
VA,1
and similarly for gas B:
VB,2
∆SB = NB k ln , (6.84)
VB,1
If NA = NB = N and V2 = 2V1 for both A and B, we get

∆SA = N k ln 2 , ∆SB = N k ln 2 , (6.85)

and the total entropy change is

∆ST OT = ∆Smix = ∆SA + ∆SB = 2N k ln 2 , (6.86)

where we have used the additive property of entropies. This entropy is

called the entropy of mixing of the system.
Notice that we here assumed that the two gases were different. Other-
wise, nothing would happen — there would not be any change in entropy
if we open a separation between two equal gases each of volume V .
We also clearly see that this process is irreversible for this isolated
system since the entropy is increasing. We would be very surprised if,
after the system was completely mixed, gas A spontaneously segregated
into the left side and gas B segregated into the right side. Indeed, such
as processes in an isolated system would violated the second law of
thermodynamics, since the total entropy of the isolated system would
increase. Your intuition is therefore perfectly aligned with the second
law of thermodynamics in this case.

6.5.7 Example: Gibb’s paradox

Sackur-Tetrode’s equation is carefully constructed to ensure that the
entropy does not change if we mix two identical gases. What is the
166 6 Micro- and Macro-states

(a) (b)
V V 2V

Fig. 6.17 Illustration of the mixing of two gases A and B, each with initial volume V .

change in entropy if we have two identical gases each with N particles

and volume V and at the same energy, E, and then we remove the barrier
between them?
If the gases were different, we know that the change in entropy would
correspond to the entropy of mixing. If the gases are identical, there
should not be any change in entropy. What does the Sackur-Tetrode
equation give us?
Before the partition is removed, the entropy of each part is S(N, V, E)
so that the total entropy of the system is 2 S(N, V, E). After the partition
is removed, the system consists of a gas with 2N particles, 2V volume and
2E energy with entropy S(2N, 2V, 2E). According to Sackur-Tetrode’s
equation the entropy is
3/2
2V 4πm(2E) 5
 !
S(2N, 2V, 2E) = (2N )k ln + . (6.87)
2N 3(2N )h2 2

We see that the two ratios 2V /2N = V /N and 2E/2N = E/N , and that
the total entropy therefore is

4πmE 5
3/2 !
V

S(2N, 2V, 2E) = 2N k ln + = 2S(N, V, E) .
N 3N h2 2
(6.88)
Sackur-Tetrode’s equation therefore correctly predicts that there is no
change in entropy when we remove such an internal barrier between two
identical gases. However, in order for this to be the case, it was essential
to include the N !-term in the multiplicity, because it is this term that
ends up as the N in the V /N term. Without the N !-term, we would
instead have
4πmE 3
 3/2 ! !
S(N, V, E) = N k ln V + . (6.89)
3N h2 2
6.6 The laws of thermodynamics 167

and the entropy of a double system, S(2N, 2V, 2E), would be:

4πm2E 3
 3/2 ! !
S(2N, 2V, 2E) = 2N k ln 2V + (6.90)
3N 2h2 2

The difference between 2S(N, V, E) and S(2N, 2V, 2E) would then be
2N k ln 2, which is the entropy of mixing. The Sackur-Tetrode equation
is therefore carefully made to ensure that this does not occur. This issue
was first raised by Gibbs, and is called Gibbs paradox.
We notice that the entropy scales in a simple way when we change
all the variables with a factor b: S(bN, bV, bE) = bS(N, V, E). We call
quantities that scale in such as simple way extensive quantities.

6.6 The laws of thermodynamics

We now have the basis for the two first laws of thermodynamics, and we
have both a microscopic and a macroscopic interpretation of the laws of
thermodynamics.
In the microscopic view, we know that a system with constant N , V ,
and E has constant energy. This is indeed how we have set the system
up. On an atomic scale, the total energy is the sum of the kinetic and
potential energies of all the atoms in the system. and this energy is
conserved. This corresponds to the first law of thermodynamics –
the conservation of energy.
In the microscopic view, we characterize a system by the multiplicity
Ω(N, V, E), which we have been able to calculate for simple systems such
as the ideal gas and the ideal crystal. We have also found two basic laws:
• The system develops in time to the macrostate with the maximum
multiplicity
• In equilibrium, the system in the macrostate with the maximum
multiplicity
We have found that we can characterize a system by its microstates. We
assume that all microstates are equally probable. There are much more
microstates for some macrostates, and these most likely macrostates will
be the most likely observed macrostates in the system. Indeed, for realistic
system sizes — i.e. N = 1020 — deviations from the maximum values of
the macrostates are extremely unlikely. The standard deviation of the
distribution of macrostates is about 10−10 of the value at the maximum.
168 6 Micro- and Macro-states

These distributions are therefore so sharp that we can characterize the

system by the value of the most likely macrostate alone.
These two laws are called the second law of thermodynamics. Here,
they have a microscopic formulation. How can we formulate similar laws
on the macroscale?
The first law of thermodynamics is easily transferrable. The total
energy of the system is converved. We call this the internal energy E.
Any change in the internal energy must come from an external source:
In the form of mechanical work W performed on the system or in the
form of thermal energy Q transferred into the system. Hence, we can
formulate the first law of thermodynamics:

First law of thermodynamics:

∆E = W + Q , (6.91)

Macroscopically, we have introduced the quantity S for the entropy of

the system, and entropy is related to the multiplicity of the system with
constant N , V , and E by

S = k ln Ω(N, V, E) . (6.92)

The two microscopic laws that (i) the system evolves towards larger mul-
tiplicity and (ii) that in equilibrium the system has maximum multiplicity
therefore corresponds to

The second law of thermodynamics:

An isolated system with constant N , V , E develops towards
larger entropy, ∆S ≥ 0, and in equilibrium, the entropy is at its
maximal value.

These are the fundamental laws we need to understand the macroscopic

world. We can use our microscopic understanding to understand the
foundation of the macroscopic laws, and to calculate the relationships
for quantities such as the entropy.
In addition, we usually supplement with two additional laws:
6.6 The laws of thermodynamics 169

The zeroth law of thermodynamics:

If systems A and B are in thermal equilibrium and systems B
and C are in thermal equilibrium, then system A and C are also in
thermal equilibrium.

This follows from the equation for thermal equilibrium - and tempera-
ture: If TA = TB and TB = TC then, indeed, TA = TC .
In addition, we will later also introduce a third law of thermodynamics,
which allows us to set an absolute scale for the entropy.

The third law of thermodynamics: The entropy of a system

approaches a constant as the temperature approaches zero.

These are the fundamental laws of thermodynamics. From these laws

we will be able to develop the complete theory of thermodynamics. We
will now used these laws, along with our microscopical theory that allows
us to calculate the entropy and the temperature of an isolated system,
in order to address the behavior of realistic, macroscopic systems.

Clickers: Thermal02

6.6.1 Example: Entropy and heat

Let us now use the laws of thermodynamics to discuss a process without
resorting to a microscopic description. Let us address the entropy change
for a real process where thermal energy, heat, is transferred from one
system to another.
First, we need to be precise in how we define the system we study.
We start with an isolated system that consists of two subsystems, A
and B. Initially, the two systems have temperatures TA and TB , energies
EA and EB , and entropies, SA and SB . The systems are also isolated
from each other. But what happens if we now transfer a small amount
of thermal energy, ∆E, from A to B, so that the total energy in the
system is conserved. This means that the change of energy in system A
is ∆EA = −∆E and the change of energy in system B is ∆EB = ∆E.
Change in entropy. The entropy change in the whole system is
170 6 Micro- and Macro-states

∆S = ∆SA + ∆SB
∂SA ∂SB
   
= ∆EA + ∆EB
∂EA NA ∂EB NB (6.93)
∂SA ∂SB
   
= (−∆E) + ∆E
∂EA NA ∂EB NB
We can now use the general relation between temperature and the
derivative of the entropy for each of the two systems:
1 ∂SA 1 ∂SB
   
= , = . (6.94)
TA ∂EA NA TB ∂EB NB

The total change in entropy is therefore:

1 1
 
∆S = ∆SA + ∆SB = − + ∆E . (6.95)
TA TB
Because the whole system is isolated, the total entropy must increase (or
be the same) in this process. This means that TA must be larger than TB
for this process to be thermodynamically possible. (Meaning probable:
Any other process would be very unlikely). This is, of course, in perfect
alignment with our intuition: Heat is transferred from objects with high
temperature to objects with lower temperature and not the opposite way.
Heat and entropy change. How is the heat transferred between the two
systems related to the entropy change in the systems? Since there is no
other energy transfer than heat between systems A and B, we know that
the change in energy corresponds to heat transferred. From the first law
of thermodynamics we know that ∆EA = QA + WA , and since WA = 0,
we see that QA = ∆EA . Similarly, for system B, QB = ∆EB . It is now
simple to relate the entropy change and the change in energy for each of
the systems. As long as the systems do not change volume or particle
number, we can use the definition of temperature to find the change in
entropy:
1 ∂S dE
 
= ⇒ dS = . (6.96)
T ∂E N,V T
This means that we can estimate the change in entropy from the heat:
Q
dS = . (6.97)
T
or we can estimate the heat from the change in entropy:
6.7 Mechanical equilibrium — Pressure 171

Q = T dS . (6.98)

Numerical example. For example, if a hot object, A, at TA = 1500 K is

put in contact with a cold object, B, at TB = 300 K, and the thermal
energy transfered during 60 s is 150 J, then we can calculate the changes
in entropy through:
−150 J
∆SA = = −0.1 J/K , (6.99)
1400 K
and
150 J
∆SB = = 0.5 J/K . (6.100)
300 K
We see that the total change in entropy is indeed positive, as it must for
this process to be possible according to the second law of thermodynamics.

6.7 Mechanical equilibrium — Pressure

So far we have developed a theory for a fluctuation in the energy of a

system: We divided an isolated system into two parts, and allowed a
small amount of energy, ∆E, to be transported from one part to another,
and found the equilibrium state as a function of the derivative of the
entropy. Now, let us see if we can expand this argument also to systems
where we allow also the volume of the system to vary and futher on, also
the number of particles to vary. But we will now use a thermodynamics
argument without resorting to the microscale. Can we pull this off?
We address the behavior of an isolated system divided into two parts
A and B. The systems are divdided by a piston: A wall that can transfer
thermal energy and mechanical energy by moving. The wall is not perme-
able, so the number of particles is constant on each side of the wall. The
total energy and volume is conserved: EA + EB = E and VA + VB = V , as
well as the number of particles in each subsystem NA , NB , as illustrated
in Fig. 6.18.
Intuitively, what would we expect to determine the equilibrium in
such as system? We already know that for thermal equilibrium to occur,
the two systems must have the same temperature, otherwise we could
increase entropy by transferring some energy from one part to another.
But what would determine mechanical equilibrium? We would expect
the piston to be in equilibrium only if the force acting from each part of
the system on the piston sum to zero, so that the net force on the piston
172 6 Micro- and Macro-states

Fig. 6.18 Illustration of

an isolated system divided A ΔE B
into two parts by a piston
so that thermal energy and
volume may be exchanged
between the systems, but
the total energy and volume
is conserved. ΔV

EA + EB = E
V A + V B= V
NA , N B

is zero. This occurs only if the pressure in each of the parts are the same,
since the force from part A on the piston is FA = pA AA , where FA is
the force and pA is the pressure, and similarly for part B, FB = pB AB .
Here, the areas on each side of the piston are the same, so that the
pressures also must be the same. Equilibrium would therefore require
equal pressures.
What are the equilibrium condition from a thermodynamical per-
spective? We know from the second law of thermodynamics, that an
isolated system — the whole system including both part A and B — is
in equilibrium when the entropy of the whole system is maximum. The
total entropy is the sum of the entropies of each subsystem:

ST OT = SA (NA , VA , EA ) + SB (NB , VB , EB ) , (6.101)

where VA + VB = V , EA + EB = Em NA , and NB are constants, so that

VB = V − VA and EB = E − EA . The entropy is therefore a function
of the macrostate described by EA and VA , and we want to find the
macrostate that maximizes the entropy. This corresponds to finding the
maximum of a two-dimensional function, which occurs as a maximum
along both axes, as illustrated in Fig. 6.19. The maximum (or extremum)
occurs when dS = 0, that is, when
∂SA ∂SA
   
dS = dSA + dSB = dEA + dVA (6.102)
∂EA NA ,VA ∂VA NA ,EA
∂SB ∂SB
   
+ dEB + dVB = 0 , (6.103)
∂EB NB ,VB ∂VB NB ,EB
6.7 Mechanical equilibrium — Pressure 173

where we now insert that dEB = −dEA and dVB = −dVA (from EA +
EB = E = const. and VA + VB = V = const.), getting:
!
∂SA ∂SB
   
dS = − dEA (6.104)
∂EA NA ,VA ∂EB NB ,VB
!
∂SA ∂SB
   
+ − dVA = 0 . (6.105)
∂VA NA ,EA ∂VB NB ,EB

For this to be true for any small dEA and dVA , we get the two conditions:
∂SA ∂SB ∂SA ∂SB
       
= , = .
∂EA NA ,VA ∂EB NB ,VB ∂VA NA ,EA ∂VB NB ,EB
(6.106)
The first condition is the same condition we found for a purely thermal
contact: The temperatures in the two parts must be the same in equilib-
rium. We guess that the second term is related to the pressures in the two
parts. But how is it related to the pressure? We can gain some insight
into this by looking at the units of this expression: dS/dV . We notice
that entropy has dimensions J/K=Nm/K, V has dimensions m3 , and
pressure has dimensions N/m2 . The derivative of S with respect to V
therefore has dimensions (N m / K ) / m3 , that is ( N / m2 ) / K. We must
therefore multiply (dS/dV ) by a temperature to get dimension pressure.
Since T is the same in both systems in equilibrium we can multiply with
T , getting to a definition of pressure:

Definition of pressure in an isolated system:

∂S
 
p=T , (6.107)
∂V E,N

This is not the only way to define pressure. We could for example also
have included a multiplicative or an additive constant, but we will see
below that this definition of pressure does reproduce the correct ideal
gas law and other know features of the systems we are interested in.

Clickers: Thermodynamics 04
174 6 Micro- and Macro-states

(a) STOT (b)

-3

-3.5

-4

-4.5

 
-5

-5.5

-6

-6.5
1
EA 0.8 1
0.6 0.8
0.4 0.6
0.4
0.2
0.2
VA   0 0
 

Fig. 6.19 a Illustration of the entropy as a function of EA and VA . The maximum of the
entropy occurs where the derivatives with respect to both EA and to VA are zero. b Plot
of the entropy, S = SA + SB , as a function of EA and VA for two ideal gases in thermal
and mechanical contact.

6.7.1 Example: Pressure of ideal gas

We are now in a position to calculate the pressure of a system, given
that we know the entropy. For an ideal gas, we found that the entropy
was given by Sackur-Tetrode’s equation:

4πmE 5
3/2 !
V

S = N k ln + . (6.108)
N 3N h2 2

However, we only need the volume dependence to find the pressure:

S = N k ln V + f (N, E) . (6.109)

From (6.107) the pressure in the ideal gas is:

∂S Nk
 
p=T =T (6.110)
∂V E,N V

which corresponds to
pV = N kT , (6.111)
which indeed is the ideal gas law. This also shows we should not in-
clude any other factors in the definition of the pressure in (6.107) — it
reproduces the correct ideal gas law at it is.

Clickers: Thermal 03
6.8 The thermodynamic identity 175

6.8 The thermodynamic identity

We now know enough to develop a differential for the entropy: We would

like to see how the entropy S changes with a change in E and V . We do
this by writing out the differential for the change dS:
∂S ∂S ∂S
     
dS = dE + dV + dN . (6.112)
∂E V,N ∂V E,N ∂N E,V

We have not looked at changes in particle number yet. Let us therefore

for now assume that dN = 0. The other two expressions we now know
∂S 1 ∂S 1
   
= , = p. (6.113)
∂E V,N T ∂V E,N T

We insert these into the differential in (6.112):

1 p
dS = dE + dV ⇒ T dS = dE + p dV . (6.114)
T T
This equation is true for any infitesimal change in any system, as long as
p and T are well defined and N is constant. It is a fundamental equation
in thermodynamics, which we will use many times throughout this book.
It is called

The thermodynamic identity:

T dS = dE + p dV − µ dN . (6.115)

(We have not yet allowed N to vary or introduced the chemical potential
µ, but we write the equation in this form for completeness. When N is
constant, dN = 0, and the last term disappears).

6.8.1 The thermodynamic identity and partial derivatives

This equation, in combination with the first and second law of thermody-
namics will be our main tool to address thermodynamic processes, as we
will see in examples and problems. However, this equation is also a very
compact way to remember the various derivatives of the entropy. From
(6.115) we can easily find the various partial derivatives by applying
simple algebra to the differentials. Let us demonstrate by examples.
176 6 Micro- and Macro-states

First, what is the derivative of S with respect to E when V and N

are constant? We can see this directly from the thermodynamic identity,
using that dV = 0 and dN = 0:

T dS = dE + p |{z}
dV −µ |{z}
dN (6.116)
dV =0 dN =0
dS 1
= . (6.117)
dE T
which gives us:
∂S 1
 
= . (6.118)
∂E N,V T
Similarly, we find the derivative of S with respect to V when E and N
are constants? In this case, dE = 0 and dN = 0, which we insert into
the thermodynamic identity in (6.115), getting

T dS = |{z}
dE +p dV − µ |{z}
dN (6.119)
dE=0 dN =0
dS p
= . (6.120)
dV T
which gives us:
∂S p
 
= . (6.121)
∂V N,E T
This is a very useful technique, which we will use again and again in
various disguised throughout this text. Learn and practice it, and you
will only have to remember a very few definitions and laws in order to
master all of thermodynamics.

Clickers: Thermal 04

6.8.2 Entropy, Heat and Work

If we compare the thermodynamic identity

T dS = dE + p dV − µ dN , (6.122)

which when dN = 0 can be rewritten as

dE = T dS − p dV , (6.123)
6.8 The thermodynamic identity 177

with the first law of thermodynamics

dE = Q + W , (6.124)

We recongnize that the work done on a system, such as a piston, with a

pressure p, when the piston is moved so that the volume changes by a
small amount dV , is W = p dV . Notice the sign! The work W in the first
law is the work done on the system. If the pressure inside the system is
p and the pistons expands so the dV > 0, the system inside the piston
does a positive work pdV on the environment, whereas the work done
on the system is the negative of this, W = −p dV .
Similarly, if there is no mechanical work done on the system, then the
change in energy corresponds to the heat, Q, transferred into the system.
Therefore, we may conclude that Q = T dS in this case.
However, these results are only correct when the processes we consider
are sufficiently close to the equilibrium, that is, when the processes
are quasistatic. Let us now briefly review how we can describe various
thermodynamic processes.

6.8.3 Process description

We describe thermodynamic systems by the macrostate variables of
the system, such as by S, E, V , N , T , and p. Various thermodynamic
processes are described by how we change these variables and by how we
change the system.
We discern between reversible and irreversible processes. A re-
versible process is a process that may go both ways — backward and
forward. That is, the process has time symmetry. This means that the
entropy change for a reversible process in an isolated system must be
zero. If it was positive, then the reverse process cannot occur since it
would lead to a reduction in entropy.
If we always keep a system very close to its equilibrium state, we
call the process quastistatic. This is an ideal case, where the process
occurs infinitely slowly so that the system always has time to reach
thermodynamic equilibrium between each time the system is changed.
In addition, we will introduce various words to describe processes
where some of the thermodynamic variables are constant. Usually, we
will use the prefix iso for the constant variable. For example, a process at
constant temperature is called isothermal, whereas a process at constant
pressure is called isobaric. The various processes are
178 6 Micro- and Macro-states

• quasistatic processes where the system is changed so slowly that

it is at equilibrium at each step
• adiabatic processes where there is no thermal energy (heat) trans-
ferred to the system
• isentropic processes = adiabatic + quasistatic processes
• isobaric processes = processes at constant pressure
• isochoric processes = processes at constant volume
• isothermal processes = processes at constant temperature
• isoenergetic processes = processes at constant energy
Most of the theory we develop will be for quasistatic processes. Just be
careful about what happens when a process is not quasistatic, then we
may have a change in entropy that is larger than what you would expect
from work or heat alone.

6.9 Diffusive equilibrium — Chemical potential

We have now seen that two systems that are in thermal contact, that is
they are allowed to exchange energy, are in equilibrium when the temper-
atures are the same. If the two systems also are in mechanical contact,
that is they are allowed to exchange volume, they are in equilibrium
when also the pressures are the same. What if we also allow the two
systems to exchange particles — what is the equilibrium condition then?
Two systems that are allowed to exchange particles are said to be in
diffusive contact. We divide an isolated system into two parts, A and B,
so that the total energy, volume and particle number is conserved:

EA + EB = E ⇒ EA = E − EB dEA = −dEB (6.125)

VA + VB = V ⇒ VA = V − VB dVA = −dVB (6.126)
NA + NB = N ⇒ NA = N − NB dNA = −dNB . (6.127)

This system is in equilibrium for the macrostate (EA , VA , NA ) that makes

the total entropy maximum, which occurs when dST OT = 0, where the
differential is
6.9 Diffusive equilibrium — Chemical potential 179

∂SA ∂SA ∂SA

     
dS = dEA + dVA + dNA
∂EA NA ,VA ∂VA NA ,EA ∂NA EA ,VA
(6.128)
∂SB ∂SB ∂SB
     
+ dEB + dVB + dNB = 0 ,
∂EB NB ,VB ∂VB NB ,EB ∂NB EB ,VB
(6.129)

which gives
!
∂SA ∂SB
   
dS = − dEA (6.130)
∂EA NA ,VA ∂EB NB ,VB
!
∂SA ∂SB
   
+ − dVA (6.131)
∂VA NA ,EA ∂VB NB ,EB
!
∂SA ∂SB
   
+ − dNA = 0 , (6.132)
∂NA EA ,VA ∂NB EB ,VB

which again gives the following conditions:

∂SA ∂SB
   
= , (6.133)
∂EA NA ,VA ∂EB NB ,VB
∂SA ∂SB
   
= , (6.134)
∂VA NA ,EA ∂VB NB ,EB
∂SA ∂SB
   
= . (6.135)
∂NA VA ,EA ∂NB VB ,EB

We introduce this new condition as the chemical potential, µ, which has

units of energy and must be the same in both systems when the two
systems are in equilibrium:
∂S
 
µ = −T . (6.136)
∂N E,V

For two systems in diffusive equilibrium, the chemical potential is the

same in both systems:
µA = µB . (6.137)
We will build more intuition for the chemical potential in Chap. 9. Here
we have simply introduced it for consistency.
Notice that the minus sign ensures that particles flow from systems
with high chemical potential to systems with low potential. We can see
this from ∂S/∂N . The system with the largest value for this will gain
180 6 Micro- and Macro-states

the most entropy by adding particles. Hence, when we use minus this
value we ensure that particles flow from high to low chemical potential.

6.9.1 Thermodynamical identity

With this new definition, all the quantities in the thermodynamic identity
are now fully introduced. The thermodynamic identify therefore becomes:
∂S ∂S ∂S
     
dS = dE + dV + dN
∂E ∂V ∂N
V,N E,N E,V
(6.138)
1 p µ
= dE + dV − dN ,
T T T
which gives:
T dS = dE + p dV − µ dN , (6.139)
and
dE = T dS − p dV + µ dN . (6.140)
This expression allows us to gain new intuition into the interpretation of
the chemical potential. For a process where S and V is fixed, we see that

dE = µ dN , (6.141)

which tells us that

∂E
 
µ= . (6.142)
∂N S,V

The chemical potential therefore is change in energy per particle added

to the system, when the particle is added so that the volume and the
entropy of the system does not change. We will return to better, and
more intuitive definitions of the chemical potential in Chap. 9.

6.9.2 Example: Chemical potential of the Einstein crystal

When we know the entropy of a system, S = S(N, V, E), we can use this
to find the chemical potential using (6.136). For the Einstein crystal, we
found that the entropy was

S = N k ln E − N k ln(N ) + N k . (6.143)

The chemical potential is therefore

6.9 Diffusive equilibrium — Chemical potential 181

∂S
 
µ = −T
∂N E,V
= −T (k ln E + k − k ln(N ) − k)
(6.144)
= −T (k ln E − k ln N )
E
= −kT ln  .
N

6.9.3 Example: Chemical potential of the ideal gas

Similarly, we can find the chemical potential for the ideal gas from the
Sackur-Tetrode equation for the entropy:

4πmE 5
3/2 !
V

S = Nk +
N 3h2 N 2
(6.145)
4πmE 5
 3/2 !
= N k ln V − ln N 5/2
+ .
3h2 2

We take the derivative with respect to N to find the chemical potential:

∂S
 
µ = −T
∂N E,V

4πmE 5 5 1
 3/2 !
= −T k ln V − ln N 5/2
+ − T Nk
3h2 2 2N
(6.146)
4πmE
3/2 !
V

= −T k ln
N 3N h2
2πmkT
3/2 !
V

= −T k ln
N h2

where we have inserted that E = (3/2)N kT for the ideal gas. We see that
the chemical potential depends on the density: Increasing the density
(N/V ) while keeping T constant would mean that (V /N ) becomes smaller,
so that the chemical potential becomes larger — the system becomes
more willing to give away particles.

Question
Clickers: Heat capacities
182 6 Micro- and Macro-states

6.10 Simulations of the microcanonical system

We have already seen several approaches to simulation of a microcanonical

system: The molecular dynamics approach, where energy conservation is
ensured by Newton’s laws of motion and conservative forces; Stochastic
models, such as for the gas; and Monte Carlo methods, such as for the
development of the Einstein crystal. Here, we will discuss these simulation
methods systematically. Typically, we use simulation methods when we
cannot find a theoretical solution for the system we are interested in.
In that case, we need several approaches that can be adapted to the
problem at hand.

6.10.1 Molecular dynamics simulations

The default molecular dynamics model is a simulation of a system in
the microcanonical system: The number of particles, N , the volume, V ,
and the energy, E, is conserved throughout the simulation. We have seen
examples of simulations of a crystal and of a gas — and we can simulate
any system you can think of using this method.
A molecular dynamics simulation produces a sequence of positions and
velocities of all the particles in the system, a sequence of microstates as
a function of time. However, since the time step in a molecular dynamics
simulation is small, the practical upper limit of a simulation is currently
around one microsecond. Now, we do not always know if the simulation
is able to span over a reasonable set of microstates during this time span.
For a liquid at high temperatures, a simulation will typically cover a
wide enough area of the possible space of microstates, but for water in
ice form or for a glass, only very small changes in the microstates will
occur during the whole simulation. We call a system ergodic if the time
sequence of microstates gives a representative view of the microstates
of a system. In principle, we expect all systems to be ergodic, but in
practice this may take too long for a simulation to be realized. We must
therefore always critically evaluate if a molecular dynamics simulation is
sufficiently close to equilibrium to provide relevant statistics.
What can we then use a molecular dynamics simulation for? We have
already seen that we can use it to predict fluctuations in variables inside
a system: We can use it to measure fluctuations in energy or the number
of particles in two parts of a gas, or we can use it to measure fluctuations
in energy in a solid. We can also measure temperature or pressure directly
6.10 Simulations of the microcanonical system 183

from the simulation. Typically, we will not find the temperature from
the entropy, but instead we will measure the temperature directly from
the average kinetic energy per particle (or degree of freedom) in the
system. We will provide many examples of the use of molecular dynamics
simulations throughout this book — hopefully these examples will provide
you with ideas and inspiration for your own use of similar methods if
you need them in your future work.

6.10.2 Stochastic simulations

Ideally, we would like to count all the microstates of a system, and use this
to find the multiplicity, and from the multiplicity the entropy. However,
this may not always be possible. We may then apply molecular dynamics
simulations, which would provide a realistic model of the system, but
only a small portion of the set of microstates of the system is probed. A
different strategy would be to choose a random sequence of microstates,
where each of the microstates are independent of each other, and then
use this set of microstates to estimate the probability and therefore also
the multiplicity of a macrostate.
Stochastic model for the sum of dice. We have already applied this
strategy to model statistical experiments. For example, if we were in-
terested in describing the probabilty distribution for the sum of 1000
six-sided dice.
This particular problem can of course be solved analytically, but let
us see how we would approach it using stochastic simulations. The idea
is to generate many microstates, where each microstate corresponds to
the result for each of the 1000 dice, and then use the set of microstates
to find the probabilities for the macrostates. If the set is representative,
we expect good results.
In this case, a microstate is a set (n1 , n2 , . . . , nN ) if N independent
dice throws, so that each ni is a random number from 1 to 6. The total
number of microstates is 6N , which is a very large number when N = 1000.
How can we expect to sample this space reasonably? The number of
macrostates is much smaller. A macrostate is the sum Z = i ni , which
P

can vary from N to 6N . We perform this statistical experiment in python

by generating 10000 experiments, each consiting of N = 1000 random
numbers:
from pylab import *
n = randint(0,6,(1000,10000))
184 6 Micro- and Macro-states

Z = sum(n,axis=0)
hist(Z), show()

and plot the resulting distribution of probabilties found from the his-
togram in Fig. 6.20. Notice that the histogram counts the number of
results, N (zi ), in intervals from zi to zi + ∆zi . If the bin size ∆zi is
non-zero, we need to divide by the bin size as well as the number of
samples to estimate the probability P (zi ) (see Sect. 4.4.4 for a discussion
of bin sizes and probabilities). Fig. 6.20 shows that the distributions
P (z) approaches a continuous distribution as the number of samples M
increases. Indeed, when M = 104 the distribution already appears to
give a good representation of the distribution, even though only a very
small fraction of the number of microstates have been probed. However,
since most of the randomly selected microstates will be in the region
with the most probably macrostates, we still get good estimates for the
probabilities of the most probable macrostates. On the other hand, the
results for the very unlikely macrostates will be poor, since there will be
few if any data points in this range of Z values.
Stochastic model for the Einstein crystal. We can apply a similar
approach to sample the microstates of the Einstein crystal, and then
use these sampled microstates to calculate the energy qA of part A of
the crystal and qB of part B, similar to what we did previously. Let us
assume that there are NA oscillators in part A and NB in part B, so that
there are N oscillators in total, and that the total energy of the system is
q. How can we generate a random microstate? We have q units of energy
and N oscillators to distribute this energy randomly onto. We do this
by placing each unit of energy in a randomly selected oscillator. That is,
for each unit of energy, we select an oscillator at random, and increase
the energy of this oscillator by one unit. When this has been done for all
q energy units, we have generated a microstate, (n1 , n2 , . . . , nN ), where
ni is the number of energy units in oscillator i. The total energy is then
q = i ni . We choose part A to be the first NA oscillators, that is, part
P

A corresponds to i = 1, . . . , NA , and part B corresponds to the remaining

NB oscillators, i = NA + 1, . . . , NA + NB . We can therefore calculate qA
and qB as
NA NAX
+NB
qA = ni , qB = (6.147)
X
ni .
i=1 i=NA +1

We generate M such microstates and calculate the energy of part A,

qA (j) for each such microstate j. From this sample of microstates we
can estimate the probability distribution for qA . The following program
6.10 Simulations of the microcanonical system 185

generates states and collects statistics following this stochastic modeling

approach:
from pylab import *
N = 200; q = 200; M = 1000
NA = N/2; NB = N/2;
EA = zeros(M); EB = zeros(M)
for istep in range (M):
state = zeros(N)
# Generate random state with given energy
placeA = randint(0,high=N,size=q);
for ip in range(len(placeA)):
i = placeA[ip]; state[i] = state[i] + 1;
# Collect statistics for this state
qA = sum(state[:NA]); qB = q-qA;
EA[istep] = qA; EB[istep] = qB;
hist(EA[:istep])
xlabel(’q_A’); ylabel(’N(q_A)’)
show()

The resulting distribution for P (qA ) is shown in Fig. 6.20 for various values
of M . We see that the distribution approaches a smooth function for
P (qA ) when M is very small compared to the total number of microstates
(which is the multiplicity of the Einstein crystal — a very huge number
as we know).
Notice that for this particular case, we know the exact solution to
the problem. However, in other situations you may not know the exact
solution, and this form for Stochastic sampling may be an efficient way
to learn about the probability for a macrostate in the system. Notice
that it may not always be this simple to generate a state with a given
total energy, and we may instead have to depend on a Monte Carlo type
model for the system.

0.02
Fig. 6.20 Plot of the esti-
M=10 2
mated probability density
N (qA )=(M "qA )

0.015 M=10 3
P (qA ) for the energy qA M=10 4

0.01 M=10 5
in the left half of an Ein-
stein crystal for various 0.005
sample sizes M . System
0
with N = 200, q = 2000, 920 940 960 980 1000 1020 1040 1060 1080 1100
qA
NA = NB = N/2.

Stochastic model for the Ideal gas. We have already develop a stochas-
tic model for the number of particles in a part of an ideal gas. A microstate
of the gas may be the position of all the particles, or simply whether
particle i is on the left side (ni = 1) or not (ni = 0). The microstate
186 6 Micro- and Macro-states

of the system would then be a list of the positions of all the particles,
(n1 , n2 , . . . , nN ), and the number of particles on the left side is n = i ni .
P

Since all microstates are equally probable, we simply select each ni to be

1 or 0 with probability 1/2 each, and generate a set of M microstates.
This is what we did in Sect. 4.4.4, where we also studied the resulting
distribution of n.

6.10.3 Monte Carlo simulations

In the stochastic model we generate a sequence of uncorrelated mi-
crostates: For each sample we generaet a new microstate randomly,
ensuring no correlation with the previous state. However, in many cases
it may be difficult to generate a microstate from scratch, but it may be
simple to make a small perturbation to the microstate, or we may be
interested in studying a slow change from an initial microstate of the
system. For example, we may want to study how heat is redistributed
in an Einstein crystal, either locally or globally. This means that we
start from a microstate s = {ni }, which specifies the state, ni , of each
particle/degree of freedom i in the system, and then make small changes
to the microstate, while ensuring that the total energy, particle number,
or volume are not changed.
This suggested procedure will result in a sequence s(j) of microstates.
Every such microstate is indeed a possible microstate, and hence equally
probable, but two subsequent microstates are not independently selected
from the set of all microstates. Instead, they are correlated. This means
that we need to sample many such states — a long sequence — before
we make statistical predictions of the probability of a macrostate.
We call this method a Monte Carlo method for the microcanonical
system. We have already introduced possible Monte Carlo method for
the Einstein crystal, and we will extend this to also include a Monte
Carlo method the introduces locations for all the oscillators and rules for
possible transitions between microstates simulating local energy trans-
port.
Global dynamics of the Einstein crystal. We already introduced a
simple model for global redistribution of energy in an Einstein crystal in
Sect. 6.3.3:
1. Generate initial, random microstate.
2. Select a random oscillator to send energy, i1 .
6.10 Simulations of the microcanonical system 187

3. Select a random oscillator to receive energy, i2 .

4. If ni,1 > 0 then ni,1 → ni,1 − 1 and ni,2 → ni,2 + 1.
5. Loop to step 2.
As we saw in Fig. 6.9 this model generates and intersting dynamics and
it allows us to study how the system approaches equilibrium and how it
fluctuates around equilibrium. However, we may add further complexity
(and realism) to the model, by insisting that each oscillator is located in
a position in space, and that an oscillator only can transfer energy to one
of its neighboring oscillator, thus opening for energy (heat) transport.
Local dynamics of the Einstein crystal. We add another level of com-
plexity to the model, by introducing local interactions between the
oscillators. All oscillators are placed on a square, two-dimensional grid.
Oscillator i will be at position xi , yi . We follow the same algorithm as
above: We select an oscillator at random, and transfer energy to another
oscillator – if possible. However, we now only allow the oscillator to
transfer energy to one of its neighbors. We select the neighbor at random
from the four nearest neighbors xi +1, yi , xi −1, yi , xi , yi +1, and xi , yi −1.
This results in local energy transport as illustrated in Fig. 6.21. You can
use the following program to study heat conduction in this simple model
of thermal behavior in a solid.
# Energy transport in an Einstein crystal
Ly = 40 # System size y-direction
LA = 40 # Length of system A in x direction
LB = 40 # Length of system B in x direction
Lx = LA + LB
NA = LA*Ly
NB = LB*Ly
qA = 3000 # Initial energy in system A
qB = 0 # Initial energy in system B
q = qA + qB # Total energy - conserved
N = NA + NB # Total number of oscillators
state = zeros((Lx,Ly),float) # 2d state matrix
# Generate initial, random states for A and B
for ia in range(qA):
ix = randint(0,LA) # Rnd position from 1 to LA
iy = randint(0,Ly) # Rnd position from 1 to Ly
state[ix,iy] = state[ix,iy] + 1 # Add energy to this site
for ib in range(qB):
ix = randint(0,LB)+LA # Rnd pos from LA+1 to LA+LB
iy = randint(0,Ly) # Rnd pos from 1 to Ly
state[ix,iy] = state[ix,iy] + 1 # Add energy to this site
# Simulate state development
nstep = 10000000 # nr of simulation steps
EA = zeros(nstep,float) # Energy per oscillator in system A
EB = zeros(nstep,float) # Energy per oscillator in system B
188 6 Micro- and Macro-states

ion() # Interactive drawing ON

for istep in range(nstep):
# Select an oscillator at random
ix1 = randint(0,Lx)
iy1 = randint(0,Ly)
# Check if this oscillator has non-zero energy
if (state[ix1,iy1]>0):
# Find a random neighbor
dx = 2*randint(0,2)-1 # +/-1 with equal prob
ix2 = mod(ix1 + dx-1,Lx) # mod for periodic boundaries
dy = 2*randint(0,2)-1 # +/-1 with equal prob
iy2 = mod(iy1 + dy-1,Ly) # mod for periodic boundaries
# Transfer energy from (ix1,iy1) to (ix2,iy2)
state[ix2,iy2] = state[ix2,iy2] + 1
state[ix1,iy1] = state[ix1,iy1] - 1
if (mod(istep,1000)==0): # Display system at regular intervals
imshow(state)
draw()

20 20 20

40 40 40
20 40 60 80 20 40 60 80 20 40 60 80
3000
qA

2900

2800
0 1 2 3 4 5 6 7 8 9 10
t #104
Fig. 6.21 Illustration of the time development of two two-dimensional Einstein crystals
in thermal contact.

6.11 Summary

• The multiplicity of a macrostate is the number of microstates in the

macrostate
• The Einstein crystal is a simple model for a solid where the atoms
do not interact and where each atom acts as a harmonic oscillator in
three dimensions
• For two Einstein crystals in contact we describe the systems by the
energy distribution between the two systems, described by the energy
q in one of the systems.
6.12 Exercises 189

• The macrostates for the two Einstein crystals do not have the same
number of microstates — some of the macrostates have vastly more
number of microstates that others.
• A system develops towards the most probably macrostate
• The number of microstates for a given macrostate is called the multi-
plicity of the macrostate
• The multiplicity of an isolated system is Ω(N, V, E).
• The entropy of an isolated system is defined as S = k ln Ω(N, V, E)
• The temperature of an isolated system is (1/T ) = (∂S/∂E)N,V
• The pressure of an isolated system is p = T (∂S/∂V )E,N
• The chemical potential of an isolated system is µ = −T (∂S/∂N )E,V
• The first law of thermodynamics states that ∆E = Q + W
• The second law of thermodynamics states that the entropy of an
isolated system must increase of be constant in any process. In equi-
librium the entropy is at a maximum.
• The thermodynamic identity is T dS = dE + p dV − µ dN
• The heat capacity of a system at constant volume is CV = (∂E/∂T )V

6.12 Exercises

Exercise 6.1: Microstates and macrostates of a random walk

The behavior of an atom or a molecule in a fluid with many atoms or
molecules can be considered to behave like a random walker. We will
here address the micro- and macrostates of a one-dimensional random
walker as a model for atomic (or molecular) diffusion.
The position ZN of a random walker after N steps can be found by
adding each of the steps, Xi , i = 1, . . . , N :
N
Z= (6.148)
X
Xi .
i=1

Here, we will assume that each step can be either to the left, Xi = −1,
or to the right, Xi = +1, with equal probability.
The microstate for such a walker will correspond to specifying each of
the steps, Xi , whereas a macrostate will refer to the position ZN after
N steps.
a) Write down all the microstates for a N = 4 steps random walk.
190 6 Micro- and Macro-states

b) Find the probabilities for each of the possible macrostates based on

the list of microstates.
c) Compute the multiplicity of each of the macrostates using the biono-
mial formula and compare with your results from counting all possible
states.

Exercise 6.2: Flipping 30 coins

You flip 30 fair coins. A possible macrostate could be defined as the
number, n, of heads.
a) How many possible outcomes (microstates) are there of this experi-
ment?
b) How many ways can you get n = 15?
c) What is the probability of n = 15?
d) What is the probability of n = 20?
e) What is the probability of n = 25?
f) What is the probability of n = 30?
g) Make a program to plot the probability, P (n), to get n.

Exercise 6.3: Stirling’s approximation

a) Use python to check the accuracy of Stirling’s approximation for
N = 50 and N = 100.
b) Make a plot to illustrate the accuracy of ln N ! = N ln N − N as a
function of N .

Exercise 6.4: Entropy

In a generic system with N particles the multiplicity is given as
Ω(E, N ) = CE 3N/2 , where C is a constant.
a) Show that E = (3/2)N kT .
6.12 Exercises 191

b) Show that !
∂2S
<0, (6.149)
∂E 2 N
and discuss what this means.

Exercise 6.5: Gas in one- and two-dimensions

a) Find an expression for the multiplicity of an ideal gas of N indistin-
guishable atoms, where each atom only can move along a one-dimensional
tube, such as along a narrow nanotube.
b) Find an expression for the multiplicity of an ideal gas of N indistin-
guishable atoms, where each atom only can move along a two-dimensional
flat sheet, such as for electrons in a graphene sheet.
 The canonical system
7

We have so far studied isolated systems, systems with constant N , V ,

and E, such as the systems we studied using molecular dynamics. Such
systems are called microcanonical systems. Microcanonical systems are
theoretically useful, but they are not equally practical from an experimen-
tal perspective. If you do an experiment inside a box in the lecture hall,
you would typically expect the system inside the box to be in thermal
equilibrium with the lecture hall — you would expect the system to be at
the same temperature as the lecture hall. Thus, we are often interested in
systems that are kept at constant temperature and not at constant energy.
This requires contact with a large thermal reservoir that can exhange
energy with the system so that the system can keep an (approximately)
constant temperature without changing the temperature of the reservoir
significantly. For this system N , V , and T are constant, and we call this
a canonical systems.
Here, we want to develop the statistical mechanics and the thermo-
dynamics of canonical systems, systems with given N , V , and T . We
will assume that the system may be in any of its states, i, and we will
first find the probability for the system to be in a state i when it is
in equilibrium with a large reservoir. We will do this using arguments
from the microcanonical theory. We will demonstrate how we can link
the microscopic scale to the macroscopic scale for canonical systems
using the partition function. We will demonstrate that the second law of
thermodynamics — the the entropy of an isolated system must increase —
is replaced by a similar principle for the canonical system, that Helmholtz
free energy will decrease and be minimal in equilibrium.

193
194 7 The canonical system

7.1 Boltzmann statistics in the Einstein crystal

Let us start from a simple example system that we master well — the
harmonic oscillator. We will address the motion of a single oscillator in
contact with a large reservoir so that the temperature of the oscillator
is constant. The reservoir will consist of a huge amount of oscillators in
contact with the single oscillator. This is a system we now are proficient
in discussing and solving — we can therefore use our experience with
the harmonic oscillator system to gain insight into the canonical system.

7.1.1 Einstein crystal – numerical example

We already know how to study a system consiting of one harmonic
oscillator in contact with a system of may oscillators as illustrated in
Fig. 7.1, because this system corresponds to two Einstein crystals in
contact, a system for which we already have found an exact solution!

Fig. 7.1 Illustration of a

single harmoic oscillator A B
(part A) in contact with a ΔE
huge reservoir consisting of
harmonic oscillators (part
B). The total system is
isolated, so that for the
total system N , V , E are
given, whereas for part A
only VA and NA are given,
whereas qA may vary.
qA qB = E0-qA
VA VB
NA= 1 NB= N-1

Model system. Our previously developed model of an Einstein crystal

divided into two parts, A and B, can be used to represent the canonical
system in contact with a large reservoir. Part A is the canonical system
and part B is the reservoir, where NA = 1 and NB = N − NA = N − 1 
NA . The two parts A and B can exchange energy, but the total energy
is conserved, qA + qB = q, as is the number of particles, NA , NB , and
the volumes, VA and VB . The total system, consisting of system A and
system B, is an isolated system.
The probability for system A to have energy qA , which is the same as
the probability for system A to be in the state qA , since for this particular
7.1 Boltzmann statistics in the Einstein crystal 195

system there is one state per energy. The probability to find system A in
state qA is
Ω(qA , NA ) Ω(qB , NB )
P (qA ) = , (7.1)
ΩT
where qA can range from 0 to q, and Ω(qA , NA ) is the multiplicity of an
Einstein crystal with NA oscillators and qA energy, and is given by
!
q A + NA − 1
Ω(qA , NA ) = . (7.2)
qA

The total multiplicity ΩT is the sum of all the multiplicities for different
values of qA .
Calculation of probabilities. The probabilities P (qA ) are plotted using
the following program. The resulting plot in Fig. 7.2a shows that the
probability falls off very rapidly as qA increases.
# Einstein solid NA - NB probability
from pylab import *
from scipy.misc import comb
NA = 1
NB = 199
q = 200
N = NA + NB
# Find multiplicity of all macrostates
omegaA = zeros(q+1)
qAvalue = zeros(q+1)
omegaB = zeros(q+1)
omegaTOT = zeros(q+1)
# Loop through all macrostates and find multiplicity
for istate in range(0,q):
qA = istate
qAvalue[istate] = qA
omegaA[istate] = comb(qA+NA-1,qA)
qB = q - qA
omegaB[istate] = comb(qB+NB-1,qB)
omegaTOT[istate] = omegaA(istate)*omegaB(istate)
PA = omegaTOT/sum(omegaTOT)
plot(qAvalue[j],PA[j],’-o’)
xlabel(’q_A’), ylabel(’log_{10}P(q_A)’), show()

Characterizing the plot of P (qA ). How can we characterize the func-

tional form of P (qA )? There are several tricks from experimental and
observational physics on how to determine the underlying functional form
of an observed probability distribution. Based on ideas from statistics
we could expect it to be of an exponential or a power-law form. An
exponential form, means that the function P (qA ) will be on the form
196 7 The canonical system

0.8 0

0.6 -5

log10 [P (qA )]
P (qA )
0.4 -10

0.2 -15

0 -20
0 10 20 30 0 10 20 30
qA qA

0 0

-5 -5
log10 [P (qA )]

log10 [P (qA )]
-10 q=100 -10
q=200
q=300
-15 q=400 -15
q=500

-20 -20
0 10 20 30 0 0.1 0.2 0.3
qA qA =q
Fig. 7.2 Probability P (qA , NA = 1) for the Einstein oscillator with q = 200, N = 200.

∗
P (qA ) = Ce−qA /q , (7.3)

where C and q ∗ are constants. How can we test if the measured distri-
bution has this form? We notice (and this is a common trick that it is
useful for you to learn) that if we take the logarithm of (7.1.1), we get:
qA
ln P (qA ) = ln C − , (7.4)
q∗
which is a linear function of qA . This means that if we plot ln P along
the y-axis and qA along the x-axis, we expect the plot to be linear if the
functional form in (7.1.1) is correct. Fig. 7.2b shows the resulting plot.
Indeed, the curve is approximately linear — our theory formulated in
(7.1.1) seems to be reasonable!
Effect of temperature. Maybe our theory becomes better if we increase
q and therefore the number of possible values for qA ? We try this by
plotting curves for P (qA ) for several values of q using the following
program:
7.1 Boltzmann statistics in the Einstein crystal 197

from pylab import *

from scipy.misc import comb
NA = 1
NB = 199
qq = array([150 200 250 300 400 450 500]);
nqq = len(qq)
for iqq in range(0,nqq):
q = qq[iqq]
N = NA + NB
# Find multiplicity of all macrostates
nstate = q+1
omegaA = zeros(nstate)
qAvalue = zeros(nstate)
omegaB = zeros(nstate)
omegaTOT = zeros(nstate)
# Loop through all macrostates and find multiplicity
for istate in range(0,nstate):
qA = istate-1
qAvalue[istate] = qA
omegaA[istate] = comb(qA+NA-1,qA)
qB = q - qA
omegaB[istate] = comb(qB+NB-1,qB)
omegaTOT[istate] = omegaA[istate]*omegaB[istate]
PA = omegaTOT/sum(omegaTOT)
j = find(qAvalue<40)
subplot(2,1,1)
hold(’on’)
plot(qAvalue[j],log10(PA[j]),’-o’)
xlabel(’q_A’), ylabel(’log_{10}[P(q_A)]’)
subplot(2,1,2)
hold(’on’)
plot(qAvalue[j]/q,log10(PA[j]),’-o’)
xlabel(’q_A/q’), ylabel(’log_{10}[P(q_A)]’)
subplot(2,1,1), show()
subplot(2,1,2), show()

The resulting curves for ln P (qA ) are shown in Fig. 7.2c. Hmmm.
Interesting. The slope of the curves, and therefore the q ∗ value, apparently
depends on the total energy q in the system. How is this dependence?
We could measure the slope and plot it as a function of q to find out that
it appears that q ∗ appears to be proportional to q. And we can again
check this hypothesis by plotting ln P (qA ) as a function of qA /q. If q ∗ is
proportional to q we would then expect all the curves for various q-values
to fall onto a common curve. The resulting plot in Fig. 7.2d shows that
this is approximately the case. You can convince yourself that this is
indeed the case, and that the approximation becomes better for larger
N - and q-values, by redoing the plot for larger values of N and q.
Now, we have shown how P (qA ) depends on the total energy q in the
system. But if we think of the system as a single oscillator – oscillator
198 7 The canonical system

A – in contact with a large system of oscillators, we can think of the

rest of the oscillators as a large reservoir with a given temperature TB
describing system B. Previously, we found that that the temperature
of an Einstein crystal in the limit when q and N is large is given by
E = q∆ = N kT , so that the temperature in system B is proportional
to q, that is, q = cT , where c is a constant.
We can therefore sum up the results of our numerical study and the
approximate theoretical model we have introduced to describe the results:
qA
P (qA ) = Ce− cT , (7.5)

where C and c are constants. Now, let us see if we can make a theory for
this behavior.

7.1.2 Einstein crystal in heat bath — Theory

We have a complete methodology from the microcanonical system which
we can use to address such a problem: What is the probability for a
single oscillator to be in a particular state i 1 with energy i ?
We consider a big system, the reservoir or heat bath R, for example
an Einstein crystal consisting of many oscillators, in contact with the
system S, here consisting of a single oscillator as illustrated in Fig. 7.3.
(We called the system A and the reservoir B in the numerical example
above). We will assume that the reservoir is so large that it is not affected
by the system: If we place a hot cup of water in the auditorium, it will
cool down to the temperature of the auditorium without increasing the
temperature of the auditorium significantly. The air and other materials
in the auditorium is an example of a large heat bath.
System and reservoir. The composite system consisting of the system
and reservoir is isolated and is characterized by constant E, V , and
N . The composite system is therefore a microcanonical system and all
microstates in this system are equally probable. We can therefore find
the probability for the composite system to be in a particular microstate
and we can use this to find the probability for the system S to be in
a state i. If the system S is in a state with energy i the reservoir has
energy E0 − i so that the total energy is E = i + (E0 − i ) = E0 . The
1
We use the term state here in the quantum mechanical meaning. A state i corresponds
to a quantum mechanical state, |ii or Ψi , and has an associated energy i . Notice that
several states may have the same energy.
7.1 Boltzmann statistics in the Einstein crystal 199

Fig. 7.3 Illustration of a

system S in thermal contact S TR R
with a large reservoir R. ΔE
The composite system
consisting of S and R is
isolated. TS

ES ER = E0 - ES
VS VR
NS NR

probability for the system to be in state i is given by the multiplicity of

the whole system, consisting of both system S and reservoir R:
ΩS ΩR
P (i) = P , (7.6)
i ΩS ΩR

where the sum is for normalization of the probability (to ensure the sum
of probabilities of all possible states is 1 — the system must be in one of
the possible states).
Probability of state i. Since the system S is in state i the multiplicity
of this state is 1: We have specified the state so it is only one way it
can be in this state. (Note that state here is not the same as the energy,
because several states can have the same energy and still be separate
states. It is therefore important to remember that we here consider states
and not only the energy.)
What is the multiplicity of the reservoir? For a given microstate i of the
system S, the reservoir R can be in many microstates. The multiplicity of
the macrostate where system R has energy E0 − i is Ω(E0 − i ), where
E0 is the total energy of the system. The probability of the reservoir is
therefore
ΩS ΩR
P (i) = P = C (1 ΩR ) , (7.7)
i ΩS ΩR
To simplify the algebra we work with the logarithm of the probability:

ln P (i) = ln C + ln ΩR (E0 − i ) . (7.8)

We can expand ln ΩR around E0 since i  E0 (the heat bath is large):

200 7 The canonical system

∂ ln ΩR (E)
 
ln ΩR (E0 − i ) = ln ΩR (E0 ) + (−i ) + O(2i ) , (7.9)
∂E V,N

Where we can now use that S = k ln Ω for the reservoir:

∂ ln ΩR (E) (1/k)∂SR
   
(−i ) = − i . (7.10)
∂E V,N ∂E V,N

Now, we use that the temperature TR of the reservoir is defined as

1/TR = (∂SR /∂)N,V when the system is in equilibrium. In equilibrium
the temperature in the reservoir and the temperature in the system will
be the same, and we can simply use T = TR = TS for the temperature
in both systems. The multiplicity is therefore
i
ln ΩR (E0 − i ) = ln ΩR (E0 ) − + O(2i ) , (7.11)
kT
The probability is therefore
ΩS ΩR −i /kT ΩR −i /kT 1
P (i) = P e =P e = e−i /kT . (7.12)
ΩS ΩR ΩR Z
Where Z is determined by the normalization rule for probabilities:
1 X −i /kT
P (i) = 1 = (7.13)
X
e .
i
Z i

and therefore
Z = Z(N, V, T ) = e−i /kT , (7.14)
X

The function Z is called the partition function.

7.1.3 Comparison with numerical model

This theory is only valid for large reservoirs. We can therefore compare
the theory with our numerical results for the Einstein crystal from above,
but we will only expect the match to be good when N and q are large.
We notice that the function we found to represent the numerical data
was
P (qA ) = Ce−qA /cT . (7.15)
The state in a system with a single harmonic oscillator is described by
the state i, so that the energy levels are i = ∆i, where we see that the
states can be enumerated by i or by qA . The theoretical results for the
7.2 The partition function and Boltzmann statistics 201

probabilities for Boltzmann statistics is therefore:

1 −(qA ∆)/(kT )
P (qA ) = e . (7.16)
Z
Where we see that the constant c = k/ and that the prefactor C is Z
(which also is a function of T , but it does not depend on qA ). The theory
therefore both explains the observed numerical results, and provides us
with a better understanding for the behavior of the system.

7.2 The partition function and Boltzmann statistics

The preceeding discussion is completely general and not limited to an

Einstein crystal. For a system S in thermal contact with a huge reservoir
R, we can assume that in equilibrium the temperature, TS , in system
S is given as the temperature, TR , of the reservoir, TS = TR = T . The
system S is therefore a canonical system with given N , V , T . The system
S may be in a set of possible states i with energies i , corresponding to
the quantum mechanical states of the system. For such as system, the
probability for the system to be in a state i is given by the Boltzman
factor:

Boltzman probabilities: The probability for a system S described

by N , V , T , to be in a state i with energy i is given as
1 −i /kT
P (i ) = e , (7.17)
Z
where the system is in equilibrium.

Partition function: The sum Z = Z(N, V, T ) is called the parti-

tion function. The sum is over all the states i of the system:

Z(N, V, T ) = e−i /kT . (7.18)

X

The symbol Z comes from the German “Zustandsumme” – sum of all

states. A common mistake is to assume that the sum is over all the
202 7 The canonical system

energies of a system, and not over all the states. However, energies may
be degenerated — there can be several states that have the same energy
— and in this case you need to include one term in the partition function
sum for each of the states in the system. Alternatively, you can introduce
the degeneracy g() of a system, the number of states of energy , and
then sum over all possible energies instead

Z(N, V, T ) = g() e−/kT . (7.19)

X

7.2.1 Average values and fluctuations

How can we use this new theory to connect the microscopic description of
the system given in the probabilities for the states with the macroscopic
description of the system given by thermodynamics? We can do this by
calculating averages of the thermodynamic quantities.
We call a system with constant N, V, E a microcanonocal system,
and a system with constant N, V, T – that is a system in contact with
a large heat reservoir of temperature T – a canonical system. In the
microcanonical system the total energy is given, and we can measure
the temperature T . The temperature T will fluctuate: If we measure
temperature in a molecular dynamics simulation, we will get a tempera-
ture that varies in time, T (t). However, we also found that for realistic
values of N – which typically are around 1020 – the values for T are very
sharp: Deviations beyond 10−10 of its average value are very unlikely.
For a canonical system, the temperature T is constant, but the energy
E will fluctuate. We can therefore find the average value (and later the
fluctuations) of the energy and use this to connect the microscopic theory
of Boltzmann statistics with the macroscopic theory of thermodynamics.
The average value (or expectation value) of a quantity Qi that depends
on the state i follows the definition of averages in Chap. 4.

The average of a quantity Qi , which depends on the state i of a

canonical system with given N , V , T , is:

Qi = P (i) Qi = (1/Z) Qi e−i /kT (7.20)

X X

i i

The average of the energy E is therefore

7.2 The partition function and Boltzmann statistics 203

Ē = hEi = i P (i ) , (7.21)

X

where the sum is over all states i of the system S. Since we have an
expression for P (i) we can simplify this expression to:
1 −i /kT
Ē = (7.22)
X
i e .
i
Z

Here, we will employ a common trick, which is to introduce β = 1/(kT ).

The equation then becomes
1 X −βi
Ē = i e . (7.23)
Z i

The trick is now to recognize that the sum is the derivative of a sum
which is simpler to calculate:
d X −βi dZ
i e−βi = − =− (7.24)
X
e .
i
dβ i dβ

This provides us with a coupling between the microscopic and the macro-
scopic descriptions of the system:

The average energy of a canonical system with states i with energies

i is
1 d d ln Z
Ē = − Z=− , (7.25)
Z dβ dβ
where Z = e−βi and β = 1/(kT ).
P
i

This expression allows us to calculate the energy E directly from the

partition function. Now we have the tools to connect the microscopic
and the macroscopic also in systems with constant N , V , and T . Let us
look at some example applications.

7.2.2 Example: Two-state system

Let us start with a very simple case: a two state system in contact with a
thermal reservoir with temperature T as illustrated in Fig. 7.4. We have
an atom/molecule in contact with a thermal bath with temperature T .
204 7 The canonical system

The system may be in two states with energies 0 and 1 . What is the
internal energy and the heat capacity of this system?

(a) (b)
S ΔE R

ε1 ε1
ε1

ε0 ε0 ε0

TS TR

State i=0 State i=1

Fig. 7.4 a Illustration of two-state system. b Two-state system in contact with a large
thermal reservoir with temperature TR .

Finding the partition function. To find the behavior of such a system,

our first step is always to find the partition function, Z. We may choose
our base energy level as we like, so let us choose 0 = 0 to simplify the
calculations.
Z= e−i kT , (7.26)
X

where the sum is over all the states. In this case, all the states corresponds
to all the energies.

Z = e−0/kT + e−/kT = 1 + e−/kT . (7.27)

Finding the average energy. The average energy is

1 X −i /kT 1  −0/kT  e−/kT

Ē = e i = 0e +  e−/kT =  . (7.28)
Z i Z 1 + e−/kT

Plotting the average energy. The energy as a function of temperature

is shown in Fig. 7.5. The results were plotted by the following program:
from pylab import *
x = linspace(0,3,1000)
E = exp(-1/x)/(1+exp(-1/x))
subplot(2,1,1)
plot(x,E)
xlabel(’kT/\epsilon’), ylabel(’U/\epsilon’)
subplot(2,1,2)
CV = diff(E)/diff(x)
xmid = 0.5*(x[1:])+x[0:-1])
plot(xmi,CV)
7.2 The partition function and Boltzmann statistics 205

xlabel(’kT/\epsilon’), ylabel(’C_V/\epsilon’)
show()

Fig. 7.5 Two-state system. 1

a Plot of energy as a
function of temperature. b

U /0
0.5
Plot of heat capacity as a
function of temperature.
0
0 0.5 1 1.5 2 2.5 3
k T /0
−3
x 10
1.5

1
C V /0
0.5

0
0 0.5 1 1.5 2 2.5 3
k T /0

Limiting behavior. We check the two limits analytically:

• What happens when T → ∞? In this case, the term e−/kT → 0. The
probabilities for both state are therefore equal and equal to 1/2, and
and the average energy is /2.
• At low temperatures, the only possible state is the state with the
lowest energy, P (1) = 1 and P (2) = 0, and the average energy is also
zero.

Heat capacity. Finally, we can find the heat capacity, CV , at constant

V from
∂E
 
CV = , (7.29)
∂T N,V
either by numerical derivation of the E(T ) curve, of by analytical deriva-
tion of
∂ 1
CV =  (7.30)
∂T e /kT +1
1 1
 
=  (−1)
2 e
/kT
− 2 (7.31)
e/kT + 1 kT
2
 e/kT

=k
2 . (7.32)
kT e/kT + 1
206 7 The canonical system

7.2.3 Example: Fluctuations and heat capacity

There is a general result, which is interesting from a computational
perspective, which allows us to calculate the heat capacity from the
fluctuations in the energy.
Average energy. We found above in (7.25) that the average of the energy
is found as:
∂
Ē = − ln Z , (7.33)
∂β
where Z = = e−i /kT and β = 1/(kT ).
P P
i i

Heat capacity. Similarly, we can express the heat capacity as the deriva-
tive of Ē with respect to β:

1 1 ∂2Z 1
! " 2 #
∂ Ē ∂ Ē ∂β ∂Z

CV = = = 2 2
− 2 , (7.34)
∂T ∂β ∂T kT Z ∂β Z ∂β

where we used that

∂β ∂ 1 1
= =− 2 . (7.35)
∂T ∂T kT kT
Fluctuations in energy. Now, we see that

1 ∂2Z
E¯2 = 2i e−βi = (7.36)
X
,
i
Z ∂β 2

Heat capacity and energy fluctuations. This gives us the relation:

1  ¯2 
CV = E − Ē 2 . (7.37)
kT
This means that if we can measure the fluctuations in E we can use this
to find the heat capacity. Nice! This is particularly useful if we study a
molecular/atomic system!

7.2.4 Sharpness of the canonical system

Based on this result from the fluctuations, we can also determine the
sharpness of the thermodynamic averages for a canonical system.
From (7.37) we know that the variation in the energy of a system,
∆E, is proportional to the square root of the heat capacity:
7.2 The partition function and Boltzmann statistics 207
r  2
∆E = E¯2 − Ē = (7.38)
p
CV kT ,

Now, we expect both the energy and the heat capacity (which is the
derivative of energy with temperature) to be extensive properties —
properties that are proportional to the number of particles N . This means
that the relative fluctuation in E is given as
√
∆E N
∝ ∝ (N )−1/2 . (7.39)
Ē N
In the limit of realistic N , typically on the order of 1020 – we see
that the relative error is 10−10 , which is generally not measurable. The
thermodynamic properties are therefore sharp.

7.2.5 Example: Einstein crystal in the canonical system

We can use the new formalism to address the behavior of an Einstein
crystal in contact with a large, thermal reservoir, for example, a vibrating
diatom in thermal contact with a large gas as illustrated in Fig. 7.6. First,
we find the probability for a single oscillator to be in a given state, and
then we find the average energy of a single oscillator.

Fig. 7.6 Illustration of a

single harmonic oscillator (a) (b)
S R
in contact with a large iΔε
ΔE
reservoir. a Illustration of TS TR
the states of the diatomic 3Δε
harmonic oscillator. b
Illustration of the oscillator 2Δε
in contact with a large Δε
reservoir.
States

States of the Einstein crystal. The states of an harmonic oscillator are

i = 0 + i ∆ , (7.40)

where 0 = ~ω/2 and ∆ = ~ω. We choose the energy level so that 0 = 0,

which is simply a small shift in energy, so that the energy levels are

i = i ∆ , i = 0, 1, 2, . . . . (7.41)
208 7 The canonical system

Partition function for the Einstein crystal. The partition function is

then given as the sum over all states:
∞
Z= e−i /kT . (7.42)
X

i=0

We can simplify the notation by introducing the notation β = 1/(kT ).

The partition function is then:
∞
Z= e−β i = e−β ∆ i , (7.43)
X X

i n=0

which is a geometric series, with sum

1
Z= . (7.44)
1 − e−β ∆
Energy of the Einstein crystal. We can find the average energy either
directly from the sum or from the expression in (7.25):
∂ ∂  
Ē = − ln Z = ln 1 − e−β ∆ (7.45)
∂β ∂β
1 1
= e−β ∆ ∆ = ∆ β ∆ . (7.46)
1−e −β ∆ e −1
We insert β = kT , getting:
1
Ē = ∆ . (7.47)
e∆/kT −1
we can introduce the characteristic temperature θE = ∆/k — the
Einstein temperature for the oscillator, which is a constant for a given
∆ — getting:
1
Ē = ∆ θ /T . (7.48)
e E −1
Plot of the energy and the heat capacity of an Einstein crystal. It
is illustrative to plot this expression as a function of T /θE using:
from pylab import *
x = linspace(0,1.5,1000)
E = 1.0/(exp(1.0/x)-1.0)
subplot(2,1,1)
plot(x,E)
xlabel(’T/\theta_E’);
ylabel(’E/\epsilon’);
7.2 The partition function and Boltzmann statistics 209

subplot(2,1,2)
CV = diff(E)/diff(x)
xx = 0.5*(x[1:]+x[0:-1])
plot(xx,CV)
xlabel(’T/\theta_E’), ylabel(’C_V/\epsilon’)
show()

Fig. 7.7 A single Einstein 1.5

crystal in contact with a
reservoir with temperature 1

U /0
T . a Plot of the energy E
0.5
as as function of T . b Plot
of the heat capacity as a 0
function of temperature. 0 0.5 1 1.5
T /θ E
1
C V /0

0.5

0
0 0.5 1 1.5
T /θ E

(I leave it for you to find an analytic expression for the heat capacity.)
Harmonic oscillators in diatomic gases. It is interesting to notice the
behavior of the heat capacity for a single harmonic oscillator. This would
be the contribution to the heat capacity for the vibration of a diatomic
molecule. Notice that when T  TE there is no contribution from the
oscillations, whereas when T  TE (or about at TE ), there is a contibu-
tion from the oscillations. This allows us to sketch the expected effects of
oscillations on the behavior of a diatomic ideal gas — where the behav-
ior goes from 3/2N kT to 5/2N kT at around TE for the corresponding
oscillation. A nice prediction.

7.2.6 Example: Single spin system

We can now apply these methods to address the behavior of a param-
agnetic system. For a single dipole in a magnetic field, there are two
possible states, one state with spin up and energy −µB and one state
with spin down and energy µB.
Partition function for a single spin. The partition function for a single
spin is therefore:
210 7 The canonical system

Z= e−β i = e−β µB + eβ µB = 2 cosh(β µB) . (7.49)

X

That was easy!

Energy for a single spin. We can find the average energy, Ē, from:
∂ 1
Ē = − ln Z = − (sinh(β µB) µB) = −µB tanh(β µB) ,
∂β cosh(β µB)
(7.50)
and, as you will see further down, to get the result for N independent
spins we simply multiply by N .
This was very simple compared with the long calculation you had to
do in the microcanonical system!

7.2.7 Example: N -spin system

How can we find the behavior of a system of N non-interacting spins?
If there is no interaction between the spins, then the energy of spin j
depends only on the direction of spin j relative to the external magnetic
field. If we write the direction of spin j as Sj = ±1, then the energy
associated with spin j is (Sj ) = −Sj µB. The total energy of the system
is therefore

(S1 , S2 , . . . , SN ) = −S1 µB − S2 µB − . . . − SN µB =
X
−Sj µB .
j
(7.51)
Partition function. The possible states of the system are then possible
values for (S1 , S2 , . . . , SN ), to find the partition function we must there-
fore sum over all possible values for S1 , all possible values for S2 , and so
on:
7.2 The partition function and Boltzmann statistics 211

ZN = e−(S1 ,S2 ,...,SN )/kT (7.52)

X

(S1 ,S2 ,...,SN )

= e−(S1 )/kT −(S2 )/kT −...−(SN )/kT (7.53)

X X X
...
S1 =±1 S2 =±1 SN =±1

= e−(S1 )/kT e−(S2 )/kT . . . e−(SN )/kT (7.54)

X X X
...
S1 =±1 S2 =±1 SN =±1

= −(S1 )/kT
e−(S2 )/kT . . . e−(SN )/kT (7.55)
X X X
e
S1 =±1 S2 =±1 SN =±1
| {z }| {z } | {z }
=Z1 =Z1 =Z1

= Z1 Z1 . . . Z1 (7.56)
= Z1N . (7.57)

Where we have introduced Z1 as the partition function for one spin, which
we found in (7.49), and ZN as the partition function for N non-interacting
spins.
Energy of N -spin system. This means that the energy, E, which de-
pends on ln Z, simply will have a factor N in front:
∂ ∂ ∂
ĒN = − ln ZN = − ln Z1N == −N ln Z1 = N Ē1 . (7.58)
∂β ∂β ∂β
However, the use of this argument depends critically on the assumption
that the spins do not interact. What happens if they do? We will address
this in the next example.

7.2.8 Example: Ising spin system

Ideal paramagnet. We have made many approximations in our model
for a paramagnet. A paramagnet consists of many magnetic dipoles.
These dipols may interact with an external magnetic field. However, each
such dipole also sets up a magnetic field, which all the other dipoles
may interact with. So far we have addressed the behavior of an ideal
paramagnet, a paramagnet where there are no interactions among the
spins. They only interact with the external field. But this is only an
approximation to a real system. How can we make such a model more
realistic?
Realistic paramagnet. To make a more realistic model, we need to
include the interaction between spins. Every spin sets up a magnetic
212 7 The canonical system

field related to the spin direction, Bi = cSi , where Si is the direction of

spin i. However, this field is not constant in space, but will decay with
the distance from spin i. The interaction between spin i and spin j will
therefore also depend on the distance rij between the spins.
As a first approximation, we will assume that the magnetic field set
up by a spin decays rapidly in space, so that we only need to include the
interactions between the nearest neighbors. If the spins are organized
in a regular lattice, such as along a line in one dimension, in a square
lattice in two dimensions or in a cubic lattice in three dimensions, then
a spin Si , will set up a magnetic field Bi , which will interact with the
neighboring spins Sj , giving rise to energies ij = −JSi Sj , where J is a
coupling constant the includes the effect of the magnetic field Bi set up
by spin i, the decay of the field with distance, and the coupling between
the field Bi and spin Sj .
Energy of a state. The total energy of the system will therefore depend
on the state (S1 , S2 , . . . , SN ) of the spin system, and the energy as a
function of the state will also include terms of the form Si Sj for all the
neighboring spins (i, j).

(S1 , S2 , . . . , SN ) = −J Si Sj + (−µB) Si . (7.59)

X X

(i,j) i

where the sum (i, j) is over all i and for each i the sum is also over all
the neighbors, j, of i. The neighbors of a spin i is illustrated in Fig. 7.8.
This model is called the Ising model for a paramagnet. Notice that the
energy now contains cross-terms S1 S2 , so we cannot use the exact same
technique as before to find the partition function.

Fig. 7.8 Neighbors in the

Ising model in one- and
two-dimensions. a The Si,j+1
neighbors for a spin Si in
one dimension are Si−1 and
Si-1,j Si,j Si+1,j
Si+1 . b The neighbors for a
spin Si,j in two dimensions Si-1 Si Si+1
are Si−1,j , Si+1,j , Si,j−1 , Si,j-1
and Si,j+1 as illustrated.

Let us look at the energy of the Ising system. We see that the energy
contribution from the spin-spin interaction is low (−J) if the two spins
are pointing in the same direction, and it is high (+J) if the two spins are
pointing in opposite directions. This spin interaction will therefore lead
7.2 The partition function and Boltzmann statistics 213

to a situation where all the spins point in the same direction — given
by the extenal field — in the lowest energy configuration. However, as
temperature increases, fluctuations will arise, and it is not so clear what
happens.
How can we find the partition function for this system? It turns out it
is possible to find the partition function for a one-dimensional system.
Solution in one dimension. For a one-dimensional Ising model with no
external magnetic field, the energy of a state (S1 , S2 , . . . , SN ) is

(S1 , S2 , . . . , SN ) = −J (S1 S2 + S2 S3 + S3 S4 + . . . + SN −1 SN ) .
(7.60)
The partition function can therefore be written as

Z= (7.61)
X X X
... eJβS1 S2 eJβS2 S3 . . . eJβSN −1 SN .
S1 =±1 S2 =±1 SN =±1

There is an old trick to calculate this particular sum. We notice that the
sum over SN is the same if SN −1 = 1 or is SN −1 = −1:

eJβSN −1 SN = eJβ + e−Jβ = 2 cosh Jβ , (7.62)

X

SN =±1

Now, when this sum is calculated, we can repeat this argument with the
sum over SN −1 , and so on, until we reach S2 . The final sum over S1 will
then only yield a factor 2. The partition function is therefore

Z = 2N (cosh Jβ)N −1 ' (2 cosh Jβ)N , (7.63)

where the approximation is good when N is large. This is exactly the

same as for the system of N independent spins, but we have replaced
µB with J. From the partition function we can find the average energy:
∂
Ē = − ln Z = −N J tanh Jβ . (7.64)
∂β
We see that when T → 0, E → −N J, that is, all the spins are parallel.
When T → ∞, E → 0, which means that the spins become random, so
that on average the sum of the interactions between the spins are zero.
ams 3: Comment on transition to ordered state? Not a phase transi-
tion in this case.
Approximate solutions in higher dimensions. It is also possible to find
the partition function for a two-dimensional system. Onsager found this
solution already in the 1940ies, and his solution was later confirmed by
214 7 The canonical system

Yang [2]. However, the mathematical methods needed are non-trivial and
will not be introduced here. Instead, we will approximate the partition
function and the equilibrium behavior of the system using approximate
solutions. We will address methods to find approximate, statistical solu-
tions in the canonical system in Sect. 7.10, and we will also address the
Ising model in detail there.

7.2.9 Degenerate states

A common mistake is to sum over the energies in a system and not
over the states when you calculate the partition function. What is the
difference? Several states may have the same energy!
We can group microstates with the same energies together. Each state
with an energy s , will contribute a term exp(−βs ) to the sum. Instead
of summing over all the states i, we can sum over all the energies, but
then we need to include how many states there are for a particular energy
s , the degeneracy g(s ):

Z= e−βi = g(s )e−βs . (7.65)

X X

states i energy levels s

We will use both methods, choosing the method that is most adapted to
the problem.

7.2.10 Example: System with degenerate states

For example, we could have studied a system where the number of states
with the same energy increases with the energy. Let us introduce a system
with the following energy states:

i = (1, 2, 2, 3, 3, 3, 4, 4, 4, 4) (7.66)

In this case the multiplicity of energy  is 1, of 2 is 2, of 3 is 3 and so

on.
The partition function is:

Z= e−βi = g(s )e−βs , (7.67)

X X

i s

where g(1) = 1, g(2) = 2, g(3) = 3 and g(4) = 4. The sum is therefore:

7.3 Thermodynamics in the canonical ensemble 215

Z = 1 e−β + 2 e−2β + 3 e−3β + 4 e−4β . (7.68)

We find the average energy in the same way:

Ē = i e−βi = g(s) s e−βs , (7.69)

X X

i s

where the sum over i is over all the states, and the sum over s is over all
the energies.
The average energy is therefore:

1 e−β + 2 × 2e−2β + 3 × 3e−3β + 4 × 4e−4β

Ē =  . (7.70)
Z

7.3 Thermodynamics in the canonical ensemble

In the microcanonocal system, all microstates have the same probability,

P (i) = 1/Ω, whereas in the canonical system, the probability of a
microstate is weighted with its energy, i , according to the Boltzmann
factor P (i) = (1/Z) e−i /kT . In the microcanonical system, the coupling
between the microscopic models and the macroscopic thermodynamics,
occurs through S(E, V, N ) = k ln Ω(E, V, N ). Can we find a similar
simple relation between the microscopic and the macroscopic for the
canonical system where (T, V, N ) are constants? In addition, we have
discovered the second law of thermodynamics, which is valid for an
isolated system. Can we find a similar law for a canonical system? Here,
we will address these questions, starting from the thermodynamic identity
and its extension to the canonical system.
We have already found a connection between the micro-scopic and
the macro-scopic representation of a canonical system: We found that
we can calculate the (average) energy Ē in the system directly from the
partition function:
∂
Ē = i P (i) = − ln Z , (7.71)
X

i
∂β

where β = 1/(kT ). We will now use this relation as a basis for the
coupling between the microscopic and the macroscopic.
216 7 The canonical system

7.3.1 Helmholtz free energy

For the microcanonical ensemble, entropy is a function of E, V, N , S =
S(E, V, N ), and the thermodynamic identity is

T dS = dE + pdV − µdN , (7.72)

Thermodynamic identity for a (S, V, N ) system. We can change from

S = S(E, V, N ) to the energy as a function of S,V ,and N :

E = E(S, V, N ) , (7.73)

and we can find the corresponding differential relations by rearranging

the thermodynamic identity:

dE = T dS − pdV + µdN . (7.74)

Indeed, from this rearrangement we see that when E is a function of

S, V, N , we can find the T , p and µ from various derivatives of E =
E(S, V, N ).
Thermodynamic identity for a (T, V, N ) system. Now, to describe the
canonical system, we would like to have a similar differential to dE, but
with the differentials dT , dV , and dN . We would like to have a relation
like:
dA = ()dT + ()dV + ()dN , (7.75)
where we need to find the unknown functions/variables shown as () in
the differential. Also, we need to figure out how we can construct A to
have these properties. There is a formal way to do this, using Legendre
transforms, which is robust (meaning that it always works) and we will
introduce it in Sect. 7.6. For now, we will motivate the Legendre transform
by observing that the differential for E almost got us there, we only need
to replace the dS term with some dT term. We need to subtract the T dS
term and then get a new XdT term with the T dependence. We see that
if we subtract d(T S) on both sides of the thermodynamic identity, we
get

dE − d(T S) = T dS − d(T S) − pdV + µdN

= T dS − T dS − SdT − pdV + µdN (7.76)
= −SdT − pdV + µdN ,
7.3 Thermodynamics in the canonical ensemble 217

which has exactly the right form! This is the quantity we were looking
for. It is an energy, since it has units energy, and it is a function of T , V ,
and N . We call this quantity F , Helmholtz free energy:

Helmholtz free energy: F (T, V, N ) = E − T S

The differential for F , Helmholtz free energy, is:

dF = −SdT − pdV + µdN . (7.77)

Application of the thermodynamic identity for the canonical system.

This identity allows us to find the entropy, S, the pressure p, and the
chemical potential µ if we have an expression for Helmholtz free energy:
∂F ∂F ∂F
     
S=− , p=− , µ= . (7.78)
∂T V,N ∂V T,N ∂N T,V

This means that if we know Helmholtz free energy, we can calculate

the remaining thermodynamical properties of the system. However, to
do this we need an expression for F (V, T, N ) either from experiments,
simulations, or theory. We have already found how we can connect the
energy E to the microscopic theory through the use of the partition
function Z(T, V, N ). How can we relate F to the partition function and
thus complete the coupling between the microscopic and the macroscopic
also for the canonical system?

7.3.2 A microscopic expression for Helmholtz free energy

We have already found that the energy E can be related to the microscopic
description of the system through the partition function:
∂ ln Z(T, V, N )
Ē = − , (7.79)
∂β
We can therefore find the energy if we know the partition function.
However, for the canonical system, we would also like to be able to
calculate quantities such as the entropy, the pressure or the chemical
potential. We would therefore like to have an expression for Helmholtz
218 7 The canonical system

free energy, because, as we saw above, we can then find all quantities
of interest from partial derivatives of F . We know that Helmholtz free
energy, F , is
F = E − TS , (7.80)
We already know E, but we do not know S — yet. However, we know
that dF = −SdT − pdV + µdN and therefore we know that
∂F
 
S= , (7.81)
∂T V,N

which we can insert back into (7.80), giving

∂F
 
F =E+T . (7.82)
∂T V,N

We recognize this as a differential equation for F .

∂ ln Z(V, T, N ) ∂F
 
F =− +T . (7.83)
∂β ∂T V,N

We can find the solutions to this differential equation, by starting from a

good guess — which of course is correct. A solution to this differential
equation is
F = −kT ln Z(V, T, N ) . (7.84)
We can demonstate that this is a solution by substitution. First, we find
the derivative of F with respect to T :
∂F ∂
 
= − kT ln Z
∂T V ∂T
∂ ln Z (7.85)
= −k ln Z − kT
∂T
E
= −k ln Z − kT 2 .
kT
Then we insert this into the differential equation for F in (7.83) and find
that
E + T (−k ln Z − (E/T )) = −k ln Z , (7.86)
which proves that F = −kT ln Z is one possible solution to the differential
equation in (7.83). However, there may in addition be a constant term
F = −kT ln Z + aT . To address the aT term, we need to invoke the
third law of thermodynamics, which states that the entropy of a system
7.3 Thermodynamics in the canonical ensemble 219

approaches a small number or zero as the temperature approaches zero.

This is because as the temperature approaches zero, the system will be
frozen into the lowest possible energy configurations, and the entropy will
only be the possible degeration of the lowest energy, which will typically
be a small number. Since the entropy is the derivative of F with respect
to T , we see that the entropy only will approach zero or a small number
if a is zero or a small number, that is, is a ' 0.
We have therefore proven the fundamental relationship between ther-
modynamics and the microscopic models for the canonical system:

Relation between Helmholtz free energy and the microscopic de-

scription of a canonical system:

F (T, V, N ) = −kT ln Z(T, V, N ) , (7.87)

where Z(T, V, N ) = e−i /kT is the partition function for the

P
i
system.

I find this equation beautiful. It is simple. It connects the microscopic

and the macroscopic worlds. It allows all types of calculations of S,
p and µ from the partition function. We can now apply this to find
all thermodynamic properties of systems described in the canonical
ensemble.

7.3.3 Helmholz free energy decreases in approach to

equilbrium
For the micro-canonical ensemble we have seen how the system develops
towards equilibrium and what characterizes the equilibrium state: The
system evolves towards a the most likely macrostate – that is, towards
the macrostate with most microstates – equivalently the system evolves
towards maximum entropy:

∆S ≥ 0 . (7.88)

This condition also characterizes the equilibrium state: The entropy is a

maximum for the equilbrium state.
For example, for the Einstein crystal, divided the system into two
parts, A and B. We saw that the total entropy of the system S = SA + SB
220 7 The canonical system

increases with time, and that the total entropy is maximal at equilibrium.
This is indeed what characterizes the equilibrium state in the system.
Do we have similar principles for the canonical system? Does the
entropy increase for a canonical system? Does the energy increase? Or is
there some other principle that we do not yet know? We can study this in
the Einstein crystal. We make a system consisting of two parts, a small
part A, which is the system, and a large part B, which is the reservoir. We
can make a system with NA = 10 and NB = 990. This should ensure that
system B is so large that the temperature in this system does not change
– it is constant and equal to the reservoir temperature TB = TR . The
total energy is q = qA + qB . We can simulate the dynamics of this system
using the monte-carlo method we developed previously. This would allow
us to simulate the time dynamics of the system as it develops through a
sequence of microstates. This is done by the following program:
# MC for a two part Einstein crystal
from pylab import *
NA = 10
NB = 990
qA = 300
qB = 9700
q = qA + qB # Total energy
N = NA + NB
nstep = 1000000
nbetween = 1000
state = zeros(N,float)
# Generate initial, random state
placeA = randint(0,NA,qA)
for ip in range(len(placeA)):
i = placeA[ip]
state[i] = state[i] + 1
placeB = randint(0,NB,qB)+NA
for ip in range(len(placeB)):
i = placeB[ip]
state[i] = state[i] +
# Simulate state development
EA = zeros(nstep,float)
EB = zeros(nstep,float)
TBSA = zeros(nstep,float)
TB = zeros(nstep,float)
for istep in range(nstep):
i1 = randiint(0,N) # Select oscillator at random
if (state[i1]>0): # Check if it has energy
i2 = randint(0,N) # Then find other oscillator
state[i2] = state[i2] + 1
state[i1] = state[i1] - 1
# Calculate T_B S_A
EA[istep] = sum(state[:NA-1])
EB[istep] = sum(state[NA:])
7.3 Thermodynamics in the canonical ensemble 221

qA = EA[istep]
qB = EB[istep]
omegaA = comb(NA+qA-1,qA)
TB[istep] = qB/NB
TBSA[istep] = TB[istep]*log(omegaA)
if (mod(istep,nbetween)==0):
subplot(4,1,1) # State
bar((1:NA),state(1:NA),’b’), hold(’on’)
bar((NA+1:N),state(NA+1:end),’r’), hold(’off’)
a = axis(); a(2) = N; axis(a);
xlabel(’i’); ylabel(’n_i’);
subplot(4,1,2) # Avg energy in each system
plot((1:istep),EA(1:istep)/NA,’-r’,...
(1:istep),EB(1:istep)/NB,’-b’);
drawnow, xlabel(’t’); ylabel(’q_A/N_A , q_B/N_B’)
subplot(4,1,3) # Plot T_B
plot((1:istep),EB(1:istep)/NB,’-b’)
xlabel(’t’); ylabel(’T_B’); drawnow
subplot(4,1,4)
plot((1:istep),EA(1:istep),’-r’,...
(1:istep),TBSA(1:istep),’-b’,...
(1:istep),EA(1:istep)-TBSA(1:istep),’-k’)
xlabel(’t’); ylabel(’E’);
legend(’E_A’,’T_B*S_A’,’E_A-T_B*S_A’);
drawnow

We need to decide on the initial distribution of energy in the system.

We try to alternatives: First, we try the case where system A has a large
amount of the energy than an even distribution, we choose qA = 300,
and qB = 9700, giving q = 10000. In this case, the average energy
per oscillator in system A is 30 and the average energy per oscillator
in system B is 9.8. We measure the entropy, SA (t) and energy qA (t) of
system A, SA (t), as a function of time t. Fig. 7.3.3 shows the development
of EA (t) and EB (t). We see that the energy in system A decreases with
time and reaches an approximately constant level – corresponding to the
equilibrium state of the system. We see that in the equilibrium state
– after a long time – the energy is evenly distributed throughout the
system, and the average energy per oscillator in system A and system B,
EA /NA and EB /NB become the same value, close to the average of the
total energy per oscillator, q/N , in the system. This is not surprising. We
expect the average energy per oscillator to be related to the temperature,
and we know that in equilibrium the temperatures in the two systems
will be the same. We also see that system B is indeed large. In the plot
that shows both EA /NA and EB /NB we see that the change in EB /NB
is very small compared with the change in system A. This is what we
mean when we call a reservoir large – it is so large that the temperature
does not change significantly. There are, however, changes in EB /NB as
222 7 The canonical system

well, as evident from the plot of EB /NB alone, but these fluctuations are
small.
fFIGURE:[fig-partfunc/einsteincanonicalmc00, height=400 width=600
frac=1.0] Time development of the energies in two systems A with
NA = 10 and B with NB = 990 in thermal contact.
The energy in system A clearly decreases with time in this system.
What about the entropy? We can measure the entropy in system A
through the formula SA = k ln ΩA (qA , NA ), which we have developed
previously. Now, qA will change with time, and therefore as will SA .
This measurement was implemented in the program above. Fig. 7.9
shows both the energy and the entropy for system A. We see that the
entropy in system A decreases in time an approaches an approximately
stationary value. We expect the fluctuations to become smaller for larger
systems and the entropy to become sharp. These observations suggest
that in equilibrium – which is what we reach after a long time – the
system has reached a minimum in entropy and a minimum in energy.
Hmmm. That was surprising – the entropy is decreasing in this system.
(Convince yourself that this is not unphysical and that the second law of
thermodynamics still is valid).
Let us try the same simulation, but from a different starting point –
the case where system A initially has less energy than the average energy,
qA = 0 and qB = 10000. We expect the same equilibrium state in this
system as in the previous since the number of oscillators is the same and
the total energy in the system is the same. How does it develop in time?
Fig. 7.9 shows that the entropy, SA (t), and energy, qA (t), of system A
in this case both increase, and then reach a stationary value which is a
maximum of both entropy and energy? Hmmm. Now it is not easy to see
what kind of rule would determine the behavior of the system: Should
the entropy be a maximum or a minimum? Should the entropy increase
or decrease?
It is time to revisit the theory and develop a better theoretical under-
standing of this process. We know that the whole system, system A and
system B, is thermally isolated and therefore described as a microcanoni-
cal system with E, V , and N constant. For this system, the energy is
constant EA + EB = E and the total entropy must be increasing:

∆S = ∆SA + ∆SB ≥ 0 . (7.89)

The change in entropy in system A is indeed what we observed in the

simulations – it could be both positive or negative. This is what we want
7.3 Thermodynamics in the canonical ensemble 223

400 150

E A /∆ϵ

E A /∆ϵ
100
200
50

0 0
0 2 4 6 8 10 0 1 2 3 4 5
t x 10
5 t 5
x 10
40 40

35
S A /k

S A /k
20
30

25 0
0 2 4 6 8 10 0 1 2 3 4 5
t x 10
5 t 5
x 10

( E A − T B S A ) //∆ϵ
( E A − T B S A ) /∆ϵ

−50 0

−100
−100
−150

−200 −200
0 2 4 6 8 10 0 1 2 3 4 5
t x 10
5 t 5
x 10
Fig. 7.9 Time development of the energies and entropies for an Einstein system with
NA = 10 and NB = 990 for two different initial distributions of energy. In the left figures
the system start with qA = 300, qB = 9700 and in the right figures the system start with
qA = 0 and qB = 10000.

to develop a theory for. What about the development of system B? Since

system B is very large, we will now assume that system B is very close to
equilibrium – so close that we can assume that system B is in equilibrium.
In this case, we know that we can describe the behavior of system B
with the thermodynamic identity:

T dS = dE + pdV − µdN , (7.90)

where dV and dN is zero for system B – there is no change in volume

or the number of oscillators. We therefore get TB ∆SB = QB = ∆EB ,
where the change in energy in system B comes from the thermal energy
QB transferred from system A to system B, which again corresponds
to ∆EB = −∆EA , and therefore we have ∆SB = −∆EA /TB , which we
insert back into (7.89), getting:
224 7 The canonical system

∆EA
∆S = ∆SA − ≥0. (7.91)
TB
We rearrange, getting

TB ∆SA − ∆EA ≥ 0 , (7.92)

where we can now assume that TB does not change – it is (approximately)

constant during this process, and we can write this as the change in
(EA − TB SA ):
∆(TB SA − EA ) ≥ 0 , (7.93)
or
− ∆(EA − TB SA ) ≥ 0 , (7.94)
giving, finally:
∆(EA − TB SA ) ≤ 0 . (7.95)
We have now found a criterion for how the system should develop in time,
and a criterion for the equilibrium. In equilibrium, we expect EA − TB SA
to be minimal and we expect EA − TB SA to decrease with time and
approach a minimal value.
In equilibrium, TA = TB , and we therefore know that in equilibrium

∆(EA − TA SA ) ≥ 0 , (7.96)

where we recognize this as Helmholtz free energy, FA = EA − TA SA for

system A. We have therefore found a new principle, valid for a canonical
system with T, V, N constant: For this system, Helmholtz free energy
will always decrease ∆F ≤ 0, and the equilibrium state is characterize
as a minimum for Helmholtz free energy, F .
First, let us check if this indeed is the case in the simulations we
performed above. We have already found EA and SA as part of the
simulations. How can we find TB from the simulations? The simplest
approach would be to notice that system B is large, so that the high
temperature and large system size approximation for the energy of an
Einstein crytal can be used, EN ' NB kTB , which we can use to find the
temperature through TN = EN /(NB k). We are now able to calculate
EA − TB SA , but we should now check that the units we use are ok. In the
program, we calculate qA , qB and ln Ω for SA . We know that EA = qA 
and EB = qB , where  is the characteristic energy (the energy level
spacing) for the harmonic oscillator in the Einstein crystal. We therefore
get:
7.3 Thermodynamics in the canonical ensemble 225

EA − TB SA = qA − (qB )/(NB k)k ln Ω(qA , NA )

(7.97)
=  (qA − (qB /NB ) ln Ω(qA , NA )) ,

which shows that we can use qA (qB /NB ) ln Ω to characterize FA in the

simulations. Fig. 7.9 shows FA (t) for the two simulated models. We see
that even if SA and EA are increasing for one system and decreasing in
the other system, Helmholtz free energy FA is decreasing in both cases,
and approaches a stationary value as time increases, corresponding to the
equilibrium value of FA . Indeed, the theory we have developed describes
the system we have studied, and provides us with additional insight:
Helmholtz free energy is minimal in equilibrium.
This means that Helmholtz free energy plays a similar role in canonocal
systems (T, V, N constant) as the entropy did for the microcanonical
system (E, V, N constant). We call the principle that the free energy is
minimal in equilibrium a minimum free energy principle, and we will see
later that such principles are very useful and will allow us to calculate
equilibrium states in for example fluid or solid mixtures, and for both
biological, geological and meteorological systems of interest.
Where does the word free energy come from? We see that if T and N
is constant, then the work that is performed by a canonical system is

(∆F )T = −p∆V = −∆W , (7.98)

so that the work ∆W done on the environment by the system at constant

temperature and N is equal to the loss in free energy for the system:
Hence the name free energy. (This is only true for reversible processes).

7.3.4 Helmholtz free energy is a minimum in equilibrium

We can also show that Helmholtz free energy is not only an extremum
value, but it is actually a minimum:
The total energy is E = ES + ER . The total entropy is

S = SR + SS
= SR (E − ES ) + SS (ES )
(7.99)
∂SR
 
' SR (E) − ES + SS (ES ) .
∂UR V,N

Where we also know that

226 7 The canonical system

∂SR 1
 
= , (7.100)
∂ER V,N T

and therefore we get

1
S = SR (E) −
FS , (7.101)
T
where FS = ES − T SS is the free energy of the system. Now SR (E) is a
constant. We also know that S = SR + SS is maximum with respect to
ES in equilibrium. There FS must be a minimum.

7.3.5 Maxwell relations

We can now find a new group of relationsm, thermodynamic relations,
based on the second derivative of Helmholtz free energy: If the energy is
continuous in its variables, we know that the order of derivatives for the
second derivative is not important, therefore:
! !
∂ ∂F ∂2F ∂2F ∂ ∂F
   
= = = .
∂T ∂V T,N V,N
∂V ∂T ∂T ∂V ∂V ∂T V,N T,N
(7.102)
We call this type of relation a Maxwell relation. We can use Maxwell
relations to find non-trivial relations between thermodynamic variables.
For example, we can now show that
! !
∂ ∂F ∂p ∂∂S ∂F
       
= = = .
∂T ∂V
T,N V,N V ∂T V,N T,N ∂V∂V T ∂T
(7.103)
We will find more such relations later as we introduce more variables.
(There are three possible permutations in this case, but the other two
involve the chemical potential, which we are not familiar with yet).

7.4 Composite systems

7.4.1 Aspects of motion

The energy of a realistic system has many contributions. For example, for
a diatomic gas, the energy of a single molecule consists of terms relating
to the translational motion of the molecule, the rotational motion of the
molecule, and the vibration of the molecule. To specify the state of the
7.4 Composite systems 227

molecule we therefore need to describe the state of each of these aspects

of the molecule, for example by listing the states i for translation, the
states j for rotation, and the states n for vibration: (i, j, n). Typically,
the contribution to the total energy from each of these aspects are
independent: The total energy is the sum of the energies from each of
the aspects of motion (or from each of the degrees of freedom, if you
prefer that term). The total energy is therefore

(i, j, n) = i + j + n , (7.104)

In order to find the partition function for the diatomic molecule, we

therefore need to sum over all the states of translation, rotation, and
vibration. If these contributions are independent, we need to sum over
all values of i, and for each value of i, we need to sum over all values of j,
and for each value of j we must sum over all values of n. We write this as

Z= e−(i +j +n )/kT = e−i /kT e−j /kT e−n /kT .
XXX XXX

i j n i j n
(7.105)
This can be further simplified by realizing that the i-terms do not vary
when we sum over j and n, and they can therefore be placed outside
those sums. Similarly, the j-terms do not vary when we sum over n, and
they can therefore be placed outside the sum over n, giving:

Z= e−i /kT e−j /kT e−n /kT = Ztrans Zrot Zvib . (7.106)
X X X

i j n
| {z }| {z }| {z }
Ztrans Zrot Zvib

The total partition function is therefore the product of the partition

functions for each of the aspects of motion — as long as the states of
the various aspects are independent.

Partition function for independent aspects of a single particle:

Z = Ztrans Zrot Zvib . (7.107)

This argument can be extended to any new degrees of motion we add

to such a system, for example, the possibility for the atom or molecule
of the gas to be in a ground state or an excited state. However, the same
technique can also be used to find the partition function for a system of
228 7 The canonical system

several particles, if the states of each of the particles are independent.

Let us see how.

7.4.2 System of two particles

Let us start by a system consisting of two different particles, each de-
scribed independently by the quantum states 1,i and 2,j . The partition
function is
Z= e−β1,i e−β2,j
XX

i j

= −β1,i
e−β2,j
X X
e
i j
(7.108)
| {z }| {z }
Z1 Z2

= Z1 Z2 .

For two non-interacting systems, with distinguishable particles, the par-

tion function is the product of the partition functions!
Now, what if the two particles are not distinguishable? In that case, we
have included too many elements in the partition function sum, because
we have included both the term for the state

e−β(2,j +1,i ) (7.109)

and the term for the state

e−β(2,i +1,j ) , (7.110)

but if the particles are indistinguishable, these two states are the same,
and should only be included once in the sum. We should therefore divide
the sum by 2:
1
Z = Z1 Z2 , (7.111)
2
This is almost correct. But the sum also includes terms where the state
of both particles are the same, but only one such term is included in the
sum:
e−β(2,i +1,i ) , (7.112)
and I have also divided this part of the sum by two. Since these terms
not have been double counted, we should not divide these parts of the
sum by two. However, the error we make by dividing the whole sum by
7.4 Composite systems 229

two will be small as long as there are few of such states — and there are
very few of these states compared with the other states. We therefore
use the approximation:
1
Z ' Z1 Z2 (indistinguishable/identical) (7.113)
2
when the two particles are indistinguishable, and the exact result

Z = Z1 Z2 (distinguishable/different) (7.114)

when the particles are distinguishable.

7.4.3 System of many particles

This result can be generalized to N particles. If the particles are distin-
guishable, we find the partition function as the product of the partition
functions for each of the particles:

Z = Z1 Z2 . . . ZN . (7.115)

However, for indistinguishable particles, we have included the terms

where all the states of all the particles are different too many times. How
many times have we included such terms? The number of ways that
we can rearrange the N different states among the N indistinguishable
particles, which is N ! ways. We must therefore divide by N ! in order to
get the correct result for these terms. In addition, there will be terms
where some of the states are the same, and we will make a small error
by also dividing these terms by N !, but this error is small as long as
there are few of cases where the states are the same compared to the
cases where all the states are different. We therefore have the following
approximate result for indistinguishable particle and exact result for
distinguishable particles:

The partition function for a system of N particles with partition

functions, Z1 , for each particle is:

ZN = Z1N (distinguishable particles) (7.116)

Z1N
ZN = (indistinguishable particles) (7.117)
N!
230 7 The canonical system

This result now makes it easy to find

• the partition function for a system of N identical harmonic oscillators:
The oscillators are distinguishable (because they are localized in
various positions in space), and hence the partition function of N
oscillators is ZN = Z1N , where Z1 is the partition function for one
oscillator
• the partition function for a system of N identical magnetic spins in an
external magnetic field: The spins are distinguishable (because they
are localized in various positions in space), and hence the partition
function of N spins is ZN = Z1N , where Z1 is the partition function
for one spin
• the partition function for a system of N identical particles in an ideal
gas: The particles are indistinguishable (since they are identical and
only limited to being inside a box), and hence the partition function
of N particles s ZN = Z1N /N !, where Z1 is the partition function for
one particle
We notice that since Helmholtz free energy depends on the logarithm
of the partition function, we see that the Helmholtz free energy for N
distinguishable particles is

FN = −kT ln Z1N = −N kT ln Z1 = N F1 . (7.118)

This corresponds to an addition principle for the system: Helmholtz free

energy is the sum of the free energies for each of the particles, as long as
the particles are distinguishable.
However, for N indistinguishable particles, Helmholtz free energy is

FN ' −kT ln Z1N /N ! ' −N kT ln Z1 +N kT (ln N +1) = N F1 +N kT (ln N +1) .

(7.119)

7.4.4 Example: Ideal gas in the canonical ensemble

We can use these new results to find the partition function for an ideal
gas. First we find Z1 for one particle in a box, and then we find the
partition function, ZN for an ideal gas of N particles in a box.
Single particle in a box. For a single particle, we may also have to include
various aspects of motion, such as the rotational and vibrational degrees
of freedom for a diatom. However, let us postpone these differences, and
start by finding the partition function for translational motion of a single
7.4 Composite systems 231

particle in a box. The box has sides L and volume V . From quantum
mechanics we know that the possible states of a particle in a box are
described by the quantum numbers nx , ny , nz , representing the states of
motion in the x-, y-, and z-directions. The energy for the state (nx , ny , nz )
is
h2  2   
n = n + n2
+ n2
= a n 2
+ n2
+ n2
, (7.120)
8mL2 x y z x y z

where a = h2 /(8mL2 ). The partition function is given by the sum over

all the states nx , ny , nz :

e−a(nx +ny +nz ) =

2 2 2 2 2 2
Z1,trans = e−anx e−any e−anz ,
XXX XXX

nx ny nz nx ny nz
(7.121)
We recognize this as the sum over three identical degrees of freedom for
motion along the x-, y-, and z-direction, and we can separate these sums
just as we did for composite systems:
2 2 2 2 2 2
Z1,trans = e−anx e−any e−anz = e−anx e−any e−anz .
XXX X X X

nx ny nz nx ny nz
| {z }| {z } | {z }
Z1,x Z1,y Z1,z
(7.122)
where each of the sums are identical and equal to ξ
∞
2
Z1,trans = ξ 3 , ξ = e−anx . (7.123)
X

nx =0

For small values of a we can approximate the sum by the integral:

Z ∞ √
π
ξ' exp(−an ) , dn = √ .
2
(7.124)
0 2 a
The partition function is therefore given by:
!3/2
2mL2 2πmkT
3/2
π 3/2

Z1,trans ' = V . (7.125)
8 β~2 π 2 h2

To simplify this expression furthre, we introduce the quantum density

nQ (T ) defined as:

2πmkT
3/2 3/2
m
 
nQ = = . (7.126)
2π~2 β h2
232 7 The canonical system

The partition function for the translational motion of a single particle in

a three-dimensional box of size V = L3 is then:
nQ
Z1,trans = V nQ = , (7.127)
n
where n = 1/V is the density of particles.
The average particle energy is given by:

∂ ln Z1,trans 3
3/2
∂ m

Ē1,trans =− =− ln V = kT . (7.128)
∂β ∂β 2π~2 β 2
This is in correspondence with the equipartition principle, because the
particle has only one degree of freedom for each direction of motion, hence
three degrees of freedom in total. The energy per particle due to the
translational motion is therefore (3/2) kT according to the equipartition
principle.
If we want to include more aspects of the gas, this would appear as
additional terms in the partition function for the individual particle. For
example, for a diatomic gas we would then also include partition function
for rotation and vibration:

Z1 = Z1,trans Z1,rot Z1,vib = ξ 3 Z1,rot Z1,vib = . (7.129)

These terms will also be included in the energy of a single particle and
therefore also in the heat capacity of a single particle in the gas. Let us
now use this result to find the partition function for a gas of N particles.
Ideal gas of N particles in a box. The particles in an ideal gas are
indistinguishable, hence the partition function for the N -particle gas is

Z1N 1
ZN = = (Z1,trans Z1,rot Z1,vib ) . (7.130)
N! N!
From this we find either the energy directly, or Helmholtz free energi,
which we then use to find the entropy and the pressure of the gas. We
start with the energy:
∂ ln ZN ∂ ln Z1
ĒN = =N (7.131)
∂β ∂β
∂
=N (ln Z1,trans + ln Z1,vib + ln Z1,rot ) (7.132)
∂β
= N E1,trans + N E1,vib + N E1,rot . (7.133)
7.4 Composite systems 233

Where we have already in (7.128) found that E1,trans = (3/2)kT . The

energy of an ideal monoatomic gas (with only translational degrees of
freedom) is therefore
3
EN = N kT . (7.134)
2
Notice that we simply sum the various energy terms to find the total
energy. Similarly, we find Helmholtz free energy from the logarithm of
the partition function

FN = −kT ln ZN = −kT (N ln Z1 − N ln N − N ) (7.135)

= −N kT (ln Z1,trans + ln Z1,vib + ln Z1,rot − ln N + 1) . (7.136)

Let us find the behavior for a monatomic gas with only translational
degrees of freedom. We insert Z1,trans = nQ /n, getting

FN = −N kT (ln(V nQ ) − ln N + 1) = N kT (ln((V /N )nQ + 1) .

(7.137)
From this we can find both the pressure and the entropy of the ideal gas.
The pressure is found from the derivative of F with respect to V :
∂F N kT
 
p=− = ⇒ pV = N kT . (7.138)
∂V N,T V

Which we recognize as the equation of state for the ideal gas. Similarly,
the entropy is
∂F ∂ V
   
S=− =− N kT ln nQ (T ) + 1 (7.139)
∂T V,N ∂T N
V ∂
 
= N k ln nQ (T ) + 1 + N kT ln nQ (T )
N ∂T
(7.140)
V ∂
 
= N k ln nQ (T ) + 1 + N kT ln T 3/2 (7.141)
N ∂T
V 5
 
= N k ln nQ (T ) + (7.142)
N 2
V 2πmkT 3/2 5
  !
= N k ln + . (7.143)
N h2 2

These results are, of course, the same as we found in the microcanonical

system, but now expressed in terms of T , V , and N .
234 7 The canonical system

7.4.5 Example: Comparing microcanonical and canonical

analysis
We now have all the tools needed to address complex and realistic
problems using both the microcanonical and the canonical system. How
do the analysis and the results differ for the two cases? We will address
this in a model for vacancies.
Real crystals are more complicated than the simple Einstein crystals.
Real crystals
Microcanonical approach. How can we adress this system using a
microcanonical approach? In the microcanonical system the total energy
of the crystal is given. In a simplified description of the system we assume
that the energy associated with moving a single atom from the interior
to the surface does not depend on the number of atoms n that are moved
from the interior to the surface: There is a fixed cost for moving an
atom from the interior to the surface. This is only valid if the number of
atoms moved from the interior to the surface is small compared with the
number of atoms N in total in the crystal, so that it is unlikely that two
atoms next to each other are removed or that two atoms are placed next
to each other on the surface, because in both these cases we would need
to consider the number of present neighbors in order to find the energy
to remove a particle from the interior and the energy to place a particle
on the surface. In this simplified picture, we assume that the energy of
an atom in the interior is −Z, where Z is the number of neighbors in
the interior, and that the energy for an atom on the surface is −z. For
example, Z = 6 for a cubic crystal and z = 1 for a surface atom on a
flat, cubic crystal surface if we only include an energy  for each binding
to each of the 6 neighbors in a cubic lattice structure as illustrated in
Fig. 7.10.

Fig. 7.10 Illustration of

energies in a small cubic T=0 T>0
crystal.

In this case, the energy of the system is given by the number of

vacancies n:
7.4 Composite systems 235

E = − (nz + (N − n)Z) . (7.144)

In order to develop a thermodynamic description of this model system,
we find the multiplicity and the entropy of this model system. The
multiplicity is given by the number of possible microstates consistent
with the macrostate, that is, consistent with a given number n of vacancies.
We can describe a microstate by the positions of the vacancies inside
the crystal and by the position of the vacancies on the surface and find
the total multiplicity as the product of the number of ways to place n
vacancies in the N interior sites (bulk), Ωb , and the number of ways to
place n vacancies in the approximately N ( 2/3) surface sites (surface) Ωs :

Ω(n, N ) = Ωb (n, N )Ωs (n, N ) , (7.145)

If the number of vacancies n is small compared with the number of

atoms, N , in the crystal, then the number of possible ways to arrange
n vacancies among N atomic positions is N !/(n!(N − n)!). Since the
number of surface sites is much smaller that the number of interior sites
(N ( 2/3)  N when N is large), we ignore Ωs . (We elaborate on this
assumption at the end). The entropy of the system is therefore
N!
S = k ln Ω = k ln . (7.146)
n!(N − n)!

We apply Stirling’s approximation ln(x!) = x ln x − x, getting:

S ' k (N ln N − n ln n − (N − n) ln(N − n)) . (7.147)

We use this to find the temperature of the system:

1 ∂S ∂S ∂n
 
= = . (7.148)
T ∂E N,V ∂n ∂E

We find the last term from

∂E d
= (−zn − Z(N − n)) = −z + Z = (Z − z) = ∆ . (7.149)
∂n dn
And we find that
∂S N −n
 
= k (−n/n − ln n + ln(N − n) + (N − n)/(N − n)) = k ln .
∂n N n
(7.150)
Thus the temperature is
236 7 The canonical system

1 k N −n
= ln . (7.151)
T ∆ n
To find the number of vacancies as a function of temperature, we
invert the expression for the temperature
1 k n
=− ln , (7.152)
T ∆ N − n
which gives:
n ∆
= e− kT , (7.153)
N −n
which we can solve for n, getting:
∆
n = e− kT (N − n) , (7.154)
 ∆
 ∆
n 1 + e− kT = N e− kT , (7.155)
∆
n e− kT
= ∆ . (7.156)
N 1 + e− kT
Canonical approach. Let us see how we can address this same system as
a canonical system. We assume that the crystal of N atoms is in thermal
equilibrium with a large reservoir so that it has a constant temperature
T . In this case, we do not specify the number of vacancies n for the
system, since this would correspond to specifying the total energy of the
system. To find the thermodynamic behavior of this system we find the
partition function and use this to find the (average) energy of the system.
The partition function is given as the sum over all the possible states i
of the system. To describe the state we need to decribe the energy i for
each of the N lattice sites in the system. The energy of the lattice site i is
i = −Z if an atom occupies the lattice at this site and i = −z if the
site is empty and the atom is on the surface. (Here, we assume that the
number of atoms on the surface is so small, that we can assume that we
can describe each site independently of the others. Otherwise, we would
need to include the interactions of an atom with all the surrounding
atoms in detail). Notice that the number of atoms, N , is not changed, it
is only where teh atoms are placed that change. Each lattice site i can
therefore be in one of two possible states, and the sum over all possible
states must therefore be a sum over all possible lattice sites, i1 = 0, 1,
i2 = 0, 1, i3 = 0, 1, . . ., iN = 0, 1:
7.4 Composite systems 237

1 X
1 1
Z= e−(i1 +i2 +...+iN )/kT (7.157)
X X
...
i1 =0 i2 =0 iN =0

This sum can be split into a product of sums since there are no interactions
between the energies in the various sites (this is our assumption):
1 1 1
Z= −i1 /kT −i2 /kT
e−iN /kT (7.158)
X X X
e e ...
i1 =0 i2 =0 iN =0
 N
1
= e−i1 /kT  (7.159)
X

i1 =0

This is indeed the general result we found above: For a system of indepen-
dent components, the partition function is the product of the partition
functions for each of the components. Here, the total partition function
is the partition function for each of the N lattice positions. We find the
single-site partition function
 
Z1 = ez/kT + eZ/kT = eZ/kT 1 + e−∆/kT (7.160)

The energy of the system is then

d ln ZN d    ∆e−∆/kT
E=− =N βZ + ln 1 + e−∆β = −N Z +
dβ dβ 1 + e−∆/kT
(7.161)
We can use this to find the number n of vacancies, through E = −N Z +
n∆, giving
e−∆/kT
n= . (7.162)
1 + e−∆/kT
Which is exactly the same result as we found for the microcanonical
system.
Full model.

7.4.6 Example: Extension of a long polymer

Motivation. A long polymer such as the DNA molecule consists of
sequence of connected monomers. The length of a monomer is ∆l. We
will here address a one-dimensional model for a polymer, consisting of N
monomers folded along the x-axis. Each monomer can be folded either
238 7 The canonical system

to the right or to the left. The number of monomers folded to the right is
NR and the number folded to the left is NL = N − NR . If one end of the
polymer is placed at x = 0, the other end will be at a position x = L:

L = ∆l (NR − NL ) = ∆l (NR − (N − NR )) = ∆l (2NR − N ) . (7.163)

Thermodynamic identity. This system is therefore not described with

a volume, but with a length L instead. How do we need to modify the
thermodynamic identity for this? For a quasi-static, reversible process,
the first law of thermodynamics gives

dE = Q + W , (7.164)

where Q = T dS. The work done on a polymer when it is extended from

L to L + dL is W = f dL, where f is the force pulling on the polymer.
(Notice that this force has the opposite direction to a pressure acting on
a piston in the case of an ideal gas). Thus, the thermodynamic identity is

dE = T dS + f dL ⇒ T dS = dE − f dL , (7.165)

Helmholtz free energy. How do we use this to modify the expression

for Helmholtz free energy, so that we can find a way to calculate f from
F and hence from the partition function for the system?
Helmholtz free energy is given as F = E − T S, therefore

dF = dE − T dS − SdT = T dS + f dL − T dS − SdT = −SdT + f dL .

(7.166)
This means that we can find the entropy and the force from Helmholtz
free energy using:
∂F ∂F
   
S=− , f= . (7.167)
∂T f,N ∂L T,N

We therefore have all the tools needed to address the behavior of a

polymer system — if we only have an expression for Helmholtz free
energy.
Partition function. The partition function can be found by summing
over all the possible states of the system. The state is described by giving
the direction of each of monomer element. We call the state of element i,
si , which may point in the positive direction, si = +1, or in the negative
direction, si = −1. The number of steps to the right is then the number
7.4 Composite systems 239

of times si = +1 and the number of steps to the left is the number of

times si = −1.
In this simplest version of a model for a polymer, we assume that the
energy of the polymer does not depend on how it is folded or whether
it folds onto itself. The energy is just a constant, N ∆, where ∆ is the
energy of a single monomer.
The partition function is the sum over all states consistent with the
macroscopic parameters T, L, N . This means that we need to sum the
energies of all the states that results in the length L:

Z(T, L, N ) = e−i /kT , (7.168)

X

where i = N ∆ is the energy of the state, which is independent on the

configuration of the polymer. The sum is over all states that given a
total length L of the polymers. This corresponds to a sum over all the
states with NR = (1/2) (N + (L/∆l)) and NL = N − NR . How many
ways can we arrange the monomer so that we have NR of the states si
being +1 and NL = N − NR begin −1? Since the order in which the
monomers are folded does not matter for NR , this is the same as the
number NR of ways that we can select NR states from N . That is, the
partition function sum consists of Ω(NR , N ) identical elements:
!
N
Z(T, L, N ) = −N ∆/kT
= e−N ∆/kT . (7.169)
X
e
i
NR

Helmholtz free energy. We can now find Helmholtz free energy from
N!  N!
 
F = −kT ln Z = −kT ln − = N ∆−−kT ln
NR !(N − NR )! kT NR !(N − NR )!
(7.170)
We apply Stirling’s approximation, ln(x!) = x ln x − x, to find an expres-
sion we can take the derivative of

F = N ∆ − kT (N ln N − NR ln NR − (N − NR ) ln(N − NR )) .
(7.171)
The force f on the polymer. We find the force f from Helmholtz free
energy
∂F ∂F ∂NR
     
f= = , (7.172)
∂L T,N ∂NR T,N ∂L
where
240 7 The canonical system

∂NR d 1
 
= ((1/2) (N + (L/∆l))) = . (7.173)
∂L dL 2∆l
and
∂F ∂
 
= (N ∆ − kT (N ln N − NR ln NR − (N − NR ) ln(N −(7.174)
NR )))
∂NR T,N ∂NR
= −kT (−(NR /NR ) − ln NR + ln(N − NR ) + 1) (7.175)
NR
= +kT ln . (7.176)
N − NR
We can now replace NR with NR = (1/2) (N + (L/∆l), giving
kT NR
f= ln (7.177)
2∆l N − NR
kT (N + L/∆l)/2
= ln (7.178)
2∆l (N − L/∆l)/2
kT (N + L/∆l)
= ln (7.179)
2∆l (N − L/∆l)
kT (1 + L/N ∆l)
= ln (7.180)
2∆l (1 − L/N ∆l)
This force is a purely entropic force. The force is due to the forced
uncurling of the polymer.

7.4.7 Example: Ising model in 1d and 2d

Motivate the introduction of Monte Carlo methods in a system where it
is not simple to find the analytical solutions to the partition function.

7.4.8 Example: Grain boundaries

Make a simple model for grain formation and grain boundaries. Develop
a 1d model. Make a 2d model. Simulate using MC methods. Show
coalesence and grain boundary dynamics in this model. How to describe
these types of processes?
7.6 Transformation between ensembles 241

7.5 On the great importance of kT

Discuss how we now can interpret the meaning of kT and how we can
use this to analyze the behavior of systems qualitatively.

7.6 Transformation between ensembles

We introduced Helmholtz free energy, F , as a function with the natural

variables T , V , N . The method we used to develop F from E is a general
method, that we can use to gain a very useful insight into the relation
between the various free energies that we will use in thermodynamics.
The various free energies may often seem unmotivated – you may remain
with the impression that they were dreamed up or that we arrived at
them in some unspecified way. Not so – they are all derived in a very
natural way.
Let us start from the microcanonical system. This system is thermally
isolated and described by the state variables E, V , N . These three state
variables provide the complete state of the system. From these state
variables – and from the relation S = S(E, V, N ), we can calculate other
state variables such as T , p, and µ for the system.
From a thermodynamic perspective – where S(E, V, N ) is a sharp
function of E, V , and N – we could instead describe the system with
the variables S, V , and N , and then find the energy E from a relation
E = E(S, V, N ). We can use the thermodynamic identity to see how we
can gain all the thermodynamic properties from E(S, V, N ) in this case:

T dS = dE + P dV − µdN ⇒ dE = T dS − P dV + µdN . (7.181)

We use these differential forms to find what are the variables describing
the system. Here is it S, V , and N , since these are the varibles for dE.
And we can also use this identity to read out the partial derivatives of E:
∂E ∂E ∂E
     
=T , = −P , =µ. (7.182)
∂S V,N ∂V S,N ∂N S,V

Now, we would like to describe the system with the variables T , V ,

and N instead of S, V , and N , just as we did for the canonical ensemble.
What is then the quantity corresponding to E – the quantity A that
when we take the differential of A we end up with T , V , and N as the
free variables. We want to develop a function A = A(T, V, N ) so that
242 7 The canonical system

∂A ∂A ∂A
     
dA = dT + dV + dN . (7.183)
∂T V,N ∂V T,N ∂N T,V

How can we create such a function and what would the partial deriva-
tives be? Fortunately, there is a general theory in the form of Legendre
transforms that allow us to transform a function E(S, V, N ) into a new
function A(T, V, N ), where T is the partial derivative of E with respect
to S. This transform is called the Legendre transform. In general, if we
have a function f = f (x, y, z) and we want to replace x with its partial
derivative conjugate, t:
∂f
 
t= . (7.184)
∂x y,z
This is achieved by the Legendre transform:

Lf (x, y, z) = g(t, y, z) = f (x(t, y, z), y, z) − xt . (7.185)

This is what we do for the transformation between the microcanonical

and the canonical ensemble: We move from a description of the system
in terms of E(S, V, N ) to A(T, V, N ), by introducing
∂E
 
A(T, V, N ) = E(S(T, V, N ), V, N ) − S(T, V, N ) = E − TS ,
∂S V,N
(7.186)
and this is what we called the Helmholtz free energy. Let us check that
this free energy indeed has the correct dependent variables by finding
the differential for F using the thermodynamic identity:

dF = d(E − T S)
= dE − T dS − SdT = dE − (dE + pdV − µdN ) − SdT (7.187)
= −SdT − pdV + µdN ,

which indeed shows that the dependent variables – the variables occuring
in the differentials on the right hand side, dT , dV , and dN – are indeed
T , V , and N as we wanted.
Now, we are ready to examine other possible combinations of state
variables. What if we want to study a system where the volume is no
longer constant – so that we want to replace V with its partial derivative
conjugate. What is the conjugate of V ? It is −p:
∂E
 
−p= . (7.188)
∂V V,N
7.6 Transformation between ensembles 243

We can now use the Legendre transform to generate the potential

∂E
 
H =E−V = E − V (−p) = E + pV . (7.189)
∂V S,N

This potential is called the enthalpy of the system. Let us check that it
has the correct dependent variables by finding the differential for H:

dH = d(E + pV )
= dE + pdV + V dp
(7.190)
= (T dS − pdV + µdN ) + pdV + V dp
= T dS + V dp + µdN .

Indeed, the dependent variables are S, p, and N .

Let us look at two further systems that we will introduce later on.
What if we want to study systems at constant T, p, N – that is, we want
to replace V with p in F ? In that case, we apply the trasform to F ,
getting
G = F + pV = E − T S + pV , (7.191)
which is called Gibbs free energy.
Similarly, we will want to study chemical reactions in open systems
with constant T , V , and µ, but where N may vary. This corresponds to
changing from N to µ in F , again applying the transform we get

Φ = F − µN , (7.192)

which is called the Grand potential.

It is also useful to develop an interpretation of free energy as the
energy needed to generate a system out of nothing. For the microcanonical
ensemble, the energy E is the energy we need to generate the system.
For the canonical system, we also need to generate the energy E of the
system, but now we do this while in equilibrium with a large reservoir
with temperature T – we therefore get some energy for free from the
reservoir, in the form of heat transfer from the reservoir. This corresponds
to T S. The energy needed to create a canonical system with T , V , and
N is therefore given by Helmholtz free energy, F = E − T S. This is also
the energy we would be able to get out of the system if we annhilated
the system – again we would gain the energy E, but we would now lose
the energy T S to the reservoir.
For a system at constant E, p, N – corresponding to the microcanonical
system, but at constant pressure instead of constant volume, we need to
244 7 The canonical system

generate the energy E, but in addition we need to make room for the
volume V of the system, which requires work done by the system on
the environment, and this work is pV . The energy needed to create the
system is therefore H = E + pV , the enthalpy of the system.
For a system in equilibrium with a heat bath with temperature T and
in an atmosphere/environment with pressure p, we need to generate the
energy E and we need to do a work pV to make room for the system,
but since we are in equilibrium with a heat bath, we get a thermal
energy of T S from the reservoir. The total energy needed is therefore
G = E − T S + pV , which is the Gibbs free energy of the system.
We therefore see that the free energies are the energies needed to
generate a system in various conditions. Tables of free energies are
therefore useful tools to determine the energy needed to create various
systems or to go from one system to another system through chemical
reactions or physical transformations, as demonstrated by the following
examples.

7.6.1 Example: Fuel cell

Fuel cells are used in cars, such as the BMW 750Hl. In a fuel cell hydrogen
and oxygen is combined to form water:
1
H2 + O2 → H2 O . (7.193)
2
What is the energy produced in this reaction, if it occurs at constant
temperature T = 298K and constant pressure?
In this case, we know that the energy we get from creating 1 mole of
water, while removing 1 mole of hydrogen and half a mole of oxygen, is
given by the Gibbs free energy. The energy from the reaction will be
1
∆G = ∆G(H2 O) − ∆G(H2 ) − ∆G(O2 ) . (7.194)
2
In the table for the Gibbs free energy for water in Schroder, it is the
total ∆G which is listed. We find that for water ∆G = −237kJ per mole.
It is negative, because to generate water from its basic compounds, we
gain energy.
Since this process occurs at constant temperature corresponding to
room temperature, we know that Gibbs free energy has two components,
one component from the change in enthalpy and one component from
7.6 Transformation between ensembles 245

the change in entropy:

∆G = ∆H − T ∆S , (7.195)

We therefore lose some of the energy to heat. How much energy is

converted to heat? We can estimate this in two ways: (i) We can simply
use the change in enthalpy tabulated in Schroder – this gives us the
total energy produced in a system that is not in thermal contact with a
heat bath; or (ii) We can calculate the changes in entropy also based on
entropy values from Schroder.
The change in enthalpy for the reaction is

∆H = −286kJ . (7.196)

This means that the entropy change, due to heat, is ∆G = ∆H − T ∆S,

giving T ∆S = Q = ∆H − ∆G = −286kJ + 237kJ = −49kJ, where the
negative sign means that the heat is transferred from the system to the
environment/reservoir in the process. This means that the efficiency of
this process is 237/286, which is 0.83, which is very high!

7.6.2 Example: Muscle cell

We can think of a muscle cell as a fuel cell, but using different chemicals
than the hydrogen fuel cell, of course. The general form of reaction in
the muscle cell is to use glucose to generate water and CO2 :

C6 H12 O6 + 6O2 → 6CO2 + 6H2 O . (7.197)

If we assume that the reaction occurs at room temperature and at one atm
pressure, what is the enthalpy and Gibbs free energy for this reaction?
How can we find ∆H and ∆G for such compound processes? In the
tables in Schroder, and in other sources, the values for ∆H and ∆G
for each compound, such as for glucose, is the enthalpy or Gibbs free
energy when the compound is made from each component in its most
elemental, stable form. For glucose, we assume that it is made from C
(graphite), O2 and H2 . We can therefore find the ∆H or ∆G for the
reaction, by first converting each compound into their elemental forms
and the converting from the elemental forms and into the new compound
– the result of the chemical reaction. We can therefore find what it would
take to turn the left side into its basic components, which is the negative
of what it would take to create the left side from its basic compents. We
246 7 The canonical system

call the enthalpy and Gibbs free energy of the left side ∆HL and ∆GL
respectively – the energies needed to create the left side. Similarly, ∆HR
and ∆GR are the energies needed to create the right side from its basic
components:

∆HL = ∆HC6 H12 O6 + ∆H6O2 = −1273kJ + 6 0kJ = −1272kJ , (7.198)

∆GL = ∆GC6 H12 O6 + ∆G6O2 = −910kJ + 6 0kJ = −910kJ . (7.199)

Similarly, for the right hand side:

∆HR = ∆H6CO2 + ∆H6H2 O = 6 − 393kJ + 6 − 286kJ = −4074kJ ,

(7.200)
∆GR = ∆G6CO2 + ∆G6H2 O = 6 − 394kJ + 6 − 237kJ = −3786kJ .
(7.201)
The change in enthalpy occurs by creating the right side and removing
the left side, thus

∆H = ∆HR − ∆HL = −4074kJ + 1272kJ = −2802kJ , (7.202)

and the change in Gibbs free energy is similar:

∆G = ∆GR − ∆GL = −3786kJ + 910kJ = −2876kJ . (7.203)

Gibbs free energy is negative, meaning that this is the energy available
for mechanical work done by the muscle on the environment.
We see that the magnitude of Gibbs free energy is larger than the
magnitude of the enthalpy. What does this imply? Since ∆G = ∆H −
T S = ∆H − Q, where Q is the heat transferred into the muscle from the
environment due to the energy in the thermal bath, we see that in this
case:

Q = ∆H − ∆G = −2802kJ + 2876kJ = 74kJ . (7.204)

which is the heat dumped from the environment to the muscle (to help
it do work at the same temperature).
7.7 Equipartition and velocity distributions 247

7.7 Equipartition and velocity distributions

7.7.1 Equipartition theorem

We can demonstrate how we can use the results from the canonical system
to prove the equipartition theorem. We address terms in the energy of a
system that are quadratic in shape:

(q) = cq 2 , (7.205)

where q is a degree of freedom, such as the position for a harmonic

oscillator, the velocity for the kinetic energy, the angular velocity for the
rotational kinetic energy, or any other quadratic term.
We will assume that the states are densely, but discretely placed in
q-space (but that they are not degenerate), so that there are states every
∆q. The partition function is therefore

Z= (7.206)
X
e−βcq 2 ,
q

which we will approximate by the integral over all q-values:

1 X −βcq2 1
Z
2
Z= e ∆q = e−βcq dq , (7.207)
∆q q ∆q
√ √
we change variable to x = βcq, so that dx = βcdq, and
1 1 ∞
Z
2
Z= √ e−x dx , (7.208)
∆q βc −∞

where the integral is simply a numerical value (which we do not need to

know) – we call it b.
1
Z= √ b = Cβ −1/2 . (7.209)
∆q βc
(Notice that we are here only conserned with the scaling with β of this
function – not the exact functional form.) The energy is then found from:
1 ∂Z 1 1 1
 
Ē = − = (−1/2)Cβ −3/2 = β = kT , (7.210)
Z ∂β Cβ −1/2 2 2
and we have proved the equipartition theorem – which is true as long
as the states are spaced in q-space with no degeneracy. Notice that this
assumption is only reasonable as long as the spacing between levels is
248 7 The canonical system

much smaller than kT , which is not true for quantum systems at low
energies.

7.7.2 Maxwell Velocity Distribution

For a single particle in an ideal gas, we know that the probability for
the particle to be in the state (nx , ny , nz ) is given by the corresponding
Boltzmann term
1 −a(n2x +n2y +n2z )/kT
P (nx , ny , nz ) = e . (7.211)
Z
But what is the probability for the particle to have a particular velocity?
What is the velocity distribution of an ideal gas.
Velocity distribution in one dimension. First, let us specify the ques-
tion more. The translational motion of a particle in an ideal gas is
independent in the x-, y-, and z-directions. Hence the state is specified by
three, independent, numbers nx , ny , and nz . These numbers are related to
the momentum of the particle. From quantum mechanics, we recall that
the momentum is px = ~(π/L) nx , and similarly for the other directions.
(This can be seen from the expression for the energy. The energy for
motion along the x-axis is (nx ) = an2x . This corresponds to the kinetic
energy of the particle, and therefore mvx2 /2 = p2x /2 = (nx ) = an2x =
~2 /(2m) (π/L)n2x , which gives px = ~ (π/L) nx — up to the sign.)
We can therefore relate the velocity to the momentum, vx = px /m =
~(π/mL) nx . What is then the probability for a particular velocity, P (vx )?
We know the answer to this. If vx is exactly equal to a value vx =
~(π/mL)nx for some nx , then the probability is P (nx ), otherwise the
probability is zero. Hmmmm. This is not that practical. Instead, we may
ask what the probability is for the velocity to be in some interval from
vx to vx + dvx , which corresponds to the nx value to be in the interval
from nx to nx + dnx . This corresponds to what we called the probability
density in Sect. 4.5.7:

P (vx is in range (vx , vx + dvx ) ) = P (vx )dv x , (7.212)

where we call P (vx ) the probability density for vx . Fortunately, the

probability density along the nx axis is simple. There is one state at
nx = 1, then one at nx = 2, then one at nx = 3. This means that
the number of states per unit nx is one. We can therefore estimate the
probility to find a state in the range from nx to nx + dnx as the probility
7.7 Equipartition and velocity distributions 249

to be in one of the states times the number of states in the interval

1 −an2x /kT
P (nx )dnx = e dnx , (7.213)
Z
The probability to be in one of the states is the Boltzmann factor for nx
and the number of states is simply dnx — since the density of states is
1 along the nx axis. The probability for the velocity to be in the range
vx to vx + dvx is the probability for nx to be in the corresponding range
from nx (vx ) to nx (vx ) + dnx , which is
dnx 1 2 mL 2
P (nx )dnx = P (vx (nx )) dvx = e−mvx /2kT dvx = Ce−mvx /2kT .
dvx Z ~π
(7.214)
The probability density for the velocity along the x-axis is therefore a
Gaussian function in vx . The constant C can be determined from the
normalization of the probability density.
Velocity distribution in three dimensions. We can use this argument
for each of the three dimensions, finding that the probability density for
the velocity in three dimensions is
2 2 2
P (vx , vy , vz ) dvx dvy dvz = C 3 e−m(vx +vy +vz )/2kT dvx dvy dvz (7.215)
−mv 2 /2kT
= C 3e dvx dvy dvz , (7.216)

where v = (vx2 + vy2 + vz2 )1/2 is the speed. This distribution is given in the
vx , vy , vz -space. What if we instead wanted to know the probability for
v to be in some interval from v to v + dv. Then we need to know what
volume in v-space this corresponds to: It corresponds to a spherical shell
from v to v + dv, which has a volume 4πv 2 dv when dv is small. Hence
the probability to find the velocity in the range from v to v + dv is
2 /2kT
P (v)dv = C 3 e−mv 4πv 2 dv , (7.217)

where the factor C is determined by normalization:

Z ∞ 2 /2kT
C 3 e−mv 4πv 2 dv = 1 ⇒ C 3 = (m/2πkT )3/2 . (7.218)
0

This distribution of speeds is called the Maxwell distribution of the

speeds of particles in an ideal gas:
250 7 The canonical system

The Maxwell distribution of the speeds v of a particle in a

three-dimensional ideal gas is
3/2
m

2 /2kT
P (v)dv = 4πv 2 e−mv dv . (7.219)
2πkT

Fig. 7.11 illustrates both the distribution of vx , the velocity along the
x-axis, and the distribution of v, the speed of the particle.

1 1
P (vx )=Pmax

P (v)=Pmax
0.5 0.5

0 0
-5 0 5 0 1 2 3 4 5
$
vx =v v=v $
Fig. 7.11 a Probability density for the velocity vx along the x axis for a particle in
p gas. b Probability density for the speed v of a particle in an ideal gas. Here,
an ideal
v ∗ = kT /m.

Average velocity. Notice that the average velocity, v, is zero, while the
average speed, v, is non-zero. This is because the distributions for vx ,
vy , and vz are symmetric around zero, while the distribution for the
magnitude, v, of the velocity of course is zero when v < 0. The average
velocity is Z ∞ 2
hvx i = Ce−mvx /2kT vx dvx = 0 , (7.220)
−∞

and the average speed is (after simplifying the integral):

s
8kT
∞ 3/2
m
Z 
−mv 2 /2kT 2
hvi = e v dv = . (7.221)
−∞ 2πkT πm
The velocity of N2 in the air. This means that for air, which mostly
consists of nitrogen, the average speed of the nitrogen molecules at
T = 300 K is:
s s
8kT 8 × 1.38 × 10−23 m2 kg s−2 K−1 × 300 K
hvi = = ' 476 m/s .
πm π × 28 × 1.66 × 10−27 kg
(7.222)
7.8 Statistical entropy 251

We could also ask for the probability for a molecule to move with at least
twice the average velocity, that is P (v ≥ 2va ), where we have written
va = hvi to simplify the notation. This is estimated by summing up all
the probabilities for v ≥ 2va , that is, by integrating P (v) from 2va to
infinity:
∞ ∞ 3/2 Z ∞
m
Z Z 
2 /2kT
P (v ≥ 2va ) = P (v) dv = 4π v 2 e−mv dv .
va va 2πkT va
p (7.223)
We change variable to simplify the exponent, introducing u = v m/2kT ,
du = m/2kT dv, and
p

q q q q
ua = va m/2kT = 2 8kT /πm m/2kT = 2 4/π ' 2.26 . (7.224)

2kT
3/2  3/2 Z ∞
m

2
P (v ≥ 2va ) = 4π x2 e−x dx . (7.225)
2πkT m ua

This integral is solved numerically in python:

from scipy.integrate import quad
def fun(x):
return x**2*exp(-x**2)
quad(fun,2.26,Inf)

Out[12]: (0.007454366605316702, 1.0002475807012429e-11)

The probability is therefore 0.0075 to have a speed of more that twice

the average speed, which corresponds to the probability for the speed of
a dinitrogen molecule to be higher than 952 m/s.

7.8 Statistical entropy

From the average energy we get:

Ē = s P (s) , (7.226)
X

Now, the total change in energy is therefore:

dE = s dP (s) + P (s)ds , (7.227)

X X

s s

Where we can write the second term as:

252 7 The canonical system

ds
P (s)ds = P (s) dV = −P̄ dV , (7.228)
X X

s s dV

and therefore the change in energy is:

dĒ = s dP (s) − P̄ dV . (7.229)

X

If we compare with the thermodynamic identiy, we see that we can

interpret:
T dS = s dP (s) , (7.230)
X

This expressions can (with some tricks) be related to another expression

for the entropy – which we could instead have started from:

T dS = s dP (s) = −kT ln ZdP (s) − kT ln P (s)dP (s)

X X X

s s s
(7.231)
where we have used that

P (s) = (1/Z) exp(−s /kT ) , (7.232)

and therefore that

s = −kT (ln Z + ln P (s)) . (7.233)

Notice that dP (s) = d P (s) = d(1) = 0. We therefore get:

P P
s s

dS = −k ln P (s)dP (s) = −k d(P (s) ln P (s)) , (7.234)

X X

s s

and
S = −k P (s) ln P (s) , (7.235)
X

which is a general expression for the entropy. This relation also holds
in the microcanonical ensemble, since then Ps = 1/Ω for all states and
therefore we find S = k ln Ω after summing.

7.9 General Monte-Carlo framework

Introduction to a general monte-carlo framework that will be used

throughout the book to study for Microcanonical systems, Polymers,
7.10 Simulations of a canonical system 253

Random walks, Vacancies, Ising model, Einstein crystals, XY model type

systems, Binary mixtures, and Lattice gases.

7.10 Simulations of a canonical system

While molecular dynamics simulations were simple to perform in the

microcanonical ensemble, they are not similarly simple in a canonical
system because energy is no longer conserved and we need to introduce
the effect of a large heat bath. There are several ways this can be done,
and we discuss these briefly below. The connection to the heat bath
means that the canonical system lends itself more naturally to stochastic
simulations methods such as Monte Carlo methods. We will therefore
start with a more systematic introduction to Monte Carlo simulations
methods and their application to the canonical system.
Here, we will first demonstrate how we can use Monte Carlo methods
for more general purposes, such as for calculating an integral, before we
address how we can apply Monte Carlo methods to collect statistical
about a canonical system.

7.10.1 Monte Carlo estimates of integrals

How can we find the integral of a function f (x)?
Z x1
y= f (x) dx . (7.236)
x0

This integral can be interpreted as the integral under the curve f (x) from
x0 to x1 . We know how to solve it numerically by various integration
methods, but we can also use a stochastic (Monte Carlo) algorithm to
estimate the area.
Hit or miss algorithm. For example, we can use a “hit or miss” algorithm:
To estimate the integral we choose a box from y = 0 to some y1 > f (x)
for all x on the interval. We then select N points randomly in the interval
x0 to x1 and y0 = 0 to y1 . We can then estimate the integral by the
fraction of points falling below the curve f (x) multiplied by the area of
the box:
Nbelow
y' (y1 − y0 ) (x1 − x0 ) . (7.237)
N
254 7 The canonical system

We expect this to become a better and better estimate as N goes√to

infinity. Indeed, it can be shown that the error is proportional to 1/ N
for this method.
Hit and miss integration examples. How does this method work for a
simple example? Let us try to integrate the function

f (x) = (sin(10 x) + cos(50 x))2 , (7.238)

This function, and N = 1000 random points, are illustrated in Fig. 7.12.
In this figure we also show the estimates integral I as a function of the
number of steps i used to estimate the integral. This was found using
the following simple implementation of the hit or miss algorithm. Notice
that we first use the symbolic package to find the exact solution for
comparison.
# Symbolic solution
from pylab import *
from pylab import plot as plt
from sympy import *
u = symbols(’u’)
f = (sin(10*u)+cos(50*u))**2
g=integrate(f)
fint = float(g.subs(u,1))
# Convert to Python function
fun = lambdify(u,f,"numpy")
# Plot f(x)
xx = linspace(0,1,1000)
yy = fun(xx);
plt(xx,yy);
hold(’on’)
# Calculate integral using hit and miss
fmax = 4.0 # Upper limit of f(x) for x (0,1)
N = 1000
hit = 0.
intest = zeros((N,1),float)
for i in range(1,N):
x = rand(1); y = rand(1)*fmax
if (y<fun(x)):
hit = hit + 1
intest[i] = hit/i*fmax
plt(array(range(0,N)),intest,’-r’)
xlabel(’i’);ylabel(’I’);

Hit and miss in high dimensions. Why would we calculate the integral
in this way? It seems very inefficient. This is true for a one dimensional
function. However, for integrals in high dimensions, ordinary integration
method does not scale well as the number of dimensions increase, but the
stochastic method introduced here is also efficient in high dimensions.
7.10 Simulations of a canonical system 255

4 1.2

1
3
0.8

2 0.6
y

I
0.4
1
0.2

0 0
0 0.2 0.4 0.6 0.8 1 0 200 400 600 800 1000
x i
Fig. 7.12 a Plot of the function f (x) and the points used to estimate the integral. The
color indicates if the point is below f (x) or not. b Plot of the estimated integral I as a
function of the number of iterations i used for the iteration.

Notice that we do not use this very simple algorithm for real integration.
Instead, there are more sophisticated methods that give a more rapid
approach to the correct answer. We will not introduce these here for
integration, but instead demonstrate examples of more advanced methods,
such as importance sampling, for simulations of a canonical system.

7.10.2 Monte Carlo method for a canonical system

For a canonical system, we want to simulate the behavior of a system
that is in equilibrium with a heat reservoir of temperature T . In the mi-
crocanonical ensemble, we know that all microstates are equally probable,
but in the canonical ensemble, the probabilty of a microstate is given by
the Boltzmann factors. How can we then sample from the microstates
and ensure that we satisfy this particular distribution?
Sampling microstates. We would like to generate a sequence of mi-
crostates of the canonical system: A sequence {i1 , i2 , . . .} of states i. We
may do this by changing the microstates at random, for example by
making a small change to one particle: Flipping a spin, changing the
energy of an oscillator, or moving a particle in a gas. However, since
the microstates are not equally probable, each of the microstates in the
sequence do not weigh equally when we calculate averages from this
sequence of microstates. A microstate i of the system has energy Ei and
should be weighted according to the probability, exp(−Ei /kT )/Z, of this
state. This would for example allow us to estimate the average energy
from the sequence {i1 , i2 , . . . , iM } of (random) states:
256 7 The canonical system

M
1 X
Ē ' Ei e−Eij /kT . (7.239)
Z j=1 j

Unfortunately, we do not know Z, but we can estimate the normaliza-

tion condition by the measured sum of all states, so that the sum of
probabilities for all the states are normalized:
M
e−Eij /kT . (7.240)
X
Z'
j=1

The estimate for the energy becomes:

PM −Eij /kT
j=1 Eij e
Ē ' PM −Ei /kT . (7.241)
j=1 e
j

This gives us a reasonable method to perform a statistical sample. We still

need the sequence of states to be representative in order for this sequence
to give reasonable results, just as we assumed in the microcanonical
system. However, this method is in general very slow! Why? Because
may of the M states will typically have high energy, and thus have very
small probabilities to occur, and therefore contribute very little to both
the sum and the average.
Importance sampling. We call the method we have introduced so far a
naive Monte Carlo method, because we sample completely at random,
and hence, we sample many states with low probability. It would be
better if we could use the Boltzmann factors to guide our choice of
random states, and instead sample mostly the important states. We call
methods to do this importance sampling methods. The idea is to generate
a sequence of microstates that are probable, while still representing a
sufficiently random sample of the the microstates.
Metropolis algorithm. An example of an importance sampling method
is the Metropolis algorithm, which was introduced by Nicholas Metropolis
in 1953 [1]. The Metropolis algorithm generates a sequence of states,
{i1 , i2 , . . .}. We may start from any random state, i1 . To generate the next
state, i2 , we first make a small perturbation of the state i1 , Ei01 = Ei1 +∆E,
for example by flipping a spin in a system of magnetic spins, by changing
the energy of a harmonic oscillator, or by moving a particle in an ideal
gas. If the energy of the new is lower, ∆E < 0 then we accept the change
since this will move the system to a more probable state. If the energy
is higher in the new state, ∆E > 0, then we accept the change with a
7.10 Simulations of a canonical system 257

probability given as e−∆E/kT . We then repeat this process to generate a

sequence of states.
Subset of states. The Metropolis algorithm generates a subset of states
that have high probabilities to occur: States with low energies will occur
frequently in the sequence, whereas states with higher energies will be
less likely, but not completely improbable. This means that averages
using this sequence may be better than for a completely random sequence
— although we cannot really here quantify how much better.
The states in the generated sequence are Boltzmann distributed.
However, the Metropolis algorithm not only generates a sequence of states
that are better suited for averages. In addition, the states in the sequence
occur with a frequency given by the Boltzmann distribution! This means
that a state i with energy i , will occur with a frequency proportional to
e−i /kT in the sequence of states. (The Metropolis algorithm can actually
be formulated generally for any distribution, not only the Boltzmann
distribution, and it can be used to generate a sequence of numbers that
are distributed according to the prescribed distribution).
How can we demonstrate that the frequency of occurence in the se-
quence is Boltzmann distributed? This is usually demonstrated using
the concepts of Markov chains, which are typically introduced in in-
troductory courses in statistical mechanics. The idea is that for two
states i and j, where i ≤ j , the probability for a transition from j → i
according to the Metropolis algorithm is P (j → i) = (1/M ), because
this transition will occur, but the probability to choose state j among M
possible trial states is (1/M ). The probability for a transition from i → j
is P (i → j) = (1/M )e−(j −i )/kT . Thus the ratio of the two transition
probabilities are

P (i → j) (1/M )e−(j −j )/kT e−j /kT

= = −i /kT . (7.242)
P (j → i) (1/M ) e
This is the ratio of the Boltzmann factors for the two states. However,
there are may other states in the system, and there are therefore also
many other ways to go from state i to state j. But even if we include
any intermediate sequence of states between i and j, the ratio of the
probability of a sequence linking state i and j, and its reverse sequence,
would still correspond to the Boltzmann factor. Hence, the frequencies
at which they occur will also be this exact ratio.
We have therefore argued that the sequence of states generated by the
Metropolis algorithm will sample the states with frequencies proportional
258 7 The canonical system

to their frequency of occurrence — a strong statement which means that

we can also use this sequence to gather information about the distribution
of states as well as use the sequence to estimate the partition function
and average values.
ams 4: The argument as presented here is weak.
Disadvantages of the Metropolis algorithm. It is important to remem-
ber that the Metropolis algorithm typically only makes small changes
to the states in each step, therefore the states in the sequence will be
correlated. We would expect that if we only use every N th state in the
sequence, the states would be sufficiently uncorrelated, but we do not
yet have a good estimate as to what value of N would be sufficient to
generate sufficiently uncorrelated statistics. In addition, we may have
started the sequence from an initial state which is unlikely, and we would
therefore also need to wait an initial time until the system has reached
a stationary and representative state, where the states in the sequence
represent the variations in the system in a good way. Again, we do not
really know how long we need to wait or what is meant by the word
representative here — it may take a very long time to reach reasonable
states. However, we will keep these limitations in mind, but still use the
Metropolis algorithm to study systems where our analytical methods are
insufficient.
Implementation of Metropolis algorithm. We can now implement an
algorithm to generate a sequence of states using the following steps of
the Metropolis algorithm:
1. Choose an initial microstate, for example, random spins in the Ising
magnet or random energies for each oscillator in the Einstein crystal.
2. Make a random change in the microstate, a trial change. Calculate
the change of energy going from the old state to the new state, ∆E.
For example, for the Ising system we may choose a spin at random
and see what happens if we flip it; for the Einstein crystal choose an
oscillator at random and change its energy by ±1.
3. Generate a random number, r, between 0 and 1
4. If r ≤ exp(−∆E/kT ) accept the change. (This formulation means
that we will always accept a change where ∆E > 0.)
5. Repeat
This generates a sequence of states that you can now make measurements
on. Let us see how this is implemented for an Ising magnet system.
7.10 Simulations of a canonical system 259

Application to Ising magnet. An Ising system consists of a set of spins,

Si , at a site i, which interacts with neighboring spins as well as with an
external magnetic field, H, so that the total energy of an Ising system
with spins (S1 , S2 , . . . , SN ) is
 

= (7.243)
X X
Si −JSj − H  ,
i j:i

where the sum over j : i are over all the neighbors of site i. We can then
implement the Metropolis algorithm to generate a sequence states for
the Ising system by selecting a spin, and then find the probability for
flipping this spin to be either up or down, and the flip the spin according
to these probabilities:
• Choose a random site i
• Find the energy of this site if Si = 1 and if Si = −1, E+ and E− .
• Set the spin up with probability exp(−βE+ )/C and down with prob-
ability exp(−βE− )/C with C = exp(−βE+ ) + exp(−βE− ).
• Repeat.
ams 5: Illustrate this algorithm with a figure.
How do we find the energy of the site? We calculate it from the energy
function. We therefore need to know mi = j Sj for the neighbors of site
P

i and calculate E+ = −Jmi − H and E− = Jmi + H, where H = µB.

We can do this for one site i at a time, as is done in the following script.
# Heat bath Monte Carlo Ising model one spin at a tim
from pylab import *
nstep = 100 # Number of MC steps
N = 100 # System size
Jdivk = 2.0/3.0 # Interaction
Hdivk = 0.0 # External field
T = 0.1 # Dimensionless temperature
JdivkT = Jdivk/T
HdivkT = Hdivk/T
# Make initial random spin configuration
spins = randint(0,2,(N,N))*2-1
sizespins = size(spins);
N2 = N*N;
% Evolve system
for istep = 1:nstep
for jstep = 1:N2
posx = randi(N,1);
posy = randi(N,1);
neighbors = [sub2ind(sizespins,mod(posx+1-1,N)+1,posy) ...
sub2ind(sizespins,mod(posx-1-1,N)+1,posy) ...
260 7 The canonical system

sub2ind(sizespins,posx,mod(posy+1-1,N)+1) ...
sub2ind(sizespins,posx,mod(posy-1-1,N)+1)];
sumneigh = sum(spins(neighbors));
thisspin = spins(posx,posy);
DeltaE = -JdivkT*thisspin*sumneigh - HdivkT*thisspin;
change = (rand(1,1)<exp(DeltaE))*(-2)+1;
spins(posx,posy) = spins(posx,posy)*change;
imagesc(spins); axis square; drawnow;
end
imagesc(spins); axis square; drawnow;
end

Unfortunately, this method is very slow in python. We can speed it

up by doing it for all states each step. Now, we cannot really do it for
all sites at the same time, because if we change both i and one of its
neighbors j at the same time, we do not really know how the energy
changed. Instead, what we can do is to change half of the lattice each
time, to ensure that there are no ambiguities. This is implemented in the
following program. Resulting simulation patterns are shown in Fig. 7.13.
% Heat bath Monte Carlo Ising model half lattice at a time
clear all; clf;
nstep = 320; % Number of lattice changes
noutput = [10 20 40 80 160 320];
N = 800; % System size in x and y direction
T = 0.8; % Dimensionless temperature
JdivkT = 1.0/T; % Interaction
HdivkT = 0.0/T; % External field
% Make initial random spin configuration
spins = randi(2,N,N)*2-3;
% Find half-matrix of sites for spin change
halflattice = zeros(N,N);
halflattice(1:2:N,2:2:N)=1;
halflattice(2:2:N,1:2:N)=1;
% Evolve system
for istep = 1:nstep
sumneighbors = circshift(spins,[0 1]) + ...
circshift(spins,[0 -1]) + ...
circshift(spins,[1 0]) + ...
circshift(spins,[-1 0]);
DeltaEdivkT = -spins.*(JdivkT*sumneighbors+HdivkT);
pboltzmann = exp(DeltaEdivkT);
changespin = -2*(rand(N,N)<pboltzmann).*halflattice+1;
spins = spins.*changespin; % Flip spins
halflattice = 1-halflattice; % Other half is next
imagesc(spins); axis square; drawnow;
end
7.10 Simulations of a canonical system 261

S t e p = 10 S t e p = 20 S t e p = 40

200 200 200

400 400 400

600 600 600

800 800 800

200 400 600 800 200 400 600 800 200 400 600 800

S t e p = 80 S t e p = 160 S t e p = 320

200 200 200

400 400 400

600 600 600

800 800 800

200 400 600 800 200 400 600 800 200 400 600 800
Fig. 7.13 Illustration of spins in the Ising model for T = 0.8.

7.10.3 Detailed balance

Are there any limits to what kinds of rules we can choose for the transi-
tions between states like we did in the Metropolis algorithm? Could we
invent any rule we like?
No! The transition rates must satisfy a principle called detailed balance.
What is this?
What do we want to model? We want to model a sequence of mi-
crostates for the system – like a time development of microstates. Let us
write the states as j: The possible states of the system is 1, 2, 3, etc. If
the system is in state i at time t we can write this as i(t).
We want to find the next state, j(t+∆t). Then, we need the probability
to get to state j for all possible configurations at time t. And not only
that, in general we may consider that case that the probability to be
in state j at time t + ∆t depends not only on the state the system was
in at a time t, but also all the previous states at earlier time. Hmmm.
That was a lot of information. We can simplify the problem if we remove
the history dependence – if we assume that the probability to be in a
state j(t + ∆t) only depends on the state i(t) it was in at time t and not
262 7 The canonical system

on what happened before that. We can think of this as implying that

the state i(t) contains all the relevant information about the system and
determines what new states can occur. If the state only depends on the
previous state and not on the history we call the sequence of states a
Markov chain.
What we need is the probability P (j, t + ∆t) to be in state j for each
possible state i at t. And we need this probability for all states j and
i. We call the probablity to go from state i at t to state j at t + ∆t the
transition probability:

Wij = W (i → j) = P (j(t + ∆t)|i(t)) , (7.244)

where the symbol | means “given that” — it is a conditional probability.

We know that Wij ≥ 0 and that j Wij = 1: Starting from state i at
P

time t we know that it must end up in some state j at time t + ∆t.

The probability P (j, t + ∆t) is therefore:

P (j, t + ∆t) = P (j, t + ∆t|i, t) = Wij P (i, t) . (7.245)

X X

i i

What determines how this system develops? The master equation – a

cool name – which is given as
dP (j, t)
=− Wji P (j, t) + Wij P (i, t) . (7.246)
X X
dt i i

ams 6: Remove or finish this discussion.

7.11 Summary

7.12 Exercises

Exercise 7.1: Numerical values for the partition function

Consider an atom with two possible states, a ground state with energy 0
and an excited state with an energy of 1 eV.
a) Find the partition function, Z(T ), for this system.
b) Plot the partition function as a function of temperature. What are
the numerical values for T = 300 K, T = 3000 K, and T = 30, 000 K?
7.12 Exercises 263

c) Plot the probability for the system to be in its ground state as a

function of T .

Exercise 7.2: Changing the energy scale

An atom can be in three states with energies −, 0, and .
a) Find the partition function for this atom. Plot the probability for the
system to be in each of the three states as a function of kT /.
We are free to choose the energy scale. If we shift the energy scale by
, the atom may instead be in states with energy 0, , and 2.
b) Find the partition function for this atom now and plot the probability
for the system to be in each of the three states as a function of kT /.
c) What aspects of this problem change when we change the energy
scale, and what aspects do not change?

Exercise 7.3: Thermal excitations in the Hydrogen atom in

the Sun
The transition energies between levels in the Hydrogen atom are given
by the Rydberg formula
1  
= Rz 2 n−2
1 − n −2
2 , (7.247)
λ
where z is the atomic number and R = 1.097373 107 m−1 . The first
few energy levels are 1 = −13.6 eV, 2 = −3.4eV, 3 = −1.51 eV, and
4 = −0.85 eV. The degree of degeneracy is g(n) = n2 — and double that
if spin degeneracy is included. The temperature in the Sun’s atmosphere
is typically T = 5800 K.
a) What is the ratio of probabilities for a Hydrogen atom to be in its
first excited states compared to its ground state? For every billion atoms
in the ground state, how many are approximately in the first excited
state?

Exercise 7.4: Thermal excitations of the water molecule

A water molecule can vibrate in many ways. Typical models for water
include both the vibration of the positions of each of the hydrogen atoms
264 7 The canonical system

relative to the oxygen atom as well as the flexing of the angle between
the two OH bonds — this angle vibrates around the average angle of
104◦ with a typical frequency of ω = 3.0 1012 s−1 . You may assume that
this vibration can be modelled as a harmonic oscillator with energy levels
(n) = ~ω(n + 1/2).
a) Find the partition function Z for T = 300 K. (It is sufficient only to
include the first three terms in the sum).
b) What is the probability for the water molecule to be in flexing states
n = 0, 1, 2 ?
c) Compare this with the vibrational states for the OH bond, for which
ω ' 3.4 1013 s−1 . Comment on the results.

Exercise 7.5: Three-level system

We will study a system consisting of a single particle that may be in
three possible states with energies −, 0, and .
a) Find the partition function, Z(T, V, N ), for the system.
b) Find Helmholtz free energy for the system, F = F (T, V, N ).
c) Find the entropy from Z. What is the limiting value when kT / → ∞,
and when kT / → 0. Comment on the results?
d) Find the energy, E, for the system from a sum over the probabilities
for each state and show that the result is in accordance with F = E − T S,
with the expressions for F and S you found above.

Exercise 7.6: Partition function of two systems

Consider a system consisting of two parts, A and B. Show the under
certain conditions, we can write the partition function of the system as
a product of the partition functions for each of the parts, Z = ZA ZB .

Exercise 7.7: Arrhenius processes

Many processes in chemistry, biology and physics are considered to be
thermally activated processes. In this case, the rate of the process is
controlled by an energy barrier that the partners in the process must
7.12 Exercises 265

overcome in order for the reaction/process to happen. The energy barrier

is called the activation energy, ∆. The probability for two partners in
a process to attain the energy ∆ is given by the Boltzmann factor,
e−∆/kT .
a) Show that if the process can occur at an energy of ∆ or higher, the
rate of the process will be proportianal to the Boltzmann factor, e−∆/kT .
Experimental results are often analyzed in an Arrhenius diagram. You
draw 1/T along the x-axis, and the logarithm of the measured quantity
along the y axis.
b) Fig. 7.14 shows experimental measurements of the etching velocity of
silicon in a chemical etch (HF:HNO3 :H2 O). Find the activation energy
for the process.

Fig. 7.14 Measurement 101

of the reaction rate as a HF:HNO :H O (65:20:15)
3 2
function of temperature for
Silicon in a chemical etch
Reaction rate (7m/min)

of (HF:HNO3 :H2 O).

100

10-1
3.1 3.2 3.3 3.4 3.5 3.6
103 =T

Exercise 7.8: Energy fluctuations

Here, we will address energy fluctuations in a system held at constant
(T, V, N ) — that is, for a canonical system.
a) Show that the energy fluctuations can be expressed as
∂E
 
h( − hi)2 i = (kT )2 . (7.248)
∂T V

b) Estimate the typical relative fluctuation in the energy of a monatomic

ideal gas with a volume of 1 liter at room temperature and atmospheric
pressure.
266 7 The canonical system

Exercise 7.9: Hydrogen gas

The H2 molecule has vibrational states with energies
 
i '  1.03i − 0.03i2 , i = 0, 1, 2, . . . , 15 (7.249)

(There are no more discrete levels beyond i = 15 because the molecule

disassociates — it breaks up.)
a) Plot i / as a function of i. How does the spacing between energy
levels change with i?
b) Write a computer program to calculate the partition function as a
function of kT /.
c) Calculate the average energy and the heat capacity, CV , for the
Hydrogen molecule and plot the results as functions of kT /.
d) Compare the results to that of an harmonic oscillator with the same
energy difference between the ground state and the first excited state.

Exercise 7.10: One-dimensional ideal gas

We study a one-dimensional ideal gas with N particles. The particles
can move freely along the x-axis over a distance L. Ignore the y- and
z-directions.
a) Find the partition function of the gas.
b) Find the energy of the gas at the temperature T .
c) Find the entropy for the gas.
Assume that the N particles can move freely in the x- and the y-
directions over an area L2 .
d) Find the partition function, the energy and the entropy for the gas.
If we limit the dimensions of the gas, we can also limit the degrees of
freedom of the gas we are studying. Quantum mechanically, we assume
that the wave function and the Hamilton operator can be separated into
x-, y-, and z-components, Ψ (r) = Ψ (x) Ψ (y) Ψ (z). The eigenvalues for the
energy can then also be written as E = Exy + Ez . The two-dimensional
case we could therefore consider to be a result of shrinking on of the
edges, Lz , of a cube with edges L. Then we know that (i) the rest energy
in the z-direction is large, (ii) the distance between the energy levels in
the z-direction is large. We can therefore assume that all the particles
will be in the ground state in the z-direction.
7.12 Exercises 267

e) Show that a good partition function for the two-dimensional case will
be
Z = Z2D (exp (−z,0 /kT ))N , (7.250)
where Z2D is the partition function found for the two-dimensional system
above, and z,0 is the ground state for motion in the z-direction.
f) Find the energy using the partition function in (7.250).

Exercise 7.11: Heat capacities of an ideal gas

The heat capacity at constant pressure and volume are defined as
∂E ∂E
   
CV = , Cp = , (7.251)
∂T V ∂T p

respectively.
a) Show that for an ideal gas Cp − CV = N k.

Exercise 7.12: Compressibility of an ideal gas

The compressibility κx for an process where x is constant is defined as:
1 ∂V
 
κx = − . (7.252)
V ∂p x

a) Find an expression for the isothermal compressibility for an ideal gas.

b) May the comressibility be negative for a system in equilibrium?

Exercise 7.13: Equipartion for a linear degree of freedom

We consider a system where the energy depends on the absolute value of
the degree of freedom, x, and not on the square of the degree of freedom:
(x) = cx, where c is a constant. (For example, for a relativistic particle,
the kinetic energy is proportional to its momentum). Show that the
equipartition theorem for this system gives an average energy of kT per
such a linear degree of freedom.
268 7 The canonical system

Exercise 7.14: Speed of an oxygen molecule

a) Show that the most likely speed of a molecule is v = 2kT /m
p

according to the Maxwell speed distribution.

b) Find the average speed, the most probable speed, and the standard
deviation of the speed of an oxygen molecule at room temperature.
c) What is the fraction of oxygen molecules travelling at less than
300 m/s at room temperature?

Exercise 7.15: Escape velocities

We will here address the escape velocity of particles in the Earth’s upper
atmosphere, at a distance of about R = 6 400 textkm from the center of
the Earth. The temperature at this height is approximately T = 1000 K.
a) What is the espace velocity of a particle at this distance from the
Earth’s center?
b) What is the probability for a nitrogen molecule, a hydrogen molecule
and a helium atom to have a velocity that is higher than the escape
velocity? Discuss the consequences.
c) Compare with the escape velocity on the moon’s surface (R =
1 737 km) and discuss the consequences of your results.

Exercise 7.16: Informational entropy

In information theory, the entropy of a signal or the entropy of a discrete
random variable with possible states (outcomes) si is defined as

H=− P (si ) ln P (si ) , (7.253)

X

where the sum is over all the states i of the system and P (si ) is the
probability for state i with outcome si .
a) Show that this formula corresponds to Boltzmann’s formula S/k =
ln Ω for the microcanonical system where the probability for all accessible
states are the same, P = 1/Ω.
b) Show that this formula also corresponds to the entropy for a Canonical
system in thermal equilibrium at temperature T , where the probability
for a state s with energy s is P (s) = (1/Z)e−s /kT .
7.13 Projects 269

7.13 Projects
 Thermodynamics and Heat engines
8

We have now developed the microscopic foundation of thermodynamics.

We have demonstrated the arrow of time and basic concepts such as
entropy and free energy and how to calculate them based on a microscopic
theory. We can, for example, calculate the entropy of a microcanonical
system or the Helmholtz free energy for a canonical system, and then find
other state variables through derivatives of these quantities. However,
there are many ways we can determine the state variables and the
equations of state for macroscopic systems: It can indeed be through a
microscopic theory, it can be through simulations, or it can be through
direct, experimental measurements. Thermodynamics addresses how we
can describe macroscopic systems when we assume that the basic laws
of thermodynamics are known as well as equations of state for all the
systems involved.
In this chapter, we will review thermodynamics, and then see how
we can apply theromodynamics to discuss heat machines, such as steam
engines, diesel engines, stirling engines, and heat exchangers. We will
place particular emphasis on the stirling engine, which is elegant in design
and frequently used for solar energy applications. Indeed, the best solar
energy stirling machines have efficiencies around 30 percent, which is
quite amazing given the primitivity of a mechanical machine compared
with high technology approaches such as solar cells.

271
272 8 Thermodynamics and Heat engines

8.1 Thermodynamics

In thermodynamics we discuss the behavior of a system , which is a part

of the universe, which is separated from the rest of the universe, which we
call the environment. The system may be thermal contact with the
environment, if it is allows to exchange energy with the environment, or
it may be thermally insulated, if there is no energy exchange with the
environment. An isolated system has interactions with the enviroment,
either through changes in shape of the system; through changes in the
number of particles in the system, or through changes in the energy in
the system. For an isolated system the energy, E, volume, V , and number
of particles, N , are constant.
Thermal equilibrium. Two systems are in thermal equilibrium with each
other if they are in thermal contact with each other and we have waited
for a long time. What characterizes thermal equilibrium? In thermal
equilibrium (or thermodynamic equilibrium), the two systems have the
same temperature. For two systems A and B in thermal equilibrium, we
know that TA = TB . The zeroth law of thermodynamics states that
if two systems are in equilibrium with a third system, they are also in
thermal equilibrium with each other. If the two systems A and B both
are in thermal equilibrium with system C, this means that TA = TC and
TB = TC , and therefore TA = TB . They are therefore also in thermal
equilibrium with each other.
State variables. We describe thermodynamic system with state vari-
ables , which describe the macroscopic state (macro-state) of a system.
State variables can be variables such as energy, E, pressure, p, volume,
V , entropy, S, number of particles, N , chemical potential, µ etc. There
are two types of state variables: extensive and intensive variables. Ex-
tensive variables depend on (are proportional to) the number of particles,
N , such as energy, E, entropy, S, and, of course, the number of particles,
N . Intensive varialbes do not depend on the number of particles, such
as the density, ρ, the pressure, p, the temperature, T , and the chemical
potential, µ.
Equations of state. We describe the behavior of a thermodynamic
system though equations of state, which are equations that relate
various state variables. For example, for the ideal gas, we know that there
is an equation relating pressure, P , volume, V , the number of particles,
N , and temperature, T . In general, we can write this equation of state as
8.1 Thermodynamics 273

f (P, V, N, T ) = 0 , (8.1)

where f () is some function. For the ideal gas, we know this function —
we know the equation of state:

P V − N kT = 0 . (8.2)

We will develop many similar equations of state for other systems through-
out this text.
State variables are path independent. It is important to notice that
any state variable depends on the state of the system, but not on how
the system got to this state. If we say that the energy E(S, V, N ) is a
state variable, we know that it is always the same for the same set of
values for S, V , and N , independent of how the system got there. This
is different for the work performed on a system or the thermal enegy
transfered to a system. These depend on how the system moves from
one state to another state, and are therefore not state variables.
Processes. Changes in the state of the system occur through processes,
which correspond to changes in the state variables of the system. We
can sketch such changes in various ways, as illustrated in Fig. 8.1. We
can describe the state of a system in P, V -space, as we will often do
for machines. The state of the system is then given by a pair P, V of
coordinates. (We have then typically assumed that N is constant). Or we
can describe the system in S, T space. A process is then a path from one
state to anther state in the system, as shown as lines in the sketch. The
two processes R1 and R2 which take the system from state 1 (P1 , V1 ) to
state 2 (P2 , V2 ) are shown in the figure. If we want to determine the work
done on the system in each of these processes we need to calculate the
integrals over the curves of P dV , so that the work for the two processes
are: Z Z
WR−1 = P dV , WR−2 = P dV , (8.3)
R1 R2

and these two work integrals are generally not equal. (For the given figure
they are clearly different, since the integrals correspond to the areas under
the P (V ) curves for the two processes). Also notice that a process does not
need to be described as function P (V ), but is more generally described
as a curve that may not have a one-to-one correspondence between P
and V or other state variables.
Reversible and irreversible processes. Processes may be reversible
or irreversible. A reverisible processes is just that: A process that can
274 8 Thermodynamics and Heat engines

Fig. 8.1 Sketch of two P

processes R1 and R2 in a
P -V digram. Each point in 1
the P, V space corresponds P1 R1
to a state of the system.

R2
P2
2

V1 V2 V

be reversed. This must, for example, mean that the entropy change in
the process must be zero. For a reversible process there cannot be any
friction forces, since such forces would ensure that we do not return to
the same state if we reverse a process.
Quasi-static processes. Often, we want to specify processes where the
system is always very — infinitesimally — close to equilibrium. We
call such processes quasi-static processes. All reversible processes are
necessarily quasi-static. We must always be very close to equilibrium for
a process to be reversible. However, there can be quasi-static processes
that are not reversible, for example, processes where the internal friction
in the system is non-negligible.

8.2 Engines

It is now time to address some of the consequences of thermodynamics.

Thermodynamics was developed before we had a good microscopic un-
derstanding. Instead, it was developed partly out of necessity - because
we needed the tools to develop and understand engines.
What is an engine? A heat engine is an engine that absorbs energy
and converts (some of) it into mechanical work. Only part of the thermal
energy can be converted into mechanical energy.
In general, heat and work are to different ways to transfer energy –
and to change the internal state of a system: ∆E = W + Q. Heat is the
8.2 Engines 275

thermal transfer of energy between systems in thermal contact. Work is

the mechanical transfer of energy to a system by changes in the external
parameters (volume, pressure, magnetic field, gravitational potential).
Indeed, we can argue that the conversion of heat into energy is one
of the most important processes in our society. It was the basis of the
industrial revolution where in particular the steam engine was important.
Still, steam engines and steam turbines are important and used to convert
heat from burning gas or coal into electric energy – in todays power plant.
Indeed, the use of hot water from solar heating or geological systems is
one possible way to produce renewable energy from our environment.
There is a fundamental difference between heat and work related to
the transfer of entropy. For reversible, quasi-static processes we know
that T dS = dE + pdV + µdN and dE = Q + W , therefore when W = 0
we know that Q = T dS. (Notice that the entropy does not change in a
quasistatic, adiabatic process).
How can we use this relation to understand engines? The idea of
an engine is to convert thermal energy transfered into a system into
mechanical energy (work) done by the system, and then returns to its
initial state. (Otherwise we could not continue to do this process).
Now, if we transfer thermal energy Q = T dS into the system, the
source of the thermal energy must decrease its entropy. However, if
all the thermal energy was converted into work - which may be done
adiabatically - we do not have any other change in entropy - and we
would have a net decrease in entropy. This is not possible - from the
second law of thermodynamics. We therefore also need to transfer entropy
in addition to doing work - and this is done through the transfer of some
of the energy that was not converted into work.

8.2.1 A simple piston

Let us now see how we can create a heat engine – a machine that converts
heat to mechanical work. The simplest machine we can imagine consists
of a single piston filled with a gas, as illustrated in fig. 8.2. We are going
to heat up the gas in the piston. What happens to the system? How does
the system behave?
What do we expect to happen if all we know is that we add a thermal
energy Q – heat – to the system. We expect the temperature in the
gas to increase and the pressure in the gas to increase. As the pressure
increases, we will expect the pressure to push on and move the piston,
276 8 Thermodynamics and Heat engines

Fig. 8.2 Illustration of a

piston which is supplied a
heat Q so that the piston
increases its temperature
and expands, doing work
on the environment in the
process.
TL , V1 TH , V2

V1 V2 V

so that the system expands – which subsequently will lead to a cooling

of the gas and a lowering of the pressure.
This process is a bit too unspecified. Let us be more precise in how
we describe the process. Let us assume that the gas starts at some
temperature TL (the Low temperature) and is then suddenly put in
contact with a reservoir with temperature TH (the High temperature).
The temperature in the gas will then gradually increase, until it reaches
the temperature TH , and the piston will move, changing the volume of
the system from some initial volume V1 to a final volume V2 .
We can be even more specific. Let us divide this process into two
parts. First, we place the gas in thermal contact with a reservoir with
temperature TH so that the gas increases its temperature to TH , but we
keep the volume constant. Then we allow the volume to change, while the
gas still is in thermal contact with the reservoir, so that the temperature
is constant in the subsequent process, where the volume increases from V1
to V2 . Now, the piston is expanded, and the temperature inside the piston
is high. The machine has performed work with the motion of the piston,
but only once. If we want the machine to continue to produce work, we
need to be able to repeat the process – we need the machine to return
to its initial state, so that we can add heat yet again and produce more
work. We need to have a cyclic process, corresponding to a loop in the
P, V diagram for the process. We therefore need to compress the piston
back to a volume V1 and reduce the temperature to TL . However, if we
first return the volume to V1 while the system is at the high temperature,
the work done will be exact the same as was done to expand the gas –
and the net work done will be zero. However, if we instead first cool the
gas to TL , and then push the piston back at the lower temperature and
8.2 Engines 277

pressure, we will do less work on the piston to return it to its original

position and the machine will be able to perform a net work.
Let us look at the various steps in this process in detail. The system
starts in state 1, where the temperature is TL and the volume is V1 . This
state is illustrated in fig. 8.3. The system then comes in contact with a
reservoir with temperature TH so that the temperature increases from TL
to TH without any change in volume in the system. This process takes
us from a point 1 at (P1 , V1 ) in the diagram to a point 2 at (P2 , V1 ). We
call such a process an isochor process, since it is a process without any
change in volume.

P
2

TL TH TH TH

1 2 2 3
1
T=TH
3

T=TL 4 TH TL TL TL

V1 V2 V 3 4 4 1
Fig. 8.3 Illustration of an engine as a sequence of processes in a P -V -diagram.

1 → 2: an isochor heating. The process is illustrated in fig. 8.3. The

work done by the system on the environment is zero, since the volume
does not change. This means that the change in energy is just due to
the thermal energy transferred. From the first law, we have ∆E1,2 =
−W1,2 + Q1,2 = Q1,2 , when W1,2 = 0. (Notice that we have changed the
sign of work when we discuss engines – we prefer to work with positive
work when the machine does work on the environment). We can therefore
find the thermal energy transferred (the heat) from the change in energy.
For an ideal monoatomic gas, the energy is E = (3/2)N kT , and the
change in energy is therefore ∆E1,2 = (3/2)N k(TH − TL ), which is also
the heat transferred to the system in this process.
2 → 3: an isothermal expansion. In the next step of the process, the
gas expands while keeping the temperature constant. We call processes
278 8 Thermodynamics and Heat engines

at constant temperature isothermal processes. How does this process

look in the P, V diagram? We can find how P depends on V from the
equation of state for the ideal gas: P V = N kT , and since this process
occurs at T = TH , we have P = (N kTH )/V . We can therefore find the
work done by the system on the environment in this step:
Z V2 Z V2 N kTH V2
W2,3 = P dV = dV = N kTh ln . (8.4)
V1 V1 V V1
If we perform work on the enviroment, but keep the temperature in the
gas constant, thermal energy must be transferred from the reservoir. This
is also evident from the first law, because ∆E2,3 = −W2,3 + Q2,3 . For
an ideal (monoatomic) gas, we know that E = (3/2)N kT . For a process
with constant T there is therefore no change in energy, and ∆E2,3 = 0,
hence Q2,3 = W2,3 : all the energy used to perform the work is taken from
the reservoir.
3 → 4: an isochor cooling. Now, we cool the system to the initial
temperature TL by putting the system in contact with another reservoir
at a lower temperature TL . The discussion for this process is similar to
the isochor heating: The heat Q3,4 corresponds to the change in energy
because the work is zero (since there is no change in volume). We therefore
have Q3,4 = ∆E3,4 = (3/2)N k(TL − TH ). Now, this is a negative value,
indicating that this heat is tranferred out of the system and not into the
system.
4 → 1: an isothermal compression. Finally, the system is brought back
to the beginning by compressing the system back to the starting volume
V1 . This is again an isothermal process at T = TL and we can find the
work by integrating P (V ) = N kTL /V :
Z V1 N kTL V1
W4,1 = dV = N kTL ln , (8.5)
V2 V V2
and this work corresponds to the heat Q4,1 = W4,1 . Both the work and
the heat are negative, indicating that heat is tranferred out of the system
and work is done on the system in this process.

8.2.2 Net work and efficiency of the machine

We have now described a complete cycle of the machine. The cycle is
illustrated in fig. 8.3.
8.2 Engines 279

We see that the work performed from 2 to 3 is the integral of P (V )

with respect to V , and we can interpret this as the area under the curve
of the P (V ) curve from 2 to 3. Similarly, the work performed from 4 to
1 is (minus) the area under the P (V ) curve from 4 to 1. The net work
done in the whole cycle is therefore the area enclosed by the P (V ) curve
for the entire, cyclic process.
The net work is the sum of all the works done in each subprocess.
This corresponds to the area enclosed by the curve in the P -V diagram.
You can see this by realizing that the system does positive work when
the system moves to the right along the P (V ) curve. The area under this
part of the curve is therefore the positive work done. The system does
negative work when the system moves to the left along the P (V ) curve.
If we subtract these two areas – as illustrated in fig. 8.4 – we get the net
work, which corresponds to the enclosed area.

P 2 P 2 P 2

3 3 Wnet 3
1 1 1

4 4 4
W2,3 W3,4 -W1,2 -W4,1
V V V

Fig. 8.4 Illustration of work done by the system on the environment for a complete cycle
of a machine.

The net heat is the sum of all the heats transferred in each subprocess.
The added heat corresponds to the sum of the positive heats (Q > 0).
We can apply the first law to a complete cycle. In this case, the change
in energy is zero, since the energy at the beginning and end of the cycle
must be the same since the energy is a state variable. Hence, the net
work must correspond to the net het added to the system: Wnet = Qnet
and ∆Enet = 0.
What is the efficiency, e, of this machine? It is reasonable to call the
efficiency how much work you get out for each unit of heat you put into
the system:
Wnet
e= . (8.6)
Qadd
We can calculate the efficiency of the machine we have described, by
inserting the values we have found for work and heat. We notice that the
added heat corresponds to the heat added in process 1 → 2 and 2 → 3,
280 8 Thermodynamics and Heat engines

that is Qnet = Q1,2 + Q2,3 . The efficiency is therefore

Wnet N kTH ln V1 /V0 − N kTL ln V1 /V0

e= = . (8.7)
Qnet N kTh ln V1 /V0 + N k(TH − TL )
This is a nice result. And as we shall soon see – the machine we have
described here corresponds to a Stirling machine, which is used for
example for solar array systems.

8.2.3 Limits on efficiency

We have found a particular efficiency in this case. But how good is this
machine? To answer that we need to know of efficient a machine can
be – is there a theoretical maximum in efficiency for a heat engine? The
answer is yes – and we can find this maximum.
Let us look at this more systematically. We can sketch an engine as in
figure 8.5.

Heat source Heat sink

TH Engine TL
QH QL

Fig. 8.5 Sketch of a heat engine.

This engine works between a heat source at TH and a heat sink at TL .

• The engine performs a cycle that should end in the engine coming
back to the state is started from (not microstate, of cource, but the
same macrostate).
• In this cycle a thermal energy QH is transferred into the engine.
• The engine does work W .
• And the engine transfers a thermal energy QL out of the engine and
into the heat sink.
8.2 Engines 281

• The heat source and heat sink are so large that their temperatures do
not change in this process.
We have so far defined the work done on the system as positive, but for
an engine we are usually more interested in the work done by the system.
We will therefore use the common convention QH , QL and W all are
positive. In this notation, the first law of thermodynamics is

∆E = QH − QL − W , (8.8)

where W is the net work done in a complete cycle, QH are all the heats
that are added to the system – this may occur in more than one of the
processs – and QL are all the heats that are removed from the system –
again this may occur in more than one processes. Notice that we have
changed the sign in front of the work, because the work W is now the
work done by the system on the environment. For a complete cycle, the
change in E must be zero, because E is a state variable and therefore
will be the same in the first and the last point in the cycle, because these
are the same states. The first law of thermodynamics for this system is
therefore
W = QH − QL , (8.9)
the net work W is equal to the net thermal energy (heat) added during
a complete cycle.
What is the efficiency of such as machine: It is what we get out (net
work) divided by what we put in (heat in QH ):
W
e= . (8.10)
QH
Ok. But what is the maximum efficiency we can get? It is limited by the
laws of thermodynamics. From the first law of thermodynamics, we see
that QH = W + QL . The efficiency is therefore:
QH − QL QL
e= =1− . (8.11)
QH QH
We can use the second law to relate QL and QH . From the second law
we know that
QH QL
dS = dSH + dSL = − + ≥0, (8.12)
TH TL
which gives:
282 8 Thermodynamics and Heat engines

QL TL
≥ . (8.13)
QH TH
and therefore
QL TL
e=1− ≤1− . (8.14)
QH TH
(Notice that we have here assumed that the heat is deposited to the low
temperature and taken from the high temperature). When the cycle is
reversible, we know that the equality holds, and in this case the efficiency
is:
TL
e=1− . (8.15)
TH
This is the maximum efficiency of an engine. A reversible engine of this
type is called a Carnot engine. Notice that this means that all engines
that operate between the same two temperatures (and that are reversible)
have the same efficiency – independent of how they are constructed.
However, in practice, no real engine reaches the theoretical efficiency,
and differences in the actual implementation is therefore important -
typical engines reach about 30-40% of the theoretical limit, often even
less.

Clicker example of cycles

8.3 The Carnot cycle

Let us show an example of a cycle – a heat engine – that achieves the

theoretical limit in an idealized setting. The carnot cycle consists of four
steps. It starts from a state 1 with a temperature TH . (It is in contact
with a heat bath with this temperature).
Step 1 (1 → 2) – Isothermal expansion: First, it executes an isothermal
expansion from 1 → 2. The system expands while in contact with the
heat source. This means that heat is transfered into the system. For
a gas this corresponds to the gas pushing on a piston turning a crank
to produce work. Since the system stays at the same temperature, the
engine must get thermal energy from the heat bath. (We assume that
heat is flowing into the system, even though the system and the bath are
at the same temperatures - for example by expanding the system in very
small steps, so that we are always infinitesimally close to equilibrium).
8.3 The Carnot cycle 283

We could then compress the system isothermally to return to state 1.

This would complete the cycle, but exactly the same same work would
be needed to push the piston back - we would not have done any net
work! We therefore need a few more steps - we could reduce the pressure
in the system, so that we could return the piston at a lower pressure,
which would require less work. But then we would need to increase the
pressure again before returning to the initial state. We could also lower
the pressure by doing an adiabatic expansion, and this is what we do:
Step 2 (2 → 3) – adiabatic expansion. We now isolate the system
(remove the thermal contact). The volume contines to increase, but no
heat is added. Then both the pressure and the temperature decreases.
The temperature decreases all the way to TL .
We also perform work in this part of the cycle - but now the energy
that goes into the work comes from the internal energy of the system,
∆E2,3 = W2,3 , since the system is thermally isolated.
Now, we can restore the system to its initial condition, starting at a
lower pressure. First, we perform an isothermal compression at TL .
Step 3 (3 → 4) – Isothermal compression. Work is done on the system
by pressing the piston back in. This would increase the temperature of
the system, but instead thermal energy is flowing out of the system and
into the heat sink (TL ).
Finally, we need to increase the temperature of the system back to its
original state.
Step 4 (4 → 1) – Adiabatic compression. We now need to increase the
temperature (and pressure) of the system back to its original state. We
remove the contact with the heat sink, and compress the system without
any thermal energy flowing into it - this is done until the temperature is
back at TH (and the pressure is back to its initial state).

8.3.1 Carnot cycle for ideal gas

Let us look at the Carnot cycle for an ideal gas in detail. The cyclic
process is sketched in the P (V ) diagram in fig. 8.6.
Isothermal expansion (1 → 2). The gas starts at T1 = TH and V1 . The
gas expands isothermally to V2 . First, how does this curve look like in
the diagram?
If the temperature is constant, we know that pV = N kT , and hence
we know that
284 8 Thermodynamics and Heat engines

Fig. 8.6 Illustration of the P

Carnot cycle for an ideal 1
gas.

T=TH
Q4,1=0
2
4
Q2,3=0
T=TL
3

N kT
p= (8.16)
V
for this curve.
Since the temperature is constant, and the energy U for an ideal gas
is linear in temperature, we know that dU = 0 for this process, hence
Q = W . The heat transferred is therefore given by the work:
V2 V2 1 V2
Z Z
W = pdV = N kT dV = N kT ln , (8.17)
V1 V1 V V1
(Notice, that you would have found the same by using that the heat is
Q = T dS for this process and that the entropy is S = N k ln V + f (N, U ),
which would give the same result for the heat).
We can draw the curve from 1 to 2 in a pV diagram, and notice that
the work is the area under the curve from 1 to 2.
Adiabatic expansion (2 → 3). Now, the gas is disconnected from the
bath at TH and thermally isolated - so that the work is given as the change
in internal energy of the system. The work done in this process is simple
to find from dU = Q + W , where now Q = 0, and therefore dU = W ,
where U = (3/2)N kT for a monatomic ideal gas and ((f + 3)/2)N kT
for a general ideal gas. The work is therefore simply
3
W23 = N k (TH − TL ) . (8.18)
2
What does the curve P (V ) look like for this process?
8.3 The Carnot cycle 285

For a quasistatic, adiabatic process the entropy is constant. The

entropy for an ideal gas is given by the Sackur-Tetrode equation:
V 3 4πmU 5
 
S = N k ln + ln + , (8.19)
N 2 3N h 2 2
where
3
U = N kT , (8.20)
2
and therefore
V 3 2πmkT 5
 
S = N k ln + ln + . (8.21)
N 2 h2 2

Constant S therefore means that ln V T 3/2 is constant, and therefore that

V T 3/2 is constant. We can use this to find the volume at the end of the
expansion:
2/3 2/3
TL V3 = TH V2 , (8.22)
that is
V3 /V2 = (TH /TL )3/2 . (8.23)
And we can also find the shape of the P (V ) curve - which we often
simply call an adiabat, by inserting T = pV /N k into V T 3/2 = const.,
which gives V 5/2 p3/2 is constant, or P V 5/3 is constant, and the curve is
P = C/V 5/3 , which we can sketch.
Isothermal compression (3 → 4). The gas is now in contact with the
bath at TL and compressed isothermally to the volume V4 . How should
we choose V4 ? We must choose V4 so that we can go along an adiabat
from 4 to 1, and therefore V1 , TH , and V4 , TL must be connected by an
adiabat, therefore
3/2 3/2
V4 T4 = V1 TH ⇒ V4 /V1 = (TH /TL )3/2 , (8.24)

where we again have that (TH /TL )3/2 = V3 /V2 = V4 /V1 .

What work is done on the gas? We find the work by integrating
Z 4 V3 V2
W34 = pdV = N kTL ln = N kTL ln . (8.25)
3 V4 V1
A corresponding energy is transferred into the system as heat, since the
internal energy does not change for an ideal gas during an isothermal
process. The heat is therefore
286 8 Thermodynamics and Heat engines

Q34 = QL = W34 . (8.26)

Adiabatic compression (4 → 1). Finally, the gas is isolated thermally

again, and the gas is compressed along an adiabat from 4 to 1. The work
done in this process is:
3
W = N k (TH − Tl ) , (8.27)
2
which cancels the work done during the isentropic expansion from 2 to 3.
The net work done by the whole cycle can be read from the pV -diagram.
Look at the area below the curve from 1 to 2, 2 to 3, 3 to 4 and 4 to 1,
and see that the net work corresponds to the area enclosed by the curve.
This net work is
V2
Wnet = QH − QL = N k (TH − TL ) ln . (8.28)
V1
Efficiency: We can put this back into the efficiency, and find that the
efficiency is
Wnet TH − TL TH
e= = =1− . (8.29)
QH TH TL

8.4 Carnot refrigerator

A refrigerator is a heat engine that works in reverse - it uses work to cool

a system down. It takes heat from the low temperature region – inside
the refrigerator – and combines it with work, and dumps heat at a high
temperature bath - outside the refrigerator.
In this case the efficiency depends on QL and W , but we call it the
coefficient of performance (COP):
QL
COP = . (8.30)
W
Again, we can use the first law to relate the three energies:

QH = QL + W , (8.31)

which gives
QL 1
COP = = . (8.32)
QH − QL QH /QL − 1
8.5 Realistic model for Stirling Engine 287

What are the limits for the efficiency of a refridgerator? The second law
gives us:
QH QL QH TH
≥ ⇒ ≥ . (8.33)
TH TL QL TL
(Notice that the inequality is in reverse compare to before because the
entropy is flowing in the opposite direction). This gives:
1 TL
COP ≤ = . (8.34)
TH /TL − 1 TH − TL
The coefficient may be higher than 1 - indeed it will usually be. For
example, for a kitchen fridge we may have TH = 298K and TL =
255K, which gives COP = 5.9. Notice that COP goes to infinity as the
temperature difference goes to zero! Also notice that we can make a
Carnot cycle refrigerator - but this is not how things are done in practice
since the cycle would take too long time. We will address a few more
realistic processes in the following.

8.5 Realistic model for Stirling Engine

We can gain more insight into how real engines work by developing a
realistic model for a Stirling engine. Here, we will work on a model engine
used in teh 2.670 class at MIT, inspired by a report by M. Byl in 2002.
The Stirling engine we will model is sketched in fig. 8.7. This engine
consist of two connected chambers. One chamber on the left, where the
displacement piston is working. The volume of the gas in this chamber
remains constant throughout the process, but the portion that is in
contact with the hot temperature, TH , and the portion that is in contact
with the cold region, TL , varies in time. We will assume that the whole
gas has the same temperature T (t). This means that the temperature
will vary in time, but we assume that the temperature in the gas is the
same everywhere along the machine.
Let us start with a quantitative description of this machine. We will
assume that there is a constant transport of heat from the surrounding
reservoirs into the machine. There is a transfer of heat into the machine
in the hot region and a transfer of heat out of the machine in the cold
region. We will apply a simple law for heat conduction to estimate the
rate at which thermal energy is transferred to the gas. The heat transfer
from outside to inside the piston is proportional to the difference in
288 8 Thermodynamics and Heat engines

y(t) x(t)

T T T θ(t)

TH TL TL
Fig. 8.7 Illustration of a realistic Stirling engine using air as the gas medium.

temperature, Tout − Tin , the area of contact, A, and a constant, µ, that

depends on the material properties of the cylinder wall. In the hot region,
the only contact is through the large cylinder on the left. The heat
transported through the wall and into the gas in a time interval ∆t is
then
QH = AH µ (TH − T ) ∆t , (8.35)
where µ = 100000W/m2 (calculated from the thermal conductance of
steel and the thickness of the cylinder wall), and AH depends on the
position of the displacer piston.
How do we describe the positions of the displacer piston and the
work piston? We notice that the two pistons are attached to a flywheel
– through some mechanism that is not properly sketched in the figure –
with the net effect that the pistons move a distance 2R back and forth
along their axis, where R is the radius of the flywheel at the point of
attachment. The position x(t) of the work piston can then be written

x(t) = R (1 + sin θ(t)) , (8.36)

with a proper choice of origin for θ. The position, y(t), of the displacer
cylinder should move with a phase difference of π/2, so that

y(t) = ymax (1 + cos θ(t)) , (8.37)

where ymax is the maximum extension of the piston. If we know x(t) and
y(t), we can find the surface areas AH and AL for the thermal conduction.
At the hot end, the area AH only depends on the maximum possible
area, corresponding to the ymax multiplied with the circumference of the
8.5 Realistic model for Stirling Engine 289

cylinder. We therefore find

AH (t) = AH,max (1 + cos θ(t)) . (8.38)

where AH,max = A0 is the total surface area of the part of the large
cylinder that contains gas.
For the cold end we need to include the heat transfer from the large
cylinder and the transfer for the small cylinder:

AL (t) = A0 (1 − cos θ(t)) + A1 (x) , (8.39)

where A1 (x) = A1,max x(t)/xmax is the area of contact for the small
cylinder to the right in the figure.
This means that in a short time interval ∆t, the change in (internal)
energy E of the gas in the system is

∆E = Q + W , (8.40)

where Q = QH −QL , with QH = AH µ(TH −T ) and QL = AL µ(T −TL ). In

addition, we need to include the work, W , done on the system. This work
is done by the work piston, which has moved a distance ∆x = (∆x/∆t)∆t
with a force F = (P − P0 )Ap acting on the piston from the gas, where P
is the pressure in the gas, P0 is the outside pressure, and Ap is the area
of the piston in contact with the gas. The work is therefore
∆x
W = −F ∆x = −(P − P0 )Ap ∆t . (8.41)
∆t
If we describe the position x(t) through the angle θ(t), we see that
∆x dR(1 + sin θ) dθ
= = R cos θ = Rω cos θ . (8.42)
∆t dt dt
Now, we have a complete description of ∆E:

∆E = QH −QL +W = AH µ(TH −T )∆t−AL (T −TL )∆t−P Ap Rω cos θ∆t ,

(8.43)
which gives a differential equation for E(t):
dE
= AH (t)µ(TH −T (t))−AL (t)(T (t)−TL )−(P (t)−P0 )Ap Rω(t) cos θ(t) .
dt
(8.44)
In addition, we need to determine the motion of the flywheel. The
flywheel has a moment of inertial I and is affected by a torque, τ , due
290 8 Thermodynamics and Heat engines

to the force on the work piston. In addition, we will include a frictional

effect due to friction in the flywheel as well as viscous damping due to
gas motion in the displacement cylinder. The total torque on the flywheel
is then
τ = F R cos θ − bω = (P − P0 )Ap R cos θ − bω , (8.45)
where b is a viscous coefficient describing the effective damping in the
system. This gives us the equation of motion for θ:

d2 θ dθ
2
= F R cos θ − b . (8.46)
dt dt
We now have two equations, describing θ and E. In addition, we need
to be able to calculate T , V and P in the system. We can find V from
geometric considerations. The volume in the displacement cylinder is
constant, Vc , but the volume in the work cylinder depends on x:

V (t) = Vc + Ap x , (8.47)

where Ap x is the volume of the gas in the work piston. We can now find
the temperature through
E
E = CV T ⇒ T = , (8.48)
CV
where CV = cV m is the heat capacity for the gas in the cylinder. And
we find the pressure using the ideal gas law:
N kT 2E
P = = , (8.49)
V 5V
where we have used that E = (5/2)N kT for the gas at realistic tempera-
tures.
This completes the set of equations needed to determine the behavior
of the machine. We can now model the machine by integrating the
equations of motion for θ and the time development of E to find the
behavior of the machine in a P (V ) diagram.
First, we need to know the values of the parameters for a realistic
machine. We will use TH = 600K, TL = 300K, R = 1.25cm (radius
of flywheel), A0 = 40cm2 (surface area of active area of displacement
cylinder), Ap = 1.9cm2 (surface area of piston), A1,max /xmax = 4.9cm
(perimeter of the work piston cylinder), Vc = 40cm3 (volume of gas in
displacement cylinder), I = 4kg· cm2 , b = 0.7·10−3 N/s (damping constant
representing viscous processes), P0 = 105 Pa (trykk i omgivelsene), CV =
8.5 Realistic model for Stirling Engine 291

717J/kg/K (heat capacity for the gas), µ = 100000W/m2 (heat transfer

coefficient for the cylinder wall), and ρ = (mass density for the gas).
The motion of the cylinder is solved by the following implementation,
where we have used Euler-Cromer for the time integration with a time
step of ∆t =.
The resulting behavior is illustrated in fig. 8.8. The simulation is for
5 seconds, but only the first second is shown for x(t) and T (t), because
the behavior is completely regular after 1 second. We see that both the
position and the temperature has a short transient behavior before they
start having a periodic behavior. We can illustrate the behavior in the
periodic part of the motion by plotting P (T ) and P (V ) for the last 2.5
seconds of the simulation. We see that the behavior of the machine indeed
is a closed loop in the P − V diagram. However, the actual path in the
P − V diagram looks very different from the idealized case we used to
model the behavior of the system.
We can estimate the efficiency of this – more realistic – Stirling engine:
e = Wnet /Qin , by measuring the heat flow into the engine during a cycle
and the net work done by the machine during a cycle. The net work
corresponds to the area enclosed by the curve in the P − V diagram, but
in the model it is easier to calculate the net work done by the piston
and the sum of the heat, QH , transferred into the system. Unfortunately,
this results in a very low efficiency. (Something may be wrong with the
calculation – can you find it?).
292 8 Thermodynamics and Heat engines

2.5 600

2
500

x [ c m]

T [K]
1.5
400
1

300
0.5

0 200
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
t [ s] t [ s]

2 2

1.8 1.8

1.6 1.6
P [ at m]

P [ at m]
1.4 1.4

1.2 1.2

1 1

0.8 0.8
40 41 42 43 44 45 2 3 4 5 6
3 T [K]
V [cm ] x 10
8

Fig. 8.8 Plots of the temporal behavior of the model Stirling engine.

8.6 Additional aspects of heat engines and

thermodynamics (not releavnt for exam)

8.6.1 Path dependence of Heat and Work

We need to be careful with how we write differences and differentials.
We have been careful not to write dW and dQ, since W and Q are
not state functions.
S, E, etc are state functions and therefore form pure differentials.
If we go a complete loop, we will come back to the same value.
The integral over a closed loop is therefore zero.
But not so for W and Q. The integral around a complete loop of W
is the net work done. And the integral around a complete loop of Q is
the same - since they are related by dE = W + Q, and the integral of
dE around a compete loop is zero.
8.6 Additional aspects of heat engines and thermodynamics (not releavnt for exam) 293

8.6.2 Heat and work at constant temperature and constant

pressure
Isothermal work. In a reversible, isothermal process T is constant and
T dS = d(T S), therefore Q = T dS = d(T S) and

W = dE − Q = dE − d(T S) = dF (8.50)

In this case Helmholtz free energy is a proper energy.

Isobaric work. Many processes occur at constant pressure - such as open
processes in the atmosphere.
If we boil a liquid at constant pressure, the piston above a boiling
system may move a distance dV , and the work due to this movement
would be −pdV = −d(pV ). This work would be included in the total
work done by the system. But if we compress the system, we get this
work for “free” from the atmosphere, and if we expand the system we
have to perform this work against the atmosphere, but this work cannot
be used for anything else. It is therefore often useful to subtract −d(pV )
from the work, and only get the effective work performed on the system.
We define this effective work as

W 0 = W + d(pV ) = dE + d(pV ) − Q = dH − Q , (8.51)

where we have introduced the entalpy H = U + pV . This plays the same

role at constant P as U plays in processes at constant V .
The term pV is required to displace the surrounding atmosphere to
get room for the system.
(We assume that there are other types of work in addition to this
pressure work).
Two classes of constant pressure processes:
Processes with no effective work Q = dH (no other work than the work
needed to expand the system against the given pressure). Evaporation of
a liquid - the heat of evaporation is the enthalpy difference between the
liquid and gas phase.
Processes at constant T and constant p. Then Q = T dS = d(T S) and
the effective work is

W 0 = dF + d(pV ) = dG , (8.52)

where we have introduced G = F + pV = E + pV − T S, the Gibbs

free energy. The effective work performed at constant pressure and
temperature is equal to the change in Gibbs free energy. Useful for
294 8 Thermodynamics and Heat engines

chemical reactions where the volume changes at constant pressure and

temperature.

8.6.3 Chemical Work

(Kittel, p 250)

8.6.4 Joule-Thompson effect

(May use for exercise instead - process on van der Waals gas).

8.7 Summary

8.8 Exercises

Exercise 8.1: Adiabatic process

The change of state of a system is called adiabatic if no heat is exchanged
with the environment along the reversible path of the process from state
1 to state 2. Let us address the behavior of 1 mol of an ideal gas. The
molar heat capacity at constant volume is cV,m . Assume that the gas is
going through an adiabatic change ∆T and ∆V .
a) Show that for the adiabatic change
∆T ∆V
cV,m +R =0, (8.53)
T V
where R = NA k is the molar gass-constant, where NA is Avogardro’s
number.
b) Integrate (8.53) of demonstrate that the following expression is valid
for any pair of states 1 and 2 connected by an adiabatic path:

T1 V1α = T2 V2α , where α = R/cV,m . (8.54)

c) Show that the result in (8.54) alternatively may be expressed as

p1 V1γ = p2 V2γ or T1 pβ1 = T2 pβ2 . (8.55)

where β = (1 − γ)/γ and γ = cp,m /cV,m = 1 + R/cV,m .

8.8 Exercises 295

d) Draw adiabatic curves (pV γ = const.) and isotherms in a p-V -diagram.

Assume that the gas is diatomic with cV,m = (5/2)R.

Exercise 8.2: Adiabatic compressibility

The compressibility of a system is defined as
1 ∂V
 
κx = − , (8.56)
V ∂p x

for a process where x is a constant. For an ideal gas we know that

pV γ = const. for an adiabatic process, where γ = cp,m /cV,m .
a) We often use the term isentropic to describe an adiabatic process.
Why?
b) Show that for an ideal gas the isentropic compressibility, κS , is smaller
than the isothermal compressibility, κT , by a factor γ.
The speed of sound in a gas is given as v = (κS ρ)−1/2 , where ρ is the
mass density of the gas.
c) Using the results from above, show that the speed is sound can be
written as v = γRT /M , where M is the average molecular weight of
p

the gas.
d) Calculate the numerical value for the speed of sound in air at 300 K
by using M ' 29 g/mol.
e) The speed of sound is closely connected to the average velocity, vavg ,
of molecules in a gas. Try to express vavg by R, T , and M and compare
with the speed of sound.
f) Discuss the difference between a sound wave that (hypothetically)
propagated as an isothermal process and one that propagated as an
isentropic process. How can you change the parameters of gas in order
to get a more isothermal process during sound propagation?

Exercise 8.3: Crossing adiabatic curves

a) Prove that two different adiabatic curves for an ideal gas cannot cross
each other in a p-V -diagram.
Hint. Assume that they cross each other, complete a cycle with an
isotherm, and show that such a process is in violation of the second law.
296 8 Thermodynamics and Heat engines

Exercise 8.4: Work of a gas

We study a He-gas in a cylinder with a piston. Initially, the system is in
equilibrium at T = 2 K, p = (1/36) atm, V = 600 cm3 . You may assume
that the internal energy of the gas is zero at this point.
a) The temperature is increased to T = 288 K at constant volume. If
you assume that He behaves as a monatomic ideal gas, how much heat
is adsorbed? And what is the internal energy of the gas? May all this
energy be extracted in the form of heat? May all this energy be used to
perform mechanical work?
b) The gas expands isentropically until the temperature is 2 K. How
much work is performed and what is the internal energy of the gas?
Check that this process does not violate the various formulations of the
second law of thermodynamics.
c) The gas is compressed isothermally back to its original volume. How
much work is done in this cycle og how much energy in the form of heat
is adsorbed? What is the efficiency of the cycle?

Exercise 8.5: String engine

We know that a rubber band can be described by S(x) = b − kB ax2 , and
K = −2akB T x (Hooke’s law), where x is the elongation of the band, a
and b are constants, T is the temperature, K is the tension in the band
(the force), and S is the entropy.
a) Draw a Carnot-cycle for the rubber-band-engine in a K-x-diagram.
(The cycle consists of two isothermal and two isentropic processes).
b) Find the work done in the process 1 → 2 and 3 → 4.
c) Find the heat adsorbed in process 1 → 2 and 3 → 4.
d) Show that the efficiency of the machine is given as the Carnot
efficiency.

Exercise 8.6: Three-point cycle

Fig. 8.9 shows a cycle for an imaginary machine that operates on an ideal
gas. You can assume that the heat capacities CV and Cp are constant
throughout the process.
8.9 Projects 297

Fig. 8.9 Illustration of a p

cyclic process in the p − V
B
plane. The process from B p1
to C is adiabatic.

Adiabatic

A
p2 C

V2 V1 V

a) Show that the efficiency of this machine is

(V1 /V2 ) − 1
e=1−γ , (8.57)
(p1 /p2 ) − 1
where γ = Cp /CV .

8.9 Projects
 Chemical potential and Gibbs
Distribution 9

We have now developed a microscopic understanding and the thermo-

dynamic theory for systems that are closed to all transport: isolated
systems with constant E, V , and N , and for systems that are only open
to thermal transport, systems with constant T , V , and N . We started
with an isolated, closed system, and then opened the possibility for energy
transport.
For the microcanonical system, we found that the link between the
microscopic and the macroscopic follows from S = k ln Ω(E, V, N ) and
we found that the multiplicity, and hence entropy, of such a system must
increase and equilibrium is obtained when the multiplicity, and hence
entropy, is at its maximum.
For the canonical system, we found that the microscopic decrip-
tion is given by the Boltzmann factors and the partition function,
Z = Z(T, V, N ), and that the macroscopic state is found through
F = −kT ln Z. The thermodynamic principle that detemines the be-
havior of this system is that the system develops towards a minimum in
Helmholtz free energy, F , and that the equilibrium is characterized by a
minimum in F .
There are, however, many systems of interest that are open to particle
transport. For example, how can we determine the equilibrium state of an
ideal gas in the atmosphere — where the particles are free to flow from
one region to another; the equilibrium state of electrons in a voltage field;
or the equilibrium state of a gas-water mixture? If we pour water into an
empty container of a given volume and keep the temperature constant,
how much of the water will be water, how much will be ice and how much

299
300 9 Chemical potential and Gibbs Distribution

will be vapor? And what principle determines this equilibrium? Similar

questions could be asked about chemical reaction, such as the reaction
between hydrogen and oxygen to form water — what is the equilibrium
concentrations of each component?
To answer such questions, we need to extend our microscopic and
thermodynamic framework to also address systems that are open to
transport. In this chapter we will introduce the basic principles needed to
address the statistical mechanics and thermodynamics of open systems —
systems where the particle number may change.

9.1 Chemical potential

We say that system is open to transport if particles can be transported in

or out through the boundaries of the system. In that case, the number of
particles is no longer a constant in the system, and we call such systems
diffusional open or open to diffusion.
In order to address the thermodynamics of systems open to diffusion,
we need thermodynamic principles similar to what we have found previ-
ously for the microcanonical and the canonical systems. We have seen
that:
• If two systems have the same temperature there is no net energy flow
between them.
• If two systems have the same pressure, there is no net change of
volumes
• If two systems have the same X, there is no net flow of particles
between them
We need to identify this X. Let us do this by expanding gradually from
systems we already know how to treat.

9.1.1 Chemical potential in the canonical system

Let us start with a system S that is in thermal contact with a large
reservoir with temperature T , but otherwise of constant particle number
and constant volume: This system is a canonical system. We divide
this system into two parts A and B that are in diffusional contact. For
example, if the system is a gas, we simply divide the container into two
parts and call one part A and the other part B as illustrated in Fig. 9.1.
9.1 Chemical potential 301

Particles can pass from A to B and the other way. This is similar to
the molecular dynamics model for the ideal gas where half the system
was empty. (Similar, but not identical, since our molecular dynamics
model was for constant energy and not constant temperature!). We can
characterize this sysytem by N the total number of particles in the
system, where NA + NB = N is constant, but NA and NB may vary.

Fig. 9.1 Illustration of a

system S in thermal contact
with a large reservoir so TR
that the temperatures
TR = TS are the same A B
in the two systems. The
system is divided into two
parts, A and B, that are in
diffusive contact so that the TS NA + NB = N
total number of particles is
conserved, NA + NB = N .

Helmholtz free energy for combined system. This system is a canoni-

cal system, and we know that it will develop towards minium Helmholtz
free energy and that the equilibrium is characterized by a minimum of
the Helmholtz free energy. What is the Helmholtz free energy for this
system? It is the sum of the energy for the components: F = FA + FB .
What happens if we look at a small change in this system: a small transfer
of dNA particles from B to A:

NA → NA + dNA (9.1)
NB → NB + dNB = NB − dNA (9.2)

In this case, there is no change in the temperature or volume of systems

A and B, only in the number of particles. Since the change in the number
of particles is small, we can find the change in Helmholtz free energy by
a Taylor expansion:
∂FA ∂FB
   
dF = d(FA + FB ) = dFA + dFB = dNA + dNB .
∂NA V,T ∂NB V,T
(9.3)
Since N = NA + NB is constant, we know that dNA = −dNB :
302 9 Chemical potential and Gibbs Distribution

∂FA ∂FB
   
dF = dNA − dNA
∂NA V,T ∂NB V,T
! . (9.4)
∂FA ∂FB
   
= − dNA
∂NA V,T ∂NB V,T

Helmholtz free energy is minimal in equilibrium. In equilibrium, F is

minimal, and this change must be zero:
∂FA ∂FB
   
= . (9.5)
∂NA V,T ∂NB V,T

This looks very similar to what we found for the canonical system. This
is the quantity related to particle transport which must be the same for
the two systems to be in diffusive equilibrium. This is the X we were
looking for!
Definition of chemical potential. We call this quantity the chemical
potential. Indeed, we will use this as the definition of the chemical
potential:

Chemical potential:
∂F
 
µ(T, V, N ) = , (9.6)
∂N V,T

or
µ(T, V, N ) = F (V, T, N + 1) − F (V, T, N ) . (9.7)

This provides us with a condition for diffusive equilibrium:

A condition for diffusive equilibrium between two systems A and

B (at constant T and V ) is the the chemical potential in the two
systems are the same: then

µA = µB . (9.8)

Chemical potential as a discrete difference. It may seem strange to

define a derivative in N , which is essentially a discrete quantity: the
number of particles. We could therefore instead have used the definition

µ = F (N + 1, V, T ) − F (N, V, T ) , (9.9)
9.1 Chemical potential 303

which is the same as the definition above for large N . This definition
also emphasizes that:

The chemical potential can be interpreted as the change in Helmholtz

free energy when we add one particle to the system, keeping the
temperature and volume constant.

Transport occurs from high to low chemical potential. We also know

how a system that it out of equilibrium develops: Helmholtz free energy
must decrease. The change in Helmholtz free energy is we transfer an
amount dNA from part A to part B is
∂FA ∂FB
   
dF = dNA − dNA (9.10)
∂NA V,T ∂NB V,T
= µA dNA − µB dNA = (µA − µB ) dNA . (9.11)

If the system is out of (diffusive) equilibrium, we know that the system

will change in such a way that the change in NA will result in a lower
value of F as the system approaches equilibrium, hence dF ≤ 0. We
therefore see that if µB > µA then dNA > 0 for dF to be negative, hence
the flux is from B to A when µB is larger than µA . We therefore conclude
that

Flux and chemical potential: Particles flow from large to small

chemical potential.

This will occur until the chemical potential in the two systems are equal
and a further change in particles does not change Helmholtz free energy.
This effects makes intuitive sense since taking a particle from a part with
large chemical potential and transferring it to a part with lower chemical
potential will reduce the total Helmholtz free energy of the system and
therefore move the system towards equilibrium.

9.1.2 Thermodynamic identity for diffusive systems

We can now extend the thermodynamic identity to also include changes
in N . We already found that dF = −SdT − pdV + XdN for Helmholtz
free energy. Previously we did not have a good intuition for X, but
304 9 Chemical potential and Gibbs Distribution

now we have defined this X as the chemical potential. The potential for
Helmholtz free energy is therefore

dF = −SdT − pdV + µdN , (9.12)

and we can use this to find the thermodynamic identity

T dS = dE + pdV − µdN . (9.13)

9.1.3 Chemical potential for multi-particle systems

If several chemical species are present, then each species will have its
own chemical potential:

Chemical potential of species j in a multi-species system:

!
∂F
µj (T, V, N ) = , (9.14)
∂Nj T,V,Nk

where all the other Nk , k 6= j, are kept constant in the derivative.

9.1.4 Example: Chemical potential of the Einstein crystal

We found the Helmholtz free energy for the Einstein crystal with energy
level spacing ∆ to be:

1
 N
ZN = , (9.15)
1 − e−∆/kT
and  
F = −kT ln ZN = N kT ln 1 − e−∆/kT , (9.16)
which gives
∂F
   
µ= = kT ln 1 − e−∆/kT . (9.17)
∂N

9.1.5 Example: Chemical potential of the Ideal gas

For an ideal gas, the partition function can be written as:
9.1 Chemical potential 305

1 N
ZN = Z , Z = Ztrans Zvib Zrot , (9.18)
N! 1

3/2
where Ztrans = nQ (T ) V and nQ (T ) = 2πmkT /h2 for a three-
dimensional ideal gas. For a monatomic gas Zvib = Zrot = 1, and
Z1 = Ztrans . Helmholtz free energy is therefore:

F = −kT ln ZN = −kT (N ln Z1 − ln N !) . (9.19)

We use Stirling’s approximation ln N ! ' N ln N − N , getting:

d d
ln N ! = (N ln N − N ) = ln N + 1 − 1 = ln N , (9.20)
dN dN
which gives
d
 
F = −kT ln Z1 − ln N ! = −kT (ln Z1 − ln N ) = kT ln(N/Z1 ) .
dN
(9.21)
We insert Z1 = Ztrans = nQ (T ) V , getting:
! !
N n
µ = kT ln = kT ln . (9.22)
V nQ (T ) nQ (T )

(We get the same result if we use µ = F (N ) − F (N − 1) as the definition

for the chemical potential, and then we do not even have to use Stirling’s
approximation!).
We see that the chemical potential increases with the number of
particles per volume, n, which is consistent with our general observation
above: Particles flow from parts with high n to systems with low n. (Notice
that particle flow therefore tends to smooth out the particle density).
We notice that for classical concentrations, that is when n/nQ  1, the
chemical potential of an ideal gas is always negative.

9.1.6 Potential energy and the chemical potential

We can better understand the chemical potential by looking at a system
with a difference in potential energy. Let us look at two systems A and
B at the same temperature that may exchange particles, but the two
systems are not yet in diffusive equilibrium. The two systems may for
example be two ideal gases with different densitites, nB > nA , so that
306 9 Chemical potential and Gibbs Distribution

µB > µA . If the two systems are placed in contact, particles will flow
from B to A.
How can we stop that flow? By changing the chemical potential, for
example by lifting one of the systems up in the gravitational field (if the
particles have masses). How does a change in potential energy change the
chemical potential? We can find out be addressing Helmholtz free energy
and then taking the derivative to find the chemical potential. Helmholtz
free energy consists of two parts, F = E − T S. We expect the energy
E of the system to depend on the position in a gravitational field. If
we lift an ideal gas from height z = 0 to a height z, this corresponds
to adding a potential energy u = mgz to all the particles. The energy
therefore depends on the height, E(z) = E(0) + mgz. However, if we
add a potential energy to the system, we only add a constant to all the
energy levels for all the particles, we do not change the distribtion of
energy levels. Therefore, we do not expect the entropy of the system to
change, S(0) = S(z). Helmholtz free energy is thererfore

F (z) = E(z)−T S(z) = E(0)+N mgz−T S(0) = F (0)+N mgz = F (0)+N u ,

(9.23)
where u = mgz is the potential energy per particle. The chemical potential
is therefore
∂F ∂F (0)
   
µ(z) = = + u = µ(0) + u . (9.24)
∂N V,T ∂N V,T

The effect of adding a potential energy to system A — such as by lifting

it in a graviational field — is to increase the chemical potential by the
potential energy per particle, u.
It is common to divide the chemical potential into two parts: The
external chemical potential is the potential energy per particle in an
external field, and the internal chemical potential energy is the chemical
potential that would be present without the external field.

Internal and external chemical potential:

µTOT = µint + µext . (9.25)

Where the external chemical potential, µext , is the potential energy

per particle in an external field, and the internal chemical potential,
µint , is the chemical potential without the external field.
9.1 Chemical potential 307

If µB > µA , we can therefore stop the flux of particles from system B

to system A by increasing the potential energy per particle in system
A by u = µB − µA . Then the chemical potential in the two systems are
equal, and there is no net flow of particles. We could use this to measure
the chemical potential: We apply a potential difference and determine at
what potential difference net particle flow stops.

9.1.7 Example: Barometric pressure formula

Fig. 9.2 illustrates a situation where a part B of an ideal gas was lifted a
height z in the gravity field — how must the density change for part A
to be in equilibrium with part B after this displacement? How can we
use the theory we haved developed for the chemical potential to predict
the distribution of densities in the atmosphere?

Fig. 9.2 Illustration of two z z

parts of an ideal gas, where B

part B has been displaced
a distance z in the vertical
direction. z

A μA B μB A

0 0

Let us for simplicity assume that the atmosphere has the same tem-
perature T at all heights. (This is generally not true). The chemical
potential of gas at a height z is then

µ(z) = µgas + µgrav = kT ln(n(z)/nQ (T )) + mgz , (9.26)

where mgz is the potential energy of a gas particle at height z above

the surface. In equilibrium, the chemical potential must be the same
everywhere — otherwise there would be a flux from regions with high
chemical potential to regions with low chemical potential. The chemical
potential at z = 0 must therefore be the same as the chemical potential
at z:

µ(z) = kT ln(n(z)/nQ ) + mgz = kT ln(n(0)/nQ ) = µ(0) , (9.27)

308 9 Chemical potential and Gibbs Distribution

which gives
n(z) = n(0)e−mgz/kT . (9.28)
What does that mean for the pressure of an ideal gas? If the temperature
does not depend on height, the pressure is
N
pV = N kT ⇒ p = kT = nkT , (9.29)
V
and
p(z) = p(0) e−mgz/kT = p(0) e−z/zc , (9.30)
where zc = kT /mg is a characteristic height for the decay of the density.
Notice that the result depends on the mass of the gas molecules!
For N2 the mass of a molecule is 28 which gives zc = 8.5 km. Lighter
molecules will extend further up - and will mostly have escaped from the
atmosphere. Notice that T is not really constant, and n(z) is generally
more complicated. And also notice that various gases have different m,
and will therefore have different values of zc , which means that the
composition of the air will change with distance z as well.

9.1.8 Example: Batteries

ams 7: Add this.

9.1.9 Example: Helmholtz free energy for system in potential

field
We study a system with where each particle is described by the states i
with energies i . We know that we can describe the thermodynamics of
this system by first finding the partition function and then Helmholtz
free energy for a system of N particles. How does the partition function
and Helmholtz free energy change if we add a potential energy 0 to each
particle?
The states, i0 , of the system after we add the potential energy can be
numbered the same way as the states i of the original system, but their
energies are changes to i + N 0 , where the factor N is included because
there are N particles, each with a potential energy, 0 , so that the total
potential energy added to the system is E0 = N 0 . The partition function,
Z 0 , for the system after adding the potential energy is therefore:
9.1 Chemical potential 309

Z0 = e−(i +N 0 )/kT = e−N 0 /kT e−i /kT = e−N 0 /kT Z , (9.31)

X X

i i

where Z is the partition function for the system without the potential
energy. We find Helmholtz free energy from the partition function
 
F 0 = −kT ln Z 0 = −kT ln e−N 0 /kT Z (9.32)
= −kT ln e−N 0 /kT − kT ln Z = N 0 + F , (9.33)

where F is Helmholtz free energy for the system without the potential
energy. The chemical potential of the system is then
∂F 0
 
µ =
0
= 0 + µ , (9.34)
∂N V,T

where µ is the chemical potential without the potential energy. Adding

a potential energy per particle therefore results in adding the potential
energy to the chemical potential.

9.1.10 Example: Modeling potential gradients

Fig. 9.3 shows a system with a variation in the chemical potential. The
system stretches from x = 0 to x = L, and within the system the chemical
potential is µ(x). Is this system in equilibrium? What potential energy
must you add in order to make the system be in equilibrium?

Fig. 9.3 Plot of the chem-

ical potential µ(x) as a μ
function of x in a system.

L x

Clearly, the system is not in equilibrium. Because if it were in equi-

librium the chemical potential would be the same everywhere. Since
the chemical potential is not the same, there will be a flow of particles
from regions with high chemical potential to regions with small chemical
potential.
310 9 Chemical potential and Gibbs Distribution

We can make this argument more quantitative. Let us assume that

the whole system is at a constant temperature T , and that diffusion is
possible inside the system, but not across the boundaries at x = 0 and
x = L. The whole system therefore has T , V , and N constant, and its
equilibrium is determined from the minimum of Helmholtz free energy.
We can find Helmholtz free energy of the whole system by dividing
the system into small parts from x to x + ∆x, where Helmholtz free
energy of this small part is F (x). If each part is independent, that is, if
the energy levels in each part are independent, then each part will have
its separate sum over states in the partition function and Helmholtz free
energy will be the sum of the free energies for each part. The total free
energy is then
Ftot = F (x) . (9.35)
X

We can use this to discuss how the chemical potential must develop as the
system approaches equilibrium. If the chemical potential is not constant,
as sketched in the figure, we can redure the total Helmholtz free energy
by moving a particle from a position with high chemical potential to a
position with low chemical potential. When we remove a particle from
a point, the decrease in F is the chemical potential at that point, and,
similarly, when we add the same particle to another point, the increase
in F is the chemical potential. If we remove a particle from a region
with high chemical potential and place it in a region with low chemical
potential, we therefore remove more from Helmholtz free energy than we
put back, hence the free energy is reduced. This process will continue
until the chemical potential is equal everywhere.
Another alternative to obtain equilibrium is to add an external poten-
tial, so that the total chemical potential is constant. For example, we
can add the external potential u(x) given as u(x) = u0 − µ0 (x), where
u0 is a constant and µ0 (x) is the initial chemical potential. The total
chemical potential is then:

µ(x) = µ0 (x) + u(x) = µ0 (x) + u0 − µ0 (x) = u0 , (9.36)

which is a constant. In practice, we may not add any potential, but only
simple potentials such as a linear potential in the case of an electrical
potential difference for charged particles or a gravitional field for particles
with non-negligible masses.
9.2 Phase equilibrium at constant volume 311

Clicker question on µ(x) as introduction.

9.2 Phase equilibrium at constant volume

We started out by wanting to study diffusive equilibrium because we were

interested in processes such as the equilibrium between different phases of
a material and chemical reactions that transform one material to another.
Now, we have introduced a new concept: the chemical potential. Let
us see how we can use this concept to address the equilibrium between
phases of a material.
The basic phases of matter are solid, liquid and gas (and plasma). A
phase equilibrium is an equilibrium between two or several phases of a
material. If I hold a cup of water in the lecture room, and place a lid on
the cup, I have a system with water molecules in a given volume, at a
given temperature (the temperature in the room is the temperature of
the heat reservoir), and a given number of molecules, particles, of water.
Inside the cup, there will be a given number of water molecules. What
determines how much of the water is a solid, how much is a liquid and
how much is a gas? If the cup contained water at room temperature, we
expect the closed cup to contain some water and some water vapor. How
can we determine the number of particles in each phase and how can we
determine the state of the phases, such as the pressure in the vapor?

9.2.1 Equilibrium between an ideal gas and an ideal crystal

To address this situation, we will discuss phase equilibrium in a simplified
model in which we can understand all the processes in detail. We will
address equilibrium between a solid, a crytal of a given particle type, and
a gas consisting of the same particles, such as an equilibrium between ice
and water vapor at sub-zero temperatures. We will model this system,
a cup with water molecules, as a system with a given temperature, T ,
volume, V , and number of particles, N . The system consists of two
subsystems: a gas system (subindex g) of Ng gas particles in the gas
phase; and a solid system (subindex s) of Ns solid particles. The gas will
be modelled as an ideal gas and the solid as an Einstein crystal. The
system is illustrated in fig. 9.4 and the symbols used to describe the
system are listed in Table ??.
312 9 Chemical potential and Gibbs Distribution

Fig. 9.4 Illustration of a

system with phase equilib-
heat reservoir, T
rium between a solid and
a gas phase of the same
particles. gas

μg
solid μs

Table 9.1 Symbols used to describe phase equilibrium between an Einstein crystal and
an ideal gas of the same particles.
Symbol Description

Ng Number of gas particles

Ns Number of solid particles
Vg Volume of gas
Vs Volume of solid
ng Density of gas
ns Density of solid

Approach. Our plan is the find the chemical potential for each of the
two subsystems. In equilibrium, the chemical potentials must be equal,
otherwise there would be a flow of particles from the part of the system
with high chemical potential towards the part with a low chemical
potential. We can calculate the chemical potential from Helmholtz free
energy, which is turn can be found from the partition function for each
of the two systems.
Helmholtz free energy for an ideal gas. We have previously found
that Helmholtz free energy for a gas in a volume Vg is

2πmkT
! 2
nQ

Fg = −Ng kT ln +1 , nQ (T ) = , (9.37)
ng h2

where ng = Ng /Vg is the density of the gas.

Helmholtz free energy for an Einstein crystal. The Helmholtz free
energy for the Einstein crystal is
 
Fs = Nosc kT ln 1 − e−∆/kT . (9.38)
9.2 Phase equilibrium at constant volume 313

Here, we have written Nosc because this is the energy is for a given
number of oscillators and not a given number of particles. For a three-
dimensional system, each particle can oscillate in three different, indepen-
dent directions (x, y, and z), and the number of oscillators is therefore
Nosc = 3Ns , where Ns is the number of particles in the solid. Helmholtz
free energy for an Einstein crystal with Ns particles is therefore:
 
Fs = 3Ns kT ln 1 − e−∆/kT . (9.39)

Comparing energy levels for Helmholtz free energy. To compare the

energy for the gas and the crystal, the reference level of the energies must
be the same. For a particle in a gas, we have assumed that the particle
has zero energy when at rest. Similarly, for the crystal, we have assumed
that the lowest energy level is at zero energy. However, we know that
the particles in a crystal are bound to the crystal, and that we need to
add an energy to such a particle to pull the particle from the crystal and
make it part of the gas. The energy added corresponds to the binding
energy, 0 , of a particle in the crystal. This means that a particle in
the lowest energy level in the crystal has an energy −0 compared with
an energy at rest in the gas, we must therefore add a term −Ns 0 to
Helmholtz free energy for the crystal to account for the binding energies.
 
Fs = 3Ns kT ln 1 − e−∆/kT − Ns 0 . (9.40)

Finding the chemical potentials. The chemical potentials for both the
gas and the solid can now be found from Fg and Fc :
!
∂Fg ng
µg (Vg , T, Ng ) = = kT ln , (9.41)
∂Ng Vg ,T
nQ (T )

and
∂Fs
   
µs (Vs , T, Ns ) = = 3kT ln 1 − e−∆/kT − 0 . (9.42)
∂Ns Vs ,T

In equilibrium we assume that the two chemical potentials are the same.
ng  
µg (Vg , T, Ng ) = kT ln = µs (Vs , T, Ns ) = 3kT ln 1 − e−∆/kT −0 .
nQ (T )
(9.43)
We solve for ng , getting:
314 9 Chemical potential and Gibbs Distribution
 
ng = nQ (T )e3 1 − e−∆/kT e−0 /kT . (9.44)

From the ideal gas law for the gas, we know that pVg = Ng kT and
therefore ng = Ng /Vg = p/kT . We insert this to get an equation for the
pressure in the gas:
 
p(T ) = kT nQ (T )e3 1 − e−∆/kT e−0 /kT . (9.45)

9.2.2 Vapor pressure

This result tells us that inside the closed cup, there will be both a solid
and a gas, and the pressure of the gas will be given by p(T ). We call
this the vapor pressure of the system and the curve p(T ) is called the
coexistence curve. We have illustrated p(T ) in Fig. 9.5. What does this
curve tell us? In this form, it describes the state inside the closed system.

Fig. 9.5 Plot of the phase p

coexistence curve p(T ) for
the ideal gas - Einstein
crystal model system.
solid μg>μs

μs>μg gas

Let us see if we can use the model and the chemical potentials to
understand what would happen if we instead of doing the experiment
at constant volume, did the experiment at constant pressure: Instead of
doing the experiment with a closed cup, we do the experiment inside a
piston, where the outside of the piston has a pressure corresponding to
the pressure in the lecture room. What would happen if we start the
9.3 Gibbs Free Energy 315

system on the p(T ) curve described by the model, and then increase the
pressure?
In the model, it is only the chemical potential of the gas that depends
on the pressure. And the chemical potential of the gas will increase if
we increase the pressure and decrease if we decrease the pressure. At
p(T ) the chemical potential in the gas and the solid is the same. If we
increase the pressure from this state, the chemical potential in the gas
will increase while the chemical potential in the solid remains constant
(this is in our model, not necessarily in the actual, physical system). As
a result particles will diffuse from the gas and into the solid – until all
the particles are in the solid. This means that the region above the p(T )
curve in the p − T diagram corresponds to a region where the whole
system is a solid. If we decrease the pressure, the chemical potential
in the gas will decrease while the chemical potential in the solid will
remain constant, hence all the particles will diffuse to the gas. The region
below p(T ) in the p − T diagram hence corresponds to a gas. The phase
coexistence curve is therefore the border between the solid and the gas
phases!
Now, there are a few problems with this argument. Primarily, the
problem is that we have described the system in terms of T , V , and
N , but the system we have sketched with the piston is not a system
with constant V , but rather a system with constant p. We therefore
need to introduce methods to address systems with constant T , p, N ,
including methods to find the equilibrium states in such systems. We
will therefore introduce the (T, p, N ) system and the corresponding free
energy — Gibbs free energy — before we return to the model system to
complete the discussion of the phase coexistence curve.

9.3 Gibbs Free Energy

We need to develop a theory to address systems where T, p, N are constant,

corresponding to a system in a piston in a large reservoir with a given
temperature and pressure as illustrated in Fig. 9.6. This corresponds
to a laboratory situation or a natural situation where systems are in a
surrounding gas or liquid and therefore in equilibrium with an external
pressure. How can we develop such a theory?
316 9 Chemical potential and Gibbs Distribution

Fig. 9.6 Illustration of a

system in thermal and
mechanical contact with a pR ,TR
huge reservoir with pressure
pR and temperature TR .

ΔE ΔV
pS ,TS

9.3.1 Definition of Gibbs free energy

Changing variables from V to p. We have already found that for the
micro-canonical system, we characterize the energy by E, the internal
energy, and the development of the system toward equilibrium is deter-
mined by the second law of thermodynamics, ∆S ≥ 0. Based on this
we developed Helmholtz free energy through a Legendre transformation:
We exchanged the variable S with the conjugate variable T (conjugate
because (∂E/∂S)V,N = T ). This was done by introducing

∂E
 
F =E− = E − TS . (9.46)
∂S V,N

Now, we want to introduce a new free energy with the natural variables T ,
p, N . We know that F has the variables T , V , and N and the differential

dF = −SdT − pdV + µdN . (9.47)

We would now like to replace pdV by some term which depends on dp. We
see that we can do this by adding pV to F , because the new differential
of d(F + pV ) will then not contain pdV , but instead a term V dp. If we
introduce G = F + pV then the differential of G is

dG = d (F + pV )
= dF + pdV + V dp
(9.48)
= −SdT − pdV + µdN + pdV + V dp
= −SdT + V dp + µdN .
9.3 Gibbs Free Energy 317

We see that G indeed has T , p, and N as natural variables.

Gibbs free energy is defined as

G(T, p, N ) = E − T S + pV , (9.49)

and its differential is

dG = −S dT + V dp + µ dN . (9.50)

We can therefore find the temperature, the volume and the chemical
potential from Gibbs free energy:
∂G ∂G ∂G
     
S=− ,V = ,µ = (9.51)
∂T p,N ∂p T,N ∂N T,p

We leave it as an exercise for the reader to show what new Maxwell

relations can be derived using Gibbs free energy.
Legendre transform method. We could also have found this by applying
the Legendre transform. We could define G by replacing V by p in
F (T, V, N ):
∂F
 
G=F −V = F − V (−p) = F + pV , (9.52)
∂V T,N

which is the same as we found above.

9.3.2 Minimum Free Energy principle

We can also demonstrate a minimization principle for Gibbs free energy
similar to what we found for Helmholtz free energy. We assume that our
system is contained in a large reservoir with pressure pR and temperature
TR . The total system, consisting of the system and the reservoir, is
isolated, and therefore a microcanonical system. We therefore know
that the entropy of the total system must increase and be maximum in
equilibrium.
The entropy of the total system consists of the entropies of the system,
SS , and the entropy of the reservoir, SR , ST OT = SR +SS . Let us now look
at a small change of the energy and the volume of the system, ∆ES and
∆VS . Since the total energy and volume is conserved, ET OT = ES + ER
318 9 Chemical potential and Gibbs Distribution

and VT OT = VS + VR , we know that ∆ES = −∆ER , and ∆VS = −∆VR :

We change the energy in the system by transferring that energy to the
reservoir and if we increase the volume in the system, the volume of
the reservoir decreases by the same amount. For any such change in
the internal distribution of energy or volume, the total entropy must
increase:
∆ST OT = ∆SR + ∆SS ≥ 0 . (9.53)
How can we express the change in entropy of the reservoir in terms of
changes in the system? First, we notice that the change in energy of the
reservoir must be due to the heat transferred to the reservoir or the work
done on the reservoir (the first law of thermodynamics):

∆ER = QR + WR = TR ∆SR − pR ∆VR . (9.54)

where we have used that the heat transferred is QR = TR ∆SR . Notice

the sign of the work done on the reservoir. This means that the change
in entropy of the reservoir is

TR ∆SR = ∆ER + pR ∆VR . (9.55)

Since the total energy and the total volume is conserved, we have that

∆ER = −∆ES , ∆VR = −∆VS , (9.56)

which we insert into (9.55), getting

TR ∆SR = −∆ES − pR ∆VS . (9.57)

We insert this into (9.53), getting the condition

TR ∆ST OT = TR ∆SR +TR ∆SS = TR ∆SR −∆ES −pR ∆VS = ∆ (TR SS − ES − pR VS ) ≥ 0

(9.58)
In equilibrium TR = TS , and we recognize this as Gibbs free energy,
GS = ES − TS SS + pS VS :

TR ∆ST OT = −∆ (ES − TS SS + pS VS ) ≥ 0 ⇒ ∆GS ≤ 0 . (9.59)

| {z }
GS

This demonstrates that Gibbs free energy always decreases and is mini-
mum in equilbrium.
9.3 Gibbs Free Energy 319

For a (T, p, N ) system Gibbs free energy is G = F + pV = E −

T S + pV . Gibbs free energy decreases as the system approaches
equilibrium:
∆G ≤ 0 (9.60)
and is minimum in equilibrium.

This principle will turn out to be a vary powerful theoretical and practical
tool as we will see below.

9.3.3 Gibbs free energy and the chemical potential

Gibbs free energy is an extensive quantity, so that Gibbs free energy for
a system of N particles is

G(T, p, N ) = N g(T, p) , (9.61)

where g(T, p) is the Gibbs free energy per particlem, which only depends
on the intensive variables T and p.
Notice that since T and p are intensive, this means that
G(T, p, N ) ∂G
 
g(T, p) = = = µ(T, p) . (9.62)
N ∂N T,p

Notice that this is true for G, but it is not true for F !

9.3.4 Gibbs free energy for a system of several species

If a system consists of k different species of particles with chemical
potentials µj (p, T ), j = 1, 2, . . . , k, then Gibbs free energy for the system
is
G= Nj µj (p, T ) . (9.63)
X

The thermodynamic identity for a k-species system becomes

k
T dS = dE + p dV − (9.64)
X
µj dNj
j=1

and the differential for G becomes

320 9 Chemical potential and Gibbs Distribution

k
dG = −T dS + V dp + (9.65)
X
µj dNj .
j=1

We can also for this expression for Gibbs free energy demonstrate that
Gibbs free energy for a system of k species is also minimum in equilibrium.
Notice that the species can be molecules or atoms of different types,
but it can also be atoms or molecules of the same type, but in different
phases, where the particles will have different chemical potentials.

9.4 Phase coexistence at constant pressure

We have now developed the basic tools needed to discuss the behavior of
a system at constant T , p, and N . This corresponds to the case where
we have a system of a given number of particles inside a box, but where
the volume of the box is varying while the pressure outside (and inside)
the box is constant. This is what happens in a system with a piston.
Now, we can address the same question as we did above for the T , V ,
and N system: What is the state of a system of particles, which may be
in several phases, inside a piston for a given set of T , p, and N values?

9.4.1 Phase coexistence in a simple model

We address the same simplified system as we did before. In our simplified
model the system consists of N particles, of which Ng are in a gas phase
and Ns are in a crystal phase. If the system is in a large reservoir with
constant temperature and pressure and the number of particles does not
change in the system, the equilibrium state of the system is determined
by the minimum of the total Gibbs free energy of the entire system,
consisting of both gas and crystal particles.
Equilibrium condition. The equilibrium of this system can be found
either as a minimum of the Gibbs free energy, or as the state when the
chemical potential in the two phases are the same. The total Gibbs free
energy for this system can be written as

GT OT = Ns gs (p, T ) + Ng gg (p, T ) , (9.66)

where gs (p, T ) = µs (p, T ) is the Gibbs free energy per particle in the
solid phase, that is for a particle in the crystal, and gg (p, T ) = µg (p, T )
9.4 Phase coexistence at constant pressure 321

is Gibbs free energy per particle in the gas phase. We can therefore look
for that state with the minimum of GT OT , as this will be the equilibrium
state. Alternatively, we realize that the minimum of GT OT occurs when
dGT OT = 0, and the differential of Gibbs free energy for a multi-species
system is
dG = −SdT + V dp + (9.67)
X
µj dNj .
j

For a system at constant T and p, we know that dT = dp = 0. We are

then left with

dG = µj dNj = µg dNg + µs dNs . (9.68)

X

Since the total number of particles is conserved, N = Ns + Nc = const.,

we know that dNs = −dNg , and therefore

dG = (µg − µs ) dNg = 0 ⇒ µg (p, T ) = µs (p, T ) . (9.69)

This means that for a system with two species, where the total number
of particles is conserved, the chemical potential of the two species must
be identical in equilibrium. That is, we can use the same principle as we
found for a system at constant (T, V, N ).
Finding the equilibrium. Our plan is therefore to find Gibbs free energy
for each of the species, and use the expressions for Gibbs free energy to find
the chemical potentials, and from this find the equilibrium concentration
of gas and solid.
Gibbs free energy for the solid. For the crystal, we find Gibbs free
energy from
Gs (T, p, Ns ) = Fs (T, Vs , Ns ) + pVs , (9.70)
where Vs is the volume of the crystal. The volume does not appear in
Fs , but we will need an expression for Vs for the pVs term. We therefore
make a small addition to the ideal crystal (Einstein crystal) model: We
assume that there is a volume v0 associated with each crystal particle,
so that the volume of the crystal is Vs = Ns v0 . We insert the result
 
Fs (T, Vs , Ns ) = 3Ns kT ln 1 − e−∆ − Ns 0 (9.71)

from our calculations above and find

 
Gs (T, p, Ns ) = 3Ns kT ln 1 − e−∆ − Ns 0 + pNs v0 . (9.72)
322 9 Chemical potential and Gibbs Distribution

We can then calculate gs (T, p) = µs (T, p):

Gs  
gs (T, p) = µs (T, p) = = 3kT ln 1 − e−∆ − 0 + pv0 . (9.73)
Ns
We see that the chemical potential for the crystal is modified by the term
pv0 compared with the expression we have above for the canonical T , V ,
N system.
Gibbs free energy for the gas. Similarly, we find Gibbs free energy for
the gas:
!
nQ
Gg (T, p, Ng ) = Fg (T, Vg , Ng ) + pVg = −Ng kT ln + 1 + pVg .
ng
(9.74)
We apply the ideal gas law, pVg = Ng kT and p = (Ng /Vg )kT = ng kT ,
which gives ng = p/(kT ):
nQ kT p
 
Gg (T, p, Ng ) = −Ng kT ln + 1 + Ng kT = Ng kT ln .
p nQ kT
(9.75)
which gives
p
gg (p, T ) = µg (p, T ) = kT ln . (9.76)
nQ kT
(This is exactly the same result for µg as we would get if we inserted
ng = p/(kT ) into the result for µg (T, V ) from our previous discussion.)
Condition for phase equilibrium. Phase equilibrium occurs for the pe ,Te
pair for which µc (pe , Te ) = µg (pe , Te ), which is given by
p  
µg = kTe ln = µc = 3kTe 1 − e−∆/kTe − 0 + pe v0 . (9.77)
nQ (Te ) kTe
Unfortunately, this equation is not simple to solve, since pe occurs both
in the logarithm and directly in the equation. However, if pe v0 is much
smaller than kT , we may neglect the pe v0 term and the resulting pe (Te )
curve will be the same as we found above.
Phase equilibrium from Gibbs free energy. Represeting the phase
equilibrium problem using Gibbs free energy provides us with a richer
description and understanding of the phase equilibrium. The system
consists of Ns of the solid species and Ng of the gas species. The total
Gibbs free energy is therefore

GT OT = Gs + Gg = Ns gs (p, T ) + Ng gg (p, T ) , (9.78)

9.4 Phase coexistence at constant pressure 323

Fig. 9.7 illustrates gs and gg as a function of T for a given value of p.

We see that since gg is proportional to kT (except for a logarithmic
dependency on T which is weaker), and gs approximately is proportional
to 3kT , the slope of gs with T is higher than that of gg . This implies
that gs < gg for small T and gs > gg for large T . We find that gs = gg at
T = Te , that is at the phase co-existence curve.
From (9.78) we therefore see that when gs < gg , we get a minimum
in the total Gibbs free energy is all the particles are in the solid phase,
whereas when gg s < gg , we get a minimum in the total Gibbs free energy
is all the particles are in the gas phase. This means that
( (
N when T < Te (gs < gg ) 0 when T < Te (gs < gg )
Ns = , Ng =
0 when T > Te (gs > gg ) N when T > Te (gs > gg )
(9.79)
as illustrated in Fig. 9.7.

Fig. 9.7 Plot of the Gibbs g gs

free energies per particles
for the gas and the crystal.
The dashed regions indicate
where the other phase has gg
a lower Gibbs free energy,
the solid lines show the
minimum of the total gas
Gibbs free energy and
hence the actual behavior
of the system.
solid
T

Phase coexistence and Gibbs free energy. For this system, the pe (Te )
represents a singular region. It is only when the pressure and temperature
are exactly on this curve that there is co-existence between the two phases
in the system. If the system starts at the co-existence curve, and we
increase the temperature in the system, then the chemical potential in the
gas will be lower than the chemical potential of the crystal (or, similarly,
the Gibbs free energy per particle will be lower in the gas than in the
crystal), and the system will only contain gas particles. If we lower the
temperature from pe (Te ), all the particles will be in the crystal phase.
Behavior at the phase coexistence curve. Let us look at what happens
as we cross the coexistence curve. We start from a temperature that is
324 9 Chemical potential and Gibbs Distribution

lower that Te (for a given p), and then gradually (and slowly) increase the
temperature. As long as T < Te , the Gibbs free energy of the crystal will
be lower than the Gibbs free energy of the solid, and all the particles will
be in the solid phase until we reach T = Te . At this point, the number
of particles in the crystal and the gas phases are not known — they are
not determined by the minimum in Gibbs free energy! Any choise of Ns
and Ng (consistent with the condition that Ng + Ns = N ) will give the
same Gibbs free energy

Gtot = Ng gg + Ns gs , (9.80)

because gg = gs when p = pe (Te ). So what determines Ns and Ng in this

case? The history! How we have come to this state. If we came to the
state from T < Te then Ng = N and Ns = 0.
Changing the relative amount of gas and solid. How can we change
how much of the system is in the gas state and how much is in the solid
state (that is, how can we change Ng and Ns ) while the system is at
T = Te ? We can do this by traferring thermal energy — heat — into
the system. This will not lead to a change in temperature in the system,
since the system will remain on the phase coexistence curve as long as
there are particles in both the crystal and gas phases in the system.
Instead, the heat will take particles from the crystal phase and into the
gas phase. We realize that this does require energy, because the particles
in the crystal are bound with a binding energy 0 . But there are other
contributions as well.
Energy for phase transformation. What is the heat needed to trans-
fer one particle (atom or molecule) from the solid to the gas — as
illustratrated in Fig. 9.8? We can analyze this using the first law of ther-
modynamics. The change in the energy of the system when Ng → Ng + 1
and Ns → Ns − 1 is
∆E = Q + W , (9.81)
where the work done on the system by the environment is W = −p∆V .
The heat transferred into the system therefore goes into work done by
the system on the environment because the volume changes and a change
in internal energy in the system:

Q = ∆E + p∆V . (9.82)

Change in internal energy per particle. We use the high-temperature

approximation for the internal energy of the solid and the gas: Eg '
9.4 Phase coexistence at constant pressure 325

Fig. 9.8 Illustration of the p0 p0

change in the system as
one particle moves from vg-v0
the solid to the gas phase. gas

v0

solid

(3/2)Ng kT , for a monatomic gass, and Es ' 3Ns kT − Ns 0 . The energy

per particle is therefore eg = (3/2)kT and es = 3kT − 0 . We notice
that without the binding energy, we would decrease the energy if we
transferred one particle from the solid to the gas, since in thermal
equilibrium each particle has kT /2 per degree of freedom of motion, and
there are more degrees of freedom for a particle in a solid than in a
(monatomic) gass. This difference becomes smaller for a diatomic gas
where there are additional degrees of freedom for rotation and vibration.
The change in energy from transferring one particle from the solid to the
gas is therefore
3 3
∆e = eg − es = kT − (3kT − ) = 0 − kT . (9.83)
2 2
This shows that if the binding energy is small (0 ' 0), then the internal
energy decreseas when a particle is transferred from the solid to the gas.
Work done per particle. What is the work done by the system −w =
p∆v per particle that is transferred from the solid to the gas? The volume
of the gas is Vg and the volume of the solid is Vs = Ns v0 , where v0 is the
volume per particle in the solid. The work per particle is therefore
!
Vg Vs
w=p − . (9.84)
Ng Ns

We insert the ideal gas law for the ideal gas: pVg = Ng kT , which gives
Vg /Ng = kT /p. We insert this and Vs = Ns v0 , getting:
!
Ng kT Ns v0
w=p − = kT − pv0 . (9.85)
p Ng Ns
326 9 Chemical potential and Gibbs Distribution

Heat per particle for phase transformation. The heat added in order
to transfer a single particle from the solid to the gas phase is therefore
3 1
q = e + w = 0 − kT + kT − pv0 = 0 − kT − pv0 . (9.86)
2 2
Usually, we will assume that pv0  kT and we can therefore neglect the
last term.
This means that in order to transfer one particle from the solid to the
gas, we need to add a thermal energy q ' 0 − kT /2 to the system. This
seems reasonable, since we need to add the binding energy 0 , we need
to perform work against the external pressure in order to increase the
volume of the gas, and we gain energy due to the difference in degrees of
freedom for a solid and a gas particle.
Latent heat in simplified model. We call the heat added to the system
in order to transform one particle from solid to gas, the latent heat of the
system. The latent heat is usually given as the heat needed to transfer
one mole of the substance from one phase to another, and we use the
symbol L for the latent heat of the system or the symbol ` for the latent
heat per particle:

The latent heat in the simplified phase coexistence model is

` ' 0 − kT /2 . (9.87)

9.4.2 General phase coexistence

For the simplified model, we only address two phases: a solid and an
gas. In general systems, there are typically three phases — gas, liquid
and solid — and the phase diagram becomes more complicated. Fig. 9.9
illustrates the more general situation. (Let us for now assume that this
plot is found from an experimental or a computational study). Phase
coexistence occurs along the lines separating the various phases. How
does the system go from one phase to another? Let us look at an isotherm
in detail: The red dashed line in Fig. 9.9 corresponds to the line T = Te .
At high pressures, p > pe , the system is in the liquid phase, and the
whole system is a liquid. If we gradually decrease the pressure in the
system, keeping the temperature constant, we move along the T = Te
line. Eventually, the system will reach the pressure p = pe corresponding
9.4 Phase coexistence at constant pressure 327

to phase coexistence between liquid and gas. As we reach p = pe the

whole system will be in liquid form. Based on our experience from the
simplified system, we expect that we must add energy in the form of heat
in order to bring particles from the liquid and into the gas state. This
occurs at constant pressure, but the volume of the system will increase as
the system changes from pure liquid, through liquid and gas coexistence,
to pure gas.

Fig. 9.9 Phase plot in P -T p T=const

space.
LIQUID Critical point

Melting Evaporation
pe p=const
M B

SOLID

Triple point
GAS
Sublimation

Te T

Isothermal process in a p-V -diagram. How do we expect this isothermal

process to look like in a p-V -diagram? We know that for an ideal gas,
an isotherm has the shape p = (N kT )/V . We also expect p(V ) to fall
rapidly with volume in the liquid state, but we do not yet know exactly
how the functional shape is. We have sketched the behavior p(V ) along
an isotherm in Fig. 9.10. If we start from a high pressure and slowly
decrease the pressure at constant temperature, we argued above that
when the system reaches p = pe , the whole system will be a liquid, but as
we add heat to the system to transfer particles from the liquid to the gas
state, the volume will increase, while the pressure remains the same. This
corresponds to a horisontal line in the p-V -diagram as shown in Fig. 9.10.
Along this horisontal line there is both liquid and gas in the system, but
the amount of gas increases (and the amount of liquid decreases) as the
volume increases, until all the liquid has been transformed into a gas,
and the system will behave as a gas, following the isothermal behavior,
p = (N kT )/V behavior of an ideal gas.
Alternative description. We could also think of this process as a process
where we gradually changed the volume: We pull on the piston slowly and
observe the internal state of the system. This will result in the behavior
328 9 Chemical potential and Gibbs Distribution

Fig. 9.10 Illustration of p

an isothermal process Isotherm, T = const
crossing the liquid-gas
phase boundary plotted in
p-V space.

LIQ
UI
D
LIQUID+GAS

GA
S

LIQUID LIQUID+GAS GAS

from Fig. 9.10. Heat will also in this case flow into the system from the
thermal bath in order to keep the system at the same temperature while
the system is expanding and particles are transferred from liquid to solid.
Notice that the pressure is a function of the volume, but that the volume
is not a function of pressure: There are many possible volumes for the
pressure pe .
Melting, evaporation, and sublimation. It is common to call the di-
agram in Fig. 9.9 a phase diagram, since it divides the p-T space into
regions that belong to different phases.
The various phase-transformation processes illustrated in Fig. 9.9
have different names. We call the transformation from a solid to a
liquid, melting. We call the tranformation from a solid directly to a
gas, sublimation, and we call the transformation from a liquid to a gas
evaporation.
Triple-point and critial point. We have also marked two other special
points in the p(T ) phase diagram in Fig. 9.9. The triple-point is the
temperature and pressure at which we have coexistence between all three
phases. We will return to methods to address the equilibrium near this
point below. The triple-point for water at a pressure of one atmosphere
is very close to 0◦ C, and this point is indeed used to define the kelvin
scale.
9.4 Phase coexistence at constant pressure 329

The critial point is the point in p, V, T space beyond which gases and
liquids are called supercritical, and there is no longer any phase transition
between a liquid and a gas. Various values for the critical temperatures
are listed in Table ??.

Table 9.2 Critical temperatures for various substances.

Material Critical T

O2 154.3 K
H2 O 647.1 K
CO2 304.2 K
N2 126.0 K
H2 033.2 K

9.4.3 General phase coexistence and Gibbs free energy

The equilibrium state of a system at constant T , p, and N is determined
by the minimum of the Gibbs free energy of the system. For a system
with three phases: gas (g), liquid (l), and solid (s), the total Gibbs free
energy can be written as

GT OT = Gg (Ng , p, T ) + Gl (Nl , p, T ) + Gs (Ns , p, T )

(9.88)
= Ng gg (p, T ) + Nl gl (p, T ) + Ns gs (p, T ) ,

where gg , gl , and gs are the Gibbs free energy per particles for the gas,
liquid and solid phases, and Ng + Nl + Ns = N are the particles in the
gas, liquid and solid phases respectively.
In general, the three free energies will have different dependencies of
p and T . In Fig. 9.11 we have sketched the behavior of the Gibbs free
energies for a sample system. Just like for the simplified model, we know
that the system will be in the phase with the minimum Gibbs free energy
per particle. Phase coexistence occurs where the Gibbs free energies
per particle are equal: The evaporation transition (e) from gas to liquid
occurs for the pe , Te values where gg (pe , Te ) = gl (pe , Te ); and the melting
(m) transition from liquid to solid occurs where gl (pm , Tm ) = gs (pm , Tm ).
This provides us with a method to calculate the phase coexistence curves
if we know the Gibbs free energies for the various phases in the system.
In the following we use this to find the shape of the p(T ) curve without
knowing all the details of the underlying system.
330 9 Chemical potential and Gibbs Distribution

Fig. 9.11 Illustration of g gl gg

the gibbs free energies per gs
particle as a function of
temperature at constant
pressure for the solid (s),
liquid (l), and gas (g)
phases. The melting and
boiling points, TM and TB ,
are illustrated.

Solid is stable Liquid is stable Gas is stable

TM(p) TB(p) T

9.4.4 Clausius-Clapeyron’s equation

Let us find a general expression for the coexistence curve, pe (Te ), using
the liquid-gas transition as an example. Coexistence occur when the
Gibbs free energy per particle in the liquid and gas phases are the same:

gl (p, T ) = gg (p, T ) . (9.89)

This means that if we start at a point p0 , T0 on the coexistence curve,

and make a small change dT to T0 + dT we must also make a change dp
in the pressure to still remain on the curve, so that (9.89) is satisfied for
the original point, (p0 , T0 ), and for the new point, (p0 + dp, T0 + dT ):

gl (p0 , T0 ) = gg (p0 , T0 ) (9.90)

gl (p0 + dp, T0 + dT ) = gg (p0 + dp, T0 + dT ) (9.91)

We subtract the two equations, getting

gl (p0 + dp, T0 + dT ) − gl (p0 , T0 ) = gg (p0 + dp, T0 + dT ) − gg (p0 , T0 ) .

| {z } | {z }
=dgl =dgg
(9.92)
For small changes dp and dT we can approximate gl (p0 + dp, T0 + dT )
by its Taylor expansion around p0 , T0 :
∂gl ∂gl
   
gl (p0 + dp, T0 + dT ) ' gl (p0 , T0 ) + dp + dT , (9.93)
∂p T ∂T p

and similarly for gg (p, T ). We recognize the partial derivatives as

9.4 Phase coexistence at constant pressure 331

∂gl ∂gl
   
= vl , = −sl . (9.94)
∂p T ∂T p

where the small symbols indicate that the volume and entropy are per
particle. We insert this into (9.92), getting

dgl = vl dp − sl dT = dgg = vg dp − sg dT . (9.95)

We rearrange to find an equation for dp/dT :

(vl − vg )dp = (sl − sg )dT , (9.96)

and the coexistence curve p(T ) is therefore determined by the equation:

dp sg − sl
 
= . (9.97)
dT along curve vg − vl

Interpretation of ∆s and ∆v. How can we interpret the two differences

∆s = sg − sl and ∆v = vg − vl ? We see that ∆s is the increase in entropy
when we transfer one molecule from the liquid to the gas phase. Similarly,
∆v is the change in volume when we transfer one molecule from the
liquid to the gas. We can relate ∆s to the heat added to the system:

Q = T ∆s = T (sg − sl ) = ` , (9.98)

which is called the latent heat of evaporation.

The latent heat of evaporation is

` = T (sg − sl ) , (9.99)

where sg is the entropy per particle in the gas and sl is the entropy
per particle in the liquid phase.

Latent heat. It is common to describe the latent heat by the latent heat
for the evaporation of one mole and not one particle. The molar latent
heat is L = T (sg − sl ), where sg and sl are the entropies of one mole of
gas and liquid phase of the molecules. (If we use molar values for the
entropy or latent heat, we also need to use molar values for the volumes
v.) Notice that the latent heat is generally dependent on temperature:
L = L(T ). We can now rewrite (9.97) as
332 9 Chemical potential and Gibbs Distribution

Clausius-Clapeyron’s equation:
dp ∆s T ∆s L(T )
= = = , (9.100)
dT ∆v T ∆v T ∆v
where L(T ) is the (molar) latent heat for the transition and ∆v is
the (molar) volume change.

Simplification for liquid-gas transition. For the liquid-gas transition,

we can simplify the expression because vg  vs , so that ∆v = vg −vs ' vg .
In addition, the gas phase can be described approximately by the ideal
gas law, so that vg = p/kT (or if we use molar values, vg = p/RT ,
where R = NA k, where NA is Avogadro’s number). The equation for the
coexistence curve then becomes
dp L(T ) L(T )p
= ' , (9.101)
dT T ∆v kT 2
and
dp L(t)dT
= . (9.102)
p kT 2
Given an expression for L(T ) we can integrate this equation to find p(T ).
Coexistence curve for a constant latent heat. If the latent heat
L(T ) ' L0 is approximately constant over a range of temperatures
T , we can find an analytical solution to the phase coexistence curve:
p dp T L(T )dT T dT
Z Z Z
= ' L0 , (9.103)
p0 p T0 kT 2 T0 kT 2
which gives
L0 L0
ln p − ln p0 = − + , (9.104)
kT kT0
and L0
p(T ) = p00 e− kT . (9.105)
For many systems of practical importance this equation is sufficiently
precise. For example, Fig. XXX shows that ln p as a function of 1/T
for water is approximately a straight line over a temperature range
corresponding to the range from ice to the critical point of water.
9.5 Van der Waals fluid model 333

9.4.5 Example: Transition from graphite to diamond

9.4.6 Latent heat and enthalpy
We can use themodynamic tables that list enthalpies to find the latent
heat for a transition. The latent heat correspond to the difference of the
enthalpy, H = E + pV , between two phases when the process occurs at
constant pressure p.
This is simple to demonstrate, because along the coexistence curve we
know that µl = µg . The theromdynamic identity is therefore

T dS = dE + pdV − (µg − µl ) dN , (9.106)

but since µg = µl the last term is zero. At constant pressure, the latent
heat is the heat T dS transferred, which is

L = T dS = dE + pdV = dH = dE + pdV + V dp = Hg − Hl . (9.107)

|{z}
=0

At constant pressure the latent heat for a transition from phase g

to phase l can be related to the difference in enthalpy for the two
phases:
L = Hg − Hl . (9.108)

Experimentally, we can find the values of H by integrating the heat

capacity at constant pressure:
dS ∂U ∂V ∂H
       
Cp = T = +p = . (9.109)
dT p ∂T p ∂T p ∂T p

and Z
H= Cp dT . (9.110)

9.5 Van der Waals fluid model

We have so far used the ideal gas model as our canonical model for a
gas. But the model is limited — it does not contain any interactions
between the gas particles. How well does this model really describe a
real gas or a real liquid? We can test this by studying the behavior of a
334 9 Chemical potential and Gibbs Distribution

real, although simulated, fluid using the molecular dynamics simulation

method we introduced previously.

9.5.1 Molecular dynamics simulation of Lennard-Jones gas

In a molecular simulation we can include any type of interactions between
the molecules in the system. We could include a model of real water
(see App. XX ), but most of the properties of gases and fluids are well
captured by a simpler model, such as the Lennard-Jones model used to
describe the Argon system.
Lennard-Jones model system. In the Lennard-Jones model, the inter-
atomic potential is
 12  6 !
σ σ
ULJ (r) = 4 − . (9.111)
r r

This potential has a strong repulsive interaction at short ranges, described

by a potential with a r−12 dependency, and a weak attractive interaction
at long distances, described by a dipole-dipole interaction with a r−6
dependency. The Lennard-Jones system therefore contain both repulsive
and attractive interactions. The Lennard-Jones system can describe
all the three phases of the Argon system: a solid, a liquid, and a gas,
depending on the pressure, density and temperature in the system.
Molecular simulation of phase equilibrium. We can use molecular
simulations to study a phase equilibrium directly. We can model a
microcanonical system with constant E, V , and N . In this system, if
the energy is low enough, there will always be an equilibrium between a
gas and a liquid phase. If we start the simulation from a regular pattern
of atoms with random velocities, the system will quickly coalesce and
coarsen: Droplets of liquid will spontaneously nucleate and grow, and
the regions between the droplets will be filled with a gas with much
lower density than the solid. Fig. 9.12 shows a sequence of snap-shots
from a simulation with the following initialization file for the LAMMPS
simulator:
# 3d Lennard-Jones coalesence
units lj
atom_style atomic
lattice fcc 0.2
region box block 0 20 0 20 0 20
create_box 1 box
9.5 Van der Waals fluid model 335

create_atoms 1 box
mass 1 1.0
velocity all create 0.1 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nvt temp 0.6 0.6 1.0
dump id all atom 100 vdw-coalesce01.lammpstrj
thermo 100
run 20000

Fig. 9.12 Snapshots from simulations of a Lennard-Jones system at a low temperature,

T = 0.1T ∗ , show coalescence and phase coexistence between a liquid and a gas phase.

In this case, the simulated system is at a constant temperature. This

is specified by the fix command. Here, we provide a fix of the (N, V, T )
type, specified by the option nvt. The remaining parts of the line
fix 1 all nvt temp 0.6 0.6 1.0

specifies what region the fix is applied to. The three numbers 0.6, 0.6,
1.0 tells that the fix should change the temperature from T0 = 0.6 to
T1 = 0.6 over a simulation time corresponding to ∆t = 1.0, and then
continue with a fix at T1 = 0.6 from that time onward. This simulation
is therefore of a canonical system. Notice that it is also possible to model
systems at a constant pressure using special techniques that change the
volume of the system to keep the pressure (approximately) constant.
The resulting snapshots demonstrate that the model is able to model
the phase coexistence between a liquid and a gas — and indeed also
the non-equilibrium transition towards a stable equilibrium with a given
density of both fluid and gas.
Measuring the equation of state in a molecular model. We can use the
molecular dynamics model also to calculate the equation of state — the
relation between the pressure, volume, particle number and temperature
for a real gas with attractive as well as repulsive interactions. To do this,
we need to vary the temperature, the number of particles, and the volume
336 9 Chemical potential and Gibbs Distribution

and measure the pressure in the system. Here, we choose to model a

microcanonical system, hence the temperature will fluctuate somewhat
during the simulation. We keep the number of particles constant, but
vary the temperature and the system size. We write a short script to
generate input scripts for the molecular simulation program, and run the
simulator from within the script. The resulting data is analyzed for each
simulation, and then data from all simulations are collected using the
follow script:
from pylab import *
import re
pressarr = array([]) # Store pressures
volarr = array([]) # Store volumes
Tarr = array([]) # Store temperatures
myvelocities = array([3.0,3.5,4.0,4.5,5.0,5.5,6.0])
myvolumes = array([0.010, 0.020, 0.040, 0.080])
for ivel in range(0,size(myvelocities)):
for ivol in range(0,size(myvolumes)):
# Change the word mydensity to myvolumes[ivol]
infile = open("in.vdwstat30",’r’)
intext = infile.read()
infile.close()
replacestring = "%f" % (myvolumes[ivol])
intext2=intext.replace(’mydensity’,replacestring)
# Change the word myvelocity to myvelocities[ivel]
replacestring = "%f" % (myvelocities[ivel])
intext3=intext2.replace(’myvelocity’,replacestring)
infile = open("in.tmp",’w’)
infile.write(intext3)
infile.close()
# Run the simulator
print "Executing lammps < in.tmp"
os.system("lammps < in.tmp") # Run lammps
# Extract data from trajectory of simulation
d = dump("tmp.lammpstrj") # Read sim states
tmp_time,simbox,atoms,bonds,tris,lines = d.viz(0)
dx = simbox[3]-simbox[0]
dy = simbox[4]-simbox[1]
vol = dx*dy # Volume of box
# t = d.time(), n = size(t)
# Read statistics calculated in simulation
l = logfile("log.lammps")
# Find pressure averaged over all timesteps
press = average(l.get("Press"))
temp = average(l.get("Temp"))
# Store calculated values in arrays
pressarr = append(pressarr,press)
volarr = append(volarr,vol)
Tarr = append(Tarr,temp)
# Plot the results
plot(Tarr,pressarr,’o’),xlabel("T"),ylabel("P"),show()
9.5 Van der Waals fluid model 337

and the file used by the program is

# 3d Lennard-Jones gas
units lj
dimension 3
atom_style atomic
lattice fcc mydensity
region box block 0 10 0 10 0 10
create_box 1 box
create_atoms 1 box
mass 1 1.0
velocity all create myvelocity 87287
pair_style lj/cut 2.5
pair_coeff 1 1 1.0 1.0 2.5
neighbor 0.3 bin
neigh_modify every 20 delay 0 check no
fix 1 all nve
dump 1 all custom 100 tmp.lammpstrj id type x y z vx vy vz
thermo 100
run 5000

Comparison with ideal gas law. These molecular simulations allow us

to test the precision of the ideal gas law, and maybe we can also learn
how we can modify the ideal gas law to better represent the material
in the simulations. The ideal gas law states that pV = N kT , which
gives us a functional form for p = N kT /V . We can measure and plot
p as a function of T for a given number N of atoms and for a given
volume V and test the accuracy of the model. Fig. 9.13 shows the
measured p(T ) 1 from molecular simulations and the corresponding curve
p = N kT /V . The ideal gas law is clearly reasonable, but not a very good
approximation. The law seems to be wrong in two ways: There is an
additional constant in the actual data: p = p0 + N kT /V , and in addition,
the slope of the p(T ) curve is not given by N kT /V , but is somewhat
adjusted, p = p0 + c(V ) N kT . Let us now address what the origin of
these differences may be.

9.5.2 Modifications to the ideal gas law

What can be the origin of these differences between the observed equation
of state and the equation of state for the ideal gas? What effects in the
Lennard-Jones model have not been included in the ideal gas law? There
are two main interaction in the Lennard-Jones potential: A repulsion at
1
The pressure p is measured inside the simulation program. We are here simply reading
the measured values from the simulation log file.
338 9 Chemical potential and Gibbs Distribution

0.8

Len n a r d -Jo n es
0.75 Id e a l g a s
Cl a u s i u s
0.7 va n d e r Wa a l s

0.65

0.6

P∗
0.55

0.5

0.45

0.4

0.35

3 3.5 4 4.5 5 5.5 6 6.5

∗
T

Fig. 9.13 Plot of p(T ) as measured in molecular dynamics simulations using the Lennard-
Jones potential, for the ideal gas law, and for various modifications of the ideal gas law.

short distances and an attraction at long distances. Neither are included in

the ideal gas law. How can we include these effect in a simple modification?
Including the effect of a repulsive interaction. First, how can we
include the repulsion at short distances? The effect of a repulsion is that
atoms will not be very close to each other. When the atoms are far apart,
that is for low densities, this effect does not matter much. But as the
atoms are pressed closer together, we will notice that a significant part
of the volume is not available for motion because it is too close to other
atoms: We have a region of excluded volume.
We can make a simple model for the effect of an excluded volume by
assuming that each atom has a small volume b associated with it, so that
the excluded volume is N b. The volume that is actually available for
the atoms are then V − N b and not V . This will lead to a modification
of the ideal gas law. However, we do not only want to modify the ideal
gas law, we want to develop a new theory for a more realistic gas. This
means that we would need expressions not only for the pressure, but also
for the entropy and the chemical potential — which can be found from
Helmholtz free energy. Let us therefore see how we can modify the free
energy. For an ideal gas (IG), the Helmholtz free energy is
9.5 Van der Waals fluid model 339

nQ (T ) V nQ (t)
   
FIG = −N kT ln + 1 = −N kT ln +1 . (9.112)
n N
In this equation, the volume only occurs once, and we can include the
excluded volume by replacing V → V − N b:
(V − N b)nQ (t)
 
FC = −N kT ln +1 . (9.113)
N
This model for a gas is called the Clausius model, and the corresponding
equation of state is found by calculating p as a derivative of F :
∂FC N kT
 
pC = − = . (9.114)
∂V T,N V − Nb

Clausius’ law:
pC = (N kT )/(V − N b) . (9.115)

We have plotted Clausius’ law in the same plot as the ideal gas law and
the observed behavior for p(T ). Clausius law is an improvement over the
ideal gas law, but there are still clear discrepancies. Let us now see how
we also can include the effect of attractive interactions.
Including the effect of attractive interactions. While Clausius law is
an improvement of the ideal gas model, it does not include the correct
shift, p0 , in pressure. We therefore need an additional term. Based on the
plot in Fig. 9.13 we see that the additional term must reduce the effective
pressure: The actual pressure must be lower than the pressure predicted
by Clausius law. What physical effect may cause this? Attraction between
the particles! We therefore need to include the effect of the attractive
part of the Lennard-Jones model. This will give rise to a binding energy
— a reduction in the energy and in Helmholtz free energy — when the
density is high.
Let us construct a simplified model for the attractive interaction
between the atoms. An atom in the gas will experience attractive interac-
tions from all the other atoms. The potential energy of this interaction is
the sum of the potential energies for all the other atoms. The potential
energy of atom i is
ui = ULJ (rij ) , (9.116)
X

j6=i
340 9 Chemical potential and Gibbs Distribution

a V b V-Nb c

b
b
b

b
b
b
b
b

Fig. 9.14 Illustration of attractive and repulsive interactions in a van der Waals gas.
a and b Illustration of the excluded volume N b , corresponding to a volume b for each
atom. c Illustration of the attractive interactions between the atoms. This gives rise to
an average binding energy.

where rij is the distance from atom i to atom j. However, since the
potential energy only depends on the distance, we can instead sum over
distances, but then we need to include the number of atoms at each
distance, which corresponds to the density of atoms measured around
atom i, ni (r) times the volume element dV :
Z
ui = ULJ (r)ni (r)dV . (9.117)
V

For each atom, the actual ni (r) will vary in time as all the other atoms
move about. Instead of calculating this integral for each atom, we will
assume that all atoms experience the same, avarage density n(r) as a
function of distance to the atom, and that this average density can be
written as n(r) = nφ(r), where n is the average density of the system and
φ(r) gives the variations relative to the average density. The potential
energy of atom i is then:
Z Z
ui = ULJ (r)nφ(r)dV = n ULJ φ(r)dV = −2an . (9.118)
V
|V {z }
=−2a

The integral does not depend on N or V . We have introduced the

notation −2a for the integral. (You will see below why we have introduced
the factor 2). The theory can be made even simpler by assuming that
φ(r) is constant in space, but this is not necessary for our argument.
The potential energy of one atom is ui . What is the potential energy
of the whole gas? The gas consists of N atoms, and it may therefore be
tempting to simply multiply ui by N . But this is not correct, because we
should only include the potential energy for each pair of particles once,
9.5 Van der Waals fluid model 341

and the number of pairs is N (N − 1)/2 and not N 2 . Since ui already

includes the interactions from N atoms, we must multiply ui by N/2
to find the total potential energy: utot = (N/2)(−2an) = −anN . This
potential energy must be added to the Helmholtz free energy without
interactions to find the (approximative) Helmholtz free energy with
attractive interactions:
nQ (T )
 
Fvdw = −N kT ln + 1 − anN (9.119)
n
V nQ (T ) N2
 
= −N kT ln +1 −a . (9.120)
N V
This model is called the van der Waals model of a gas. The model
was introduced in the doctoral dissertation of Johannes Diderik van der
Waals (1837-1923; Nobel 1910). The equation of state for the van der
Waals gas is found from the pressure:

∂Fvdw N kT N 2a
 
p=− = − 2 , (9.121)
∂V T,N V − Nb V

which also can be written as

!
N 2a
p + 2 (V − N b) = N kT , (9.122)
V

which is called the van der Waals equation of state.

The van der Waals model for a gas:

(V − N b)nQ (T ) N2
 
Fvdw = −N kT ln +1 −a (9.123)
N V
Equation of state for the van der Waals system:
!
N 2a
p + 2 (V − N b) = N kT , (9.124)
V

The van der Waals model is described by two parameters, the excluded
volume b and the attractive interaction integral, a. First, let us see how
these parameters are determined, then let us see how we can describe the
342 9 Chemical potential and Gibbs Distribution

behavior of the system in a general way, where the details of the particular
parameters values only are included as dimensionless parameters.
Parameters for the Lennard-Jones system. How can we determine
the values of b and a for an actual gas? We can find these either from
experimental data, by simulations of more complicated fluids such as
water, or by theoretical considerations for simple fluids such as a Lennard-
Jones system. The actual values of a and b will be different for different
substances, and they may even vary for the same substance.
For the Lennard-Jones system we may estimate the values of a and
b. The excluded volume corresponds to a volume that is not available
for motion. This must be related to σ, the characteristic length of the
potential. We could assume that the excluded volume starts where the
potential is zero, that is for a volume of radius σ in the Lennard-Jones
potential. However, this is too large, since the atoms often may be
compressed into this zone. Instead, we will guess that the exluced volume
is one half of the volume corresponding to σ, that is

b = vex = (1/2)(4π/3)σ 3 = (2π/3)σ 3 . (9.125)

We can estimate a from the integral of the potential, assuming that the
density is constant, n(r) = n (φ(r) = 1):
∞ σ6 8π 3
Z
a' − 4πr2 dr = σ . (9.126)
σ/21/3 r6 3

This p(T ) curve for this set of parameter values are plotted in Fig. 9.13,
and the resulting model fits excellently to the data measured in molecular
dynamics simulations of the Lennard-Jones model.

9.5.3 Properties of the van der Waals system

The van der Waals gas describes a gas with interactions — both attraction
and repulsion. How does the resulting gas behave? And does this system
display the behavior of both a gas and a fluid, just as we saw the
simulation system did? In order to answer these questions we need to
analyze the behavior of the van der Waals gas. We start by studing the
behavior of isotherms, which shows that the system indeed displays a
phase equilibrium, then we use the Gibbs free energy to discuss the phase
equilibrium of the system, and discover that phase equilibrium only occurs
at low temperatures and pressures, at sufficiently high temperatures and
9.5 Van der Waals fluid model 343

pressures — beyond what we call the critical point — there is no longer

a difference in phase.
Non-dimensional equation of state. The equation of state provides
the relation between p and V for a given temperature, T , and number of
particles (atoms/molecules) in the gas. However, the equation of state
contains two parameters, a, and b, that are material specific — the values
of a and b depens on the particular gas we are studying. But the general
form of the equation is state is general. Could we make it even more
general by rescaling the pressure, volume and temperature of the gas? We
may say that the values of a and b can be included as characteristic values
for the pressure, volume and temperature, and then we can measure all
the quantities in these quantities. For example, it seems reasonable to
measure the volume in terms of the excluded volume, N b. We can rewrite
the equation of state in a form where this is clear:

(N b)2 a
!
V − Nb N kT

p+ = , (9.127)
V 2 b2 Nb Nb

and we see that we may measure pressure in terms of a/b2 , giving:

(N b)2
!
a p V − Nb N kT

+ = . (9.128)
b2 a/b2 V2 Nb Nb

We introduce the characteristic volume, V ∗ = N b, and the characteristic

pressure, p∗ = a/b2 , and divide by a/b2 on both sides, getting

(V ∗ )2
!
p kT b2
+ (V ∗ − 1) = . (9.129)
p∗ V2 ab

Finally, we introduce the characteristic temperature, T ∗ = a/kb, leaving

us with a very simple equation of state:
2 !
p V∗ T

+ (V ∗ − 1) = . (9.130)
p∗ V T∗

Why did we introduce these reduced coordinates? Now, all the material-
specific details are in the characteristic values, whereas the equation of
state is general. We can therefore study general properties of the van der
Waals gas — properties that are valid for all van der Waals gases.
While the characteristic values p∗ , V ∗ , and T ∗ we have introduced
are very simple and lead to a simple equation of state in the rescaled
344 9 Chemical potential and Gibbs Distribution

coordinates, it is customary to use a slightly different set of characteristic

values, which is only a numerical rescaling: pc = p∗ /27 = a/27b2 , Vc =
3V ∗ = 3N b, and kTc = 8/27 kT ∗ = 8/27 ka/b. We can rewrite the
equation of state using these variables (see exercise), getting:
3 1 8
  
p0 + V0− = T0 . (9.131)
V 02 3 3
where p0 = p/pc , V 0 = V /Vc , and T 0 = T /Tc . The only reason to do this
is that the behavior of the van der Waals gas with these rescaled (non-
dimensional) parameters, have simpler numerical values for important
points in the phase diagram. We will therefore use this rescaling.
Law of corresponding states. This equation contains no remnants of a
or b! There are only numbers and the non-dimensional pressure, volume
and temperature. We can therefore study the behavior of this equation
of state and learn about all possible van der Waals gases! This way of
writing the equation of state is called the law of corresponding states:

The law of corresponding states:

3 1 8
  
p0 + V0− = T0 . (9.132)
V 02 3 3
In terms of the dimensionless variables T 0 = T /Tc , V 0 = V /Vc ,
and T 0 = T /Tc , where pc , Tc , and Vc correspond to the pressure,
temperature and volume at the critical point of the gas.

Notice that the behavior of real gases is not to a high precision in

correspondence with this equation.
Plotting the behavior of a van der Waals gas. How can we now
characterize the behavior of the van der Waals gas? We could start to
look at isotherms. We know that for an ideal gas the isotherms behave
as p = (N kT )/V in a p-V -diagram. How does the isotherms of a van der
Waals gas look like? We plot isotherms for various values of T 0 using the
following script:
from pylab import *
That = [0.9,0.95,1.0,1.05]
for i in range(len(That)):
T = That[i]
V = linspace(0.5,2.0,1000)
9.5 Van der Waals fluid model 345

p = 8.0/3.0*T/(V-1/3)-3.0/(V**2)
plot(V,p); hold(’on’)
xlabel(’V/V_c’); ylabel(’p/p_c’)

Where we have plotted p(V ) for values of T 0 = T /Tc that are below and
above 1 in Fig. 9.15.

Fig. 9.15 Plot of four 2

T/Tc =0.90
isotherms for the van der T/Tc =0.95
Waals gas. 1.5 T/Tc =1.00
T/Tc =1.05

p=pc
1

0.5

0
0.5 1 1.5 2 2.5 3 3.5 4
V =Vc

Unphysical isotherms. We can now compare these isotherms with the

sketch from Fig. 9.10. First, we notice two aspects of the plotted isotherms
for the van der Waals gas: (i) When T < Tc there are multiple possible
volumes for a given pressure, which means that V (p) is not well defined.
This was also the case in Fig. 9.10, where this was interpreted as a two-
phase system, where the volume changed as the ratio of gas to liquid was
changed. (ii) When T < Tc there are region where p(V ) is an increasing
function of V . This is clearly unphysical! It would mean that the pressure
increases when the volume increases while the temperature is constant.
(We will address this behavior using the concept of compressibility in
the exercises). This means that there is a region of the p(V ) curve that
cannot represent an equilibrium situation. How can we determine how
this system should behave in equilibrium? We need to find the minimum
of the Gibbs free energy of this system.
Gibbs free energy for the van der Waals gas. We find Gibbs free
energy from Helmholtz free energy from G = F + pV and then by
ensuring that the end results is only a function G(T, p, N ) of T , p, and
N . We use the expression for Helmholtz free energy from (9.123) to find
Gibbs free energy:

(V − N b)nQ (T ) N2
 
G = −N kT ln +1 −a + pV , (9.133)
N V
346 9 Chemical potential and Gibbs Distribution

We can rewrite Gibbs free energy using the same non-dimensionalization

as we did above: p0 = p/pc , V 0 = V /Vc , and T 0 = T /Tc , where pc =
a/(27b2 ), Vc = 3N b, and kTc = 8a/27b:
V N aN b 3N bV
   
G = −N kT ln 3 −1 b nQ (T ) + 1 − +p
3N b Vb 3N b
(9.134)
G N kT aN 3bpV 0
=− (ln (3V 0 − 1) b nQ (T ) + 1) − +
N kTc N kTc 3bN kTc V 0 kTc
(9.135)
G aN 27b 3bpV 0 27b
= −T 0 ln (3V 0 − 1) + f (T 0 ) − +
N kTc 3bN V 0 a 8a
(9.136)
G 9 3
= −T 0 ln (3V 0 − 1) + f (T 0 ) − + p0 V 0 (9.137)
N kTc 8V 0 8
G 8 3
= − T 0 ln (3V 0 − 1) − 0 + p0 V 0 + f 0 (T 0 ) . (9.138)
(3/8)N kTc 3 V

where the function f 0 (T 0 ) only depends on T 0 and not on p0 or V 0 , and

where we introduce g = G/(3/8)N kTc as the dimensionless Gibbs free
energy per particle.
Gibbs free energy is supposed to be a function of p, T , and N , and
g = G/N should only depend on p and T , but the our result now depends
on both T 0 , p0 , and V 0 . The challenge is that we found from the plot of
p(V ) that for T < Tc V (p) is not a unique function. We therefore cannot
replace V 0 by V 0 (p0 ). Instead, we must use that Gibbs free energy is
minimal in equilibrium. Let us therefore plot g(V 0 ; p0 , T 0 ) and find the V 0
that makes Gibbs free energy minimal — this must be the equilibrium
value for V 0 . When we plot the function, it is more practical to plot g 0
as a function of n0 = 1/V 0 :

g 0 (n0 ; p0 , T 0 ) = −(8/3) T 0 ln(3/n0 − 1) − 3n0 + p0 /n0 + f 0 (T 0 ) . (9.139)

Here, we will neglect the last term, since this only depends on T 0 and
will not influence the n0 that makes g 0 minimal.
In Fig. 9.16 we have plotted g 0 (n0 ; p0 , T 0 ) near the values p0 = 0.647
and T 0 = 0.9, which are on the phase coexistence curve. In Fig. 9.16a
we have varied the pressure p0 . If we start with a pressure below phase
coexistence, at p0 = 0.55, we see that Gibbs free energy has a clear
minimum at n0 = 0.315. This means that in equilibrium, the van der
9.5 Van der Waals fluid model 347

Waals gas will only contain a single phase with this density — a gas
phase.
However, if we increase the pressure, Gibbs free energy will go from
a configuration with only one minimum, to a configuration with two
(global) minimum values, here occuring for p0 = 0.647. For pressures just
below this value, there is only one minimum in Gibbs free energy, and the
whole system is a gas, but when the pressure reaches this value, there are
suddenly to minima at n01 = 0.42 and n02 = 1.65 so that g 0 (n01 ) = g 0 (n02 ).
We interpret this to mean that there are now two phases in the system,
one phase with a high density — a fluid phase — and one phase with
a low density — the gas phase. Notice that at this particular set of p0 ,
T 0 values, all the densities along the stapled line between n01 and n02 are
possible values for the density. When the density of the system is n01 it
means that all of the system is in the low density gas phase, whereas
when the density of the system is n02 all of the system is in the high
density liquid phase. If half of the system is in the gas phase, and half
is in the liquid phase, the density of the system is n0 = 0.5 n01 + 0.5 n02 .
This means that Gibbs free energy is the same value, and minimal, for
all densities n0 in the interval from n01 to n02 , and that the density is
determined by how much of the system is in the gas or in the liquid
phase. Gibbs free energy therefore has the form ge0 (n0 ; p0 , T,0 )

 g (n ; p , T )
0 0 0 0
 n0 < n01
ge0 (n0 ; p0 , T 0 ) g 0 (n1 ; p0 , T 0 ) n01 < n0 < n02 , (9.140)
g (n ; p , T )
 0 0 0 0 n02 < n0


when p0 , T 0 are on the phase coexistence curve.

If we increase the pressure only slightly away from the phase coexistence
curve, we see that there is again only one density that gives a minimal
Gibbs free energy, and the whole system is in the liquid phase, as
illustrated for p0 = 0.75.
We can present a similar picture if we keep the pressure constant and
change the temperature, as illustrated in Fig. 9.16b.
Toward the critical point. What happens if we now increase both the
pressure and the temperature of the system, keeping the system on the
coexistence curve? We have plotted Gibbs free energy for increaseing
p0 , T 0 in Fig. 9.17. This illustration gives a very clear picture of what
happens as p0 → 1 and T 0 → 1: The densities of the two phases are
converging, and Gibbs free energy goes from having two minima to only
have a single minima at p0 = 1, T 0 = 1. We call this point the critical
348 9 Chemical potential and Gibbs Distribution

Fig. 9.16 Plot of -3.5

g 0 (n0 ; p0 , T 0 ) near the phase

coexistence curve.

g=gc
-4

p'=0.550
p'=0.647
p'=0.750
-4.5
0 0.5 1 1.5 2 2.5
n=nc

-3.5

g=gc
-4

T'=0.85
T'=0.90
T'=0.95
-4.5
0 0.5 1 1.5 2 2.5
n=nc

point. Beyond the critical point, there is only a single phase in the system,
and there is no longer any phase coexistence or phase transition. This
is the reason for the choice of the numerical prefactors for pc and Tc , so
that the critical point occurs at (p0 , T 0 ) = (1, 1).

Fig. 9.17 Plot of -3.5

g 0 (n0 ; p0 , T 0 ) near the phase

coexistence curve.
g(n0 ; p0 ; T 0 )

-4 p=0.80, T=0.38
p=0.90, T=0.65
p=0.95, T=0.81
p=0.98, T=0.92
p=0.99, T=0.96
-4.5
0 0.5 1 1.5 2 2.5 3
n0

Behavior of van der Waals gas below Tc . The full behavior of the van
der Waals gas system is address in project XX.
p(T ) and latent heat for the van der Waals gas.

9.5.4 Surface tension

ams 8: Draft only. Not curriculum in Fys2160.
We introduce ∆µ = µg − µl as the difference in chemical potential
between the vapor surrounding a liquid droplet and the liquid in bulk,
that is for a very large drop (no surface effects).
If ∆µ > 0 it means that the liquid has lower free energy than the
gas/vapor - and liquid droplets may spontaneously form.
9.5 Van der Waals fluid model 349

However, we must also include the surface free energy, because this
tends to increase the energy of the liquid.
For a very small droplet, with a very small radius of curvature, the
surface energy will be dominating and the drop can be unstable with
respect to the gas/vapor.
Let us study this by addressing Gibbs free energy when a droplet of
radius R forms. We introduce

∆G = Gl − Gg = −(4π/3)R3 nl ∆µ + 4πR2 γ , (9.141)

where nl is the number density of the liquid (the concentration), and γ

is the energy it costs to form a surface - per area.
The liquid drop will grow when Gl < Gg . There is a maximum at
d∆G
= 0 = −4πR2 nl ∆µ + 8πRγ , (9.142)
dR
which gives
2γ
Rc = . (9.143)
nl ∆µ
We call this the critical radius of nucleation. When the radius is
smaller than R the drop will grow smaller - and disappear. When the
radius is larger than R the drop will continue to grow larger and larger.
We may (or maybe not?) assume that the gas is a an ideal gas. In
that case the chemical potential is

∆µ = kT ln(p/peq ) , (9.144)

where peq is the equilibrium vapor pressure of the bult liquid.

We can use realistic values then to estimate Rc for water at 300K and
p = 1.1peq . In addition, we need a value for γ, which we can find from
tables, γ = 72 10−3 N/m. This gives Rc ' 10−8 m = 10 nm.

9.5.5 Lattice gases

ams 9: lattice gas model: model of general system, mean-field theory for
thermodynamics. Gibbs free energy minimization. p(T) curve. Lattice
gas automata.

9.5.6 Stability
350 9 Chemical potential and Gibbs Distribution

ams 10: If the gas starts in a local minimum in g(n), it may take some
time for the system to reach the global minimum. Metastability and
stability, supercooled and superheated water, heterogeneous nucleation,
glasses

9.6 Solutions

The systems we have been studying so far mostly only contain one type of
particles. The state of the system is described by the number of particles,
N , the pressure, p, the temperature, T , and possible also the volume in
the case of phase coexistence.
If we introduce another type of particles, we get a mixture or a solution.
There are many types of solutions: a solution may be a mixture of gases;
in a liquid we can dissolve gases or solids; in solids we can dissolve
gases, liquids or other solids. For example, hydrogen-gas can be dissolved
in palladium, or mercury can be dissolved in gold and in many other
metals; tin and zink can be dissolved in copper. Solutions are one of the
most important classes of systems for chemist, and our understanding of
solutions is important for our understanding of many problems of both
theoretical and practical importance.
We will here primarily study dilute solutions, which are where one
of the components are so dilute that they do not affect the energy of
the other component. Typical examples are trace gases in air or dilute
solutions in water.

Solution definitions:
• the component that is dissolved is called the solute
• the substance it is being dissolved in is called the solvent
• the resulting product is called the solution.
For example, salt water is a solution, made from a solvent, water,
and a solute, salt.

While solutions in general may include complex interactions, ideal

solutions are simple, since the interactions between the solvent and the
element that is dissolved primarily is through the entropy of mixing term.
Let us start by looking at mixed ideal gases, often called ideal mixtures.
9.6 Solutions 351

9.6.1 Ideal mixture

An ideal mixture is a mixture where the species are non-interacting:

Ideal mixture: An ideal mixture is a system

• where the species are non-interacting, so that there is no coupling
between the energy states of the various species
• with an additional entropy of mixing that contributes to the
total entropy of the system
Ideal mixtures are good approximations for mixtures in the dilute
limit.

Let us develop a thermodynamic theory for the behavior of an ideal

mixture of several species. Fig. 9.18 illustrates the mixing of two non-
interacting species. First, we consider one of the species in a system of
volume V , pressure p and temperature T . Gibbs free energy for species i
is then
G0i = Ei0 − T Si0 + pVi0 = Ni gi0 , (9.145)
where the superscript 0 indicates that this is the values for species i on
its own. If we now place several species into the same box (with the same
volume V = Vi0 for all the species), then the energies do not change,
because the particles do not interact. Similarly, the pV terms do not
change even if we add more particles into the same box. But what about
the entropy? The total entropy will increase, because there are now more
ways to rearrange the various particles. For ideal gases, the entropy for
each species would depend on the number of particles, the mass of the
particles, the size of the box and the temperature. This internal entropy
would not change. In addition, there would be an increase in entropy
due to the mixing of the various species. We will therefore assume that
the Gibbs free energy for the mixed system will have two contributions:
Gibbs free energy for each of the species and a term related to the entropy
of mixing:

G(p, T, {Ni }) = Ni gi0 (p, T, Ni ) − T Smix , (9.146)

X

where gi0 (p, T, Ni ) is Gibbs free energy per particle for a system consisting
only of type i particles in the same box. (The superscript 0 indicates
352 9 Chemical potential and Gibbs Distribution

that this is the Gibbs free energy without any interactions). What is the
entropy of mixing?

A+B

Fig. 9.18 Illustration of a system A of NA particles of type A and a system B of NB

particles of type B mixed into a common system.

Entropy of mixing. We can find the entropy of mixing from the multi-
plicity of mixing. For an gas or for a solid of N = i Ni particles, how
P

many ways can we rearrange the particles? We know that for two species,
1 and 2, the total number of ways that we rearrange N1 particles of type
1 and N2 particles of type 2 is

(N1 + N2 )!
!
N
Ω(N1 , N2 ) = = (9.147)
N1 N1 !N2 !

and for k species:

N!
Ω({Ni }) = (9.148)
N1 !N2 ! . . . Nk !
We find an approximate expression for the mixing entropy, Smix = k ln Ω
using Stirling’s formula, ln y! = y ln y − y:
9.6 Solutions 353

k
Smix /k = ln Ω = N ln N − N − (Ni ln Ni − Ni ) (9.149)
X

i=1
k k
= |{z}
N ln N − N − Ni ln Ni + (9.150)
X X
Ni
P
= Ni i=1 i=1
i | {z }
=N
k k
= Ni ln N − Ni ln Ni (9.151)
X X

i=1 i=1
k
Ni
=− Ni ln (9.152)
X
.
i=1
N

It is common to use the fraction, xi = Ni /N to describe the relative

number of the various components. In terms of fractions, xi , the entropy
of mixing is
Smix = k xi ln xi (9.153)
X

Gibbs free energy of an ideal mixture. We insert the entropy of mixing

into Gibbs free energy, finding
k k
Ni
G(p, T, {Ni }) = Ni gi0 (p, T, Ni ) + kT Ni ln (9.154)
X X
.
i=1 i=1
N

We use the term xi = Ni /N for the mole fraction or the fraction of

species i:
k k
G(p, T, {Ni }) = G0i (p, T, Ni ) + N kT xi ln xi . (9.155)
X X

i=1 i=1

(Notice that xi = Ni /N = ( Ni )/N = 1).

P P P
i i i

Chemical potential of individual species in an ideal mixture. The

chemical potential of component i can then be found from the partial
derivative of Gibbs free energy:
∂G
 
µi = (9.156)
∂Ni p,T,{Nj }

where we keep Nj constant for all j 6= i. (Notice that N = Ni and

P
i
therefore N is not a constant in this derivative).
354 9 Chemical potential and Gibbs Distribution

∂G
 
µi = (9.157)
∂Ni p,T,{Nj }
k
! !
∂G0i ∂ X
= + Nr ln xr (9.158)
∂Ni p,T,{Nj }
∂Ni r=1
p,T,{Nj }
k
1 ∂xr
= µ0i (p, T ) + kT ln xi + (9.159)
X
Nr
r=1
xr ∂Ni
k
∂xr X
= µ0i (p, T ) + kT ln xi + N xr (9.160)
∂Ni r=1
| {z }
=1
= µ0i (p, T ) + kT ln xi . (9.161)

where µ0i (p, T ) = gi0 (p, T ) is the chemical potential of the system when
only species i is present.
The chemical potential for component i is therefore only dependent
on xi = Ni /N = ni /n = mi , and not on the other xj values. The only
dependence on the other atoms is through the N in the denominator of
xi = Ni /N . We can check the consistency of the expression by noticing
that for x1 = 1 we only have the pure substance, and µ1 = µ01 , as it
should.
The ideal mixture approximation is versatile. This simple expression
for the chemical potential is an approximation to case of a general
mixtures. We call this approximation the ideal mixture approximation.
This approximation is versatile. It is clearly applicable to ideal gases,
where there are no interactions between the particles, but it is also a
reasonable approximation to many other systems. It is well demonstrated
the the ideal mixture approximation is:
• almost exact for mixtures of isotopes
• good for mixtures of similar molecules
• very good for the solvent in dilute solutions
• reasonably good for use in chemical equilibriums
Let us now see how we can address an ideal mixture of ideal gases.
Solution of ideal gases. Mixtures of ideal gases are ideal mixtures,
since there are no interactions between the particles in the gas. For a
multi-component gas, with Ni particles (atoms/molecules) of species i,
the total number of particles is N = i Ni . The pressure of the gas is
P
9.6 Solutions 355

k k
N kT Ni kT
p= = = (9.162)
X X
pi ,
V i=1
V i=1

where pi = Ni kT /V is called the partial pressure of component i. Since

the particles are non-interacting, you can think of a multi-species ideal
gas as a set of k gases, each in a volume V , and each with a pressure pi .
The total pressure is the sum of the (partial) pressures of each of the
gases, p = i pi .
P

For an ideal gas the fraction xi can be expressed in terms of the partial
pressure: xi = Ni /N = Ni kT /N kT = pi V /pV = pi /p, which gives the
following expression for the chemical potential of species i in a mixture
of ideal gases:

µi (p, T ) = µ0i (p, T ) + kT ln xi = µ0i (p, T ) + kT ln pi /p . (9.163)

Where µ0i (p, T ) = kT ln ni /nQ = kT ln pi /(nQ (T ) kT ) for an ideal gas.

9.6.2 Binary mixtures

The expressions become simpler when there are only two species, 1 and
2, present. In this case, we can describe the state of the solution only
by x = x2 = N2 /N and the total number of particles, N . We see that
x1 + x2 = 1 gives x1 = 1 − x2 = 1 − x. In this case we can simplify the
expression for Gibbs free energy:

G(p, T, N1 , N2 ) = N1 g10 (p, T ) + N2 g20 (p, T ) + N kT xi ln xi (9.164)

X

i
= N (1 − x)g10 (p, T ) + N xg20 (p, T ) + N kT ((1 − x) ln(1 − x) + x ln x) ,
(9.165)

where the last term is due to the entropy of mixing: Smix =

k ((1 − x) ln(1 − x) + x ln x).
Fig. 9.19 shows plots of Gibbs free energy (and the mixing entropy) as
a function of x. We illustrate two cases: The case of no mixing and the
case of mixing. The case of no mixing corresponds to the case where all
of species 1 are on the left side and all of species 2 are on the right hand
side of the system. This corresponds to a Gibbs free energy without any
additional entropy of mixing, which is simply the linear combination of
the Gibbs free energy for each of the two species: G = (1 − x)G01 + xG02
356 9 Chemical potential and Gibbs Distribution

(illustrated by the red line). The total Gibbs free energy is clearly smaller
when the system is mixed due to the effect of the entropy of mixing.

2 0.8

0.6
1.5

0.4
G

S
1
0.2

0.5 0
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
x x
Fig. 9.19 Plots of the Gibbs free energy for a mixed (blue) and an unmixed (red) system.

In non-ideal mixtures interactions between particles of different species

will change the internal energies as well, opening also for unmixed states
in thermal equilibrium as discussed in Sect. XX

9.6.3 Dilute solutions

The results can be further simplified in the case of a dilute solution — a
solution where there is few of the solute (2) compared to the solvent (1):
N2  N1 . In this case x = N2 /N  1, and we see that x = N2 /N =
N2 /(N1 + N2 ) ' N2 /N1 .
Chemical potential for the solvent in a dilute solution. The chemical
potential for the solvent (1) in an ideal mixture is

µ1 (p, T ) = µ01 (p, T ) + kT ln x1 = µ01 (p, T ) + kT ln(1 − x) , (9.166)

when x = x2  1, we can approximate ln(1 − x) by the first term in its

Taylor expansion, ln(1 − x) ' −x, getting:

µ1 (p, T ) = µ01 (p, T ) − kT x , (9.167)

where µ01 (p, T ) was the chemical potential of the solvent before the solute
was added. Notice that this means that the chemical potential of the
solvent therefore becomes lower after the solute is added.
Chemical potential for the solute in a dilute solution.
ams 11: May add derivation of this?
9.6 Solutions 357

9.6.4 Example: Vapor pressure above a dilute solution

We have already found an equation for the vapor pressure for a liquid.
What happens if we dissolve a substance in the liquid? How will the
vapor pressure change?
We can apple Le Chaatlier’s principle to gain a quick understanding
of what will happen. According to the principle, the system will respond
in such a way as to oppose the interaction — the system will therefore
respond by increasing the amount of liquid in order to make the solution
thinner, at the expense of the vapor. That is, the system will respond by
lowering the vapor pressure. But by how much?
The system is illustrated in Fig. 9.20. A closed container at constant
pressure, p, and temperature, T , contains N atoms/molecules of a solvent,
Ng are in the gas phase and Nl are in the liquid phase. In addition, there
are N 0 dissolved atoms in the liquid. We assume that we can neglect the
amount of dissolved atoms in the gas — its vapor pressure is negligible.

Fig. 9.20 Illustration of T,p0 T,p

a system with a solvent
at equilibrium with its
vapor at T, p0 . Then a
solute is dissolved in the
solvent, and the system dissolve
reaches equilibrium at a solute
new equilibrium vapor
pressure p.

solvent solvent+solute

The behavior of the system is determined by Gibbs free energy: In

equilibrium Gibbs free energy is minimal. Gibbs free energy for the
system is

G = Ng gg (T, p) + Nl gl (T, p) + N 0 g 0 (T, p) + N 0 kT (ln N 0 /Nl − 1) ,

(9.168)
where we have used the approximation N 0 /(N 0 + Nl ) ' N 0 /Nl  1. In
equilibrium, the chemical potential of the two phases of the solvent must
be equal, that is, µl = µg . This is true for p0 , T before the substance is
dissolved in the solvent, and for p, T after the substance was added. We
want to find the change in vapor pressure, that is, p − p0 = ∆p. We find
358 9 Chemical potential and Gibbs Distribution

the chemical potential by derivation of Gibbs free energy:

!
∂G
µg = = gg (T, p) , (9.169)
∂Ng T,p,Nl ,N 0

∂G N0
 
µl = = gv (T, p) − kT . (9.170)
∂Nl T,p,Ng ,N 0 Nl
Here, we now insert p = p0 + ∆p, and we approximate gv (T, p0 + ∆p)
with its first order Taylor expansion, and similarly for gl (T, p0 + ∆p):
∂gg
 
µg (p0 + ∆p, T ) = gg (p0 + ∆p, T ) = gg (p0 , T ) + ∆p , (9.171)
∂p T

and similarly
N0 ∂gl N0
 
µl (p0 +∆p, T ) = gl (p0 +∆p, T )−kT = gl (p0 , T )+ ∆p−kT ,
Nl ∂p T Nl
(9.172)
We introduce x = N 0 /Nl , where x  1. We notice that
∂gl ∂gg
   
= vl , = vg , (9.173)
∂p T ∂p T

are the volume of a single atom of gas or liquid.

In equilibrium the chemical potentials must be equal:

µl (p, T ) = gl (p0 , T ) + vl ∆p − kT x = µg (p, T ) = gg (p0 , T ) + vg ∆p .

(9.174)
vl ∆p − kT x = vg ∆p ⇒ (vg − vl ) ∆p = −kT x , (9.175)
which can be simplified further since the volume per particle in the liquid
is much smaller than in the gas, vl  vg , and therefore vg − vl ' vg , and
for an ideal gas we know that vg = kT /p0 . We have therefore obtained
Raoult’s law:
∆p N0
vg ∆p = =x= . (9.176)
p0 Nl
Notice that the change in the vapor pressure is proportional to the
amount of dissolved substance. But this change in vapor pressure is
independent of what type of substance is dissolved — it is only the
number of dissolved particles that matter. We may use this to determine
the atomic weight of a substance. If we have a solvent with a known
molecular weight, such as for water, we may weigh a given amount, Nl , of
9.6 Solutions 359

the water (solvent). Then we dissolve a given mass m of a substrance with

an unknown molecular weight in the water and measure the reduction in
vapor repssure, ∆p, and find the molecular weight from
m mp0
M= 0
=− . (9.177)
N Nl ∆p

9.6.5 Example: Osmotic pressure

We can use our newfound theory for dilute solutions to find the osmotic
pressure of a solution. Biological membranes ensure that the chemistry
inside and outside of the membrane can be different – thus ensuring
the machinery of life inside a cell, while living in a potentially hostile
environment. Membranes can be made to allow the transport of a solvent
such as water while restricting the transport of a solute, such as salt.
Fig. 9.21 illustrates a membrane with pure solvent on one side and a
solution on the other side. The membrane allows the solvent to move
from one side to another, but not the solute.

Fig. 9.21 Illustration of a

solution (left) and a pure
solvent (right) separated
by a membrane that is
permeable to the solvent, p1
p2
but not to the solute.

If the system starts in this situation, with a pure solvent on one

side and a solution on the other, the system will clearly not be in
thermodynamic equilibrium, because the chemical potentials are not the
same on each side of the membrane. The chemical potential in the solvent
is µ2 (p, T ) = µ0s (p, T ), while the chemical potential in the (dilute) solution
is approximately µ1 (p, T ) = µs (p, T ) = µ0s (p, T ) − kT ln x. Transport will
therefore occur from the side with the solvent to the side with the solution.
However, we can stop the transport by increasing the pressure in the
solution until the chemical potentials are equal. This occurs when the
pressure in the solution is p1 and the pressure in the solvent is p2 . The
chemical potentials are then

µ2 (p2 , T ) = µ0s (p2 , T ) = µ1 (p1 , T ) = µ0s (p1 , T ) − kT x . (9.178)

360 9 Chemical potential and Gibbs Distribution

If the pressure difference ∆p = p2 − p1 is small compared with p1 , we

can approximate µ0s (p2 , T ) with the Taylor expansion around p = p1 :
!
∂µ0s
µ0s (p2 ) ' µ0s (p1 ) + (p2 − p1 ) . (9.179)
∂p T,N,x

We can find the derivative of the chemical potential from the derivative
of Gibbs free energy, because µ0s = G0s /N , and therefore

1
! !
∂µ0s ∂G0s V
= = = vs0 , (9.180)
∂p T,N,x
N ∂p T,N,x
N

that is, the volume per particle in the pure solute. This gives
V
µ0s (p2 ) ' µ0s (p1 ) + ∆p (9.181)
N
which we can insert into (9.178), getting
V
µ0s (p2 ) ' µ0s (p1 ) + ∆p = µ0s (p1 , T ) − kT x , (9.182)
N
and therefore
N kT n N kT NB NB kT
∆p = − =− =− , (9.183)
V VN V
where B represent the solute. This pressure difference is called the os-
motic pressure. And the equation describing the osmotic pressure of
a dilute solution is called van’t Hoff’s formula. This means that the
pressure difference needed corresponds to the pressure in an ideal gas
with the concentration NB /V of the solute.
This pressure can be quite significant for realistic, biological systems.
For example, for a cell there are about 200 water molecules for every
molecule of some other type – this is therefore a dilute solution. Since
the atomic weight of water is 18u, the number of water molecules in a
liter of water (1 kg) is
1kg
Ns = , (9.184)
18 · 1.66 · 10−27 kg
and the pressure in a gas of Ns /200 particles in 1 liter is
9.7 Chemical reactions 361

N kT (1/200)Ns kT
p= =
V 0.001m3 (9.185)
1.38 · 10−23 J/K · 300K
= = 6.9N/m2 .
200 · 18 · 1.66 · 10−27 · 0.001m3
The pressure difference between the inside and the outside of a cell in
equilibrium with pure water is therefore approximately 7 atm – which is
high!
If the pressure difference is smaller than the osmotic pressure, molecules
from the pure solvent will flow through the membrane and into the so-
lution. However, if the pressure difference is larger than the osmotic
pressure, the pressure will drive the solvent from the solution and into
the pure solvent. This process is, not surprisingly, called reverse osmo-
sis and currently represents one of the best techniques for desalination
(removing salt from seawater) to make freshwater. Making good mem-
branes for this process is an area of great current technological and
industrial interest.

9.6.6 Example: Change in boiling point above a dilute

solution
9.6.7 Example: Henry’s law and the solubility of gases
9.6.8 Non-ideal mixtures
9.6.9 Surface growth processes
9.7 Chemical reactions

We now have what we need to address chemical reactions - how some

species are transformed into other species without changing the total
number of atoms. (The total number of particles may change).
Let us analyze chemical reactions from an example: The formation of
water from hydrogen and oxygen.

H+ + OH− ↔ H2 O , (9.186)

It is common to describe chemical reactions by it stochiometric coeffi-

cients, νj :
ν1 A1 + ν2 A2 + . . . + νk Ak = 0 , (9.187)
where Aj describes a chemical species such as O or O2 or H2 O. The
reaction above would then be described as
362 9 Chemical potential and Gibbs Distribution

+ 1 H+ + 1 OH− − 1 H2 O = 0 , (9.188)

that is

ν1 = 1 , A1 = H+ , ν2 = 1 , A2 OH− , ν3 = −1 , A3 = H2 O . (9.189)

What characterizes the equilibrium of a system with hydrogen, oxygen

and water present? Typically, we will consider equilibrium at constant
pressure and temperature, dp = dT = 0, which corresponds to typical
laboratory conditions. In this case, thermodynamic equilibrium occurs
when Gibbs free energy is minimal, that is, when dG = 0. In general, we
have that c
dG = −SdT + V dp + µi dNi = 0 , (9.190)
X

i=1

which for dT = dp = 0 simplifies to

c
dG = µi dNi = 0 . (9.191)
X

i=1

We have equilibrium when one more chemical reaction – one transfor-

mation from the left to the right or opposite – does not change Gibbs
free energy. For the chemical reaction at hand, the changes in Ni are not
arbitrary – they are linked by the coefficients νi of the reaction. For a
single reaction, dNj = νj . We therefore have that in equilibrium:

dG = µj dNj = µ j νj = 0 . (9.192)
X X

j j

We can determine this condition for any reaction if we only know the
chemical potential µj for each of the species. (Notice that this relation is
derived for constant p and T , but it also applies to the equilibrium of
reactions at constant T and V ).
Gibbs-Duhem relation. Notice that it is usual to call the relation

dG = −SdT + V dp + µj dNj = 0 , (9.193)

X

(which it is in equilibrium) the Gibbs-Duhem relation when rewritten

as
µj dNj = −SdT + V dp . (9.194)
X

j
9.7 Chemical reactions 363

9.7.1 Chemical equilibrium for Ideal Gas systems

For a multi-component ideal mixture, such as a system of ideal gases
or any other system well described as an ideal mixture, the chemical
potential of component i is

µi = µ0i + kT ln xi , (9.195)

where xi = Ni /N and µ0i is the chemical potential of the pure gas –

without any other gases present. The equilibrium condition from (9.192)
then gives
µ i νi = µ0i νi + kT ln xi = 0 , (9.196)
X X X

i i i

which gives
ln xi = − (9.197)
X X
kT µ0i νi .
i i

If we now multiply by Avogadro’s number, NA , in each side we get

νi ln xi = −NA µ0i νi = −∆G0 , (9.198)

X X
RT
i i

where ∆G0 is the change in Gibbs free energy for the reaction – which is
the hypothetical change in G when one mole of H2 reacts with one mole
of OH− , forming one mole of water at 1 bar. This value of ∆G0 you can
usually find in reference tables. If we take the exponential on both sides,
we find
∆G0
xi i = e− RT = K(T ) , (9.199)
Y ν

where K(T ) is only a function of T – and you can often find the value
of ∆G0 in chemical tables. The constant K is usually knows as the
equilibrium constant – and we see that we can acually calculate the value
of K(T ) if we know the chemical potentials µ0i for all the components in
the reaction.
For ideal gases, the relation can be rewritten in terms of the concen-
trations, ni = Ni /V , instead:
Y  Ni νi Y  Ni kT νi
xνi i = =
Y

i i
N i
N kT
νi Y  Ni νi  kT νi (9.200)
Y Ni kT
= = = K(T ) ,
i
pV i
V pV

which gives
364 9 Chemical potential and Gibbs Distribution

Y  Ni νi K(T )
= Q  νi = K1 (p, T ) . (9.201)
i
V kT
i pV

These equations are called the law of mass action

Notice that when we calculate K(T ) we must be very careful to choose

a consistent value for all the energies - we need to select the zero level in
the same way for all the particles. One way to define energies that are
internally consistent can be explained through a disassociation reaction,
where a molecule A2 disassociates into 2A. In this case, we should choose
the zero level of each composite particle (A2 ) to be the energy of the
disassociated particles (A) at rest. That is, if the binding energy for A2 is
 (this is the energy needed to place the two consituents of A2 infinitely
far away from each other), we place the ground state of the composite
particle (A2 ) at −.

9.7.2 Example: Disassociation of hydrogen

We start with the reaction

H2 ↔ 2H , (9.202)

which also can be written as

H2 − 2H = 0 , (9.203)

The law of mass action gives:

ν
nj j = K(T ) , (9.204)
Y

where j = 1 corresponds to H2 , so that ν1 = 1, and j = 2 corresponds to

H, so that ν2 = −2. It is usual to write

nH2 as [H2 ] (9.205)

The law of mass action is therefore

[H2 ]
[H2 ] [H]−2 = = K(T ) , (9.206)
[H]2
9.7 Chemical reactions 365

This means that

[H2 ] 1
= , (9.207)
[H] 1/2
[H2 ] K 1/2
so that the relative concentration of hydrogen is inversely proportional
to the concentration of H2 .

9.7.3 Example: pH and the Ionization of water

Water goes through the process

H2 O ↔ H+ + OH− , (9.208)

when in liquid form. This process is called the disassociation of water.

The law of mass action gives:

H+ OH− = [H2 O] K(T ) . (9.209)

  

In pure water each of the concentrations are

H+ = OH− = 10−7 mol l−1 . (9.210)

   

We can change this concentration by introducing a proton donor. This

increases the number of H+ ions and decreases the number of OH− ions
to ensure the product of the concentrations is constant.
It is common to introduce the pH through

pH = − log10 H+ . (9.211)
 

The pH of pure water is therefore 7.

Strong acids have low pH values. An apple has pH around 3.

9.7.4 Example: Kinetics, reaction rates, and catalytic

processes
What if we study the process

A + B ↔ AB , (9.212)

Then the rate at which the concentrations changes are related by

dnAB
= CnA nB − DnAB , (9.213)
dt
366 9 Chemical potential and Gibbs Distribution

where C describes how AB is formed in collisions and D is the reverse

process.
In equilibrium the concentrations does not change, and

CnA nB = DnAB , (9.214)

which also are related by the law of mass action:

nA nB D
= = K(T ) . (9.215)
nAB C
Now, what if AB is not formed by collisions between A and B, but in a
two step process involving a catalyst E:

A + E ↔ AE , AE + B ↔ AB + E , (9.216)

where E is returned to its original state after the reaction.

What is the point of E? It may increase the rates significantly?
How? The rates are not only determined by the energy of the final
configuration, but also by an energy barrier. The rate is determined
by the height of the energy barrier and the temperature (Arrhenius
processes). However, by introducing E we may lower the energy barrier
in each step, increasing the rate of the reaction.
Now, if the process is rapid so that E is short lived, then AE does
not form a significant quantity of A. Then the ratio nA nB /nAB is the
same as we found above - it is given by the law of mass action. The route
taken by the reaction is not important - the end result is the same.
In equilibrium, the direct and the inverse reaction rates must be the
same - what we assumed above - is called the principle of detailed balance.

9.7.5 Example: Dissolved Oxygen

Henry’s law.

9.7.6 Example: Charge distribution of the Hemoglobin

Molecule
Longer analytical and numerical example.
9.8 Old Stuff 367

9.8 Old Stuff

9.8.1 Intensive, Extensive Variables, and G

We take two identical systems - for example two system of ideal gas or
two systems of the einstein crystal - and put them together, forming a
system with double the number of particles.
Some variable will change and some will not in this process.
We call the variable that dot not change intensive variables. They
are p, T , µ.
Other variables are linear in N : They double when the system doubles.
We call these variables extensive variables. Examples are U , S, V , N ,
F , G.
If G is directly proportional to N , we can write G = N g(p, T ), where
g = G/N . What is g? It is simply
∂G
 
µ= =g, (9.217)
∂N T,p

and
G(T, p, N ) = N µ(T, p) . (9.218)

9.8.2 Multi-component systems

How can we generalize all our results to multi-component systems?
Originally, we introduced the thermodynamic identity from the micro-
canonical ensemble, and we found that we could write the entropy, S as
a function of U and V , which gave us the differentiale
1 p
dS = dU − dV , (9.219)
T T
then we extended to a system with N particles, getting
1 p µ
dS = dU − dV − dN . (9.220)
T T T
This can now directly be extended to a system with j = 1, . . . , k different
species by introducing a term related to the diffusive equilbrium for each
of the species, resulting in (review the original introduction if you are in
doubt):
1 p X µj
dS = dU − dV − dNj . (9.221)
T T j
T
368 9 Chemical potential and Gibbs Distribution

Similarly, we can generalize the chemical potential we found from

Helmholtz free energy (which is the same as the one we found from
the entropy): !
∂F
µj = . (9.222)
∂Nj T,V,{N }
j

And similarly for Gibbs free energy:

G(T, p, {Nj }) = (9.223)

X
Nj µj .
j

The thermodynamic identity then becomes

T dS = dU + pdV − (9.224)
X
µj dNj ,
j

and the differential for G becomes

dG = −SdT + V dp + (9.225)
X
µj dNj .
j

We will use this to address reactions between different chemical compo-

nents - by introducing the fundamental laws of chemistry.

9.8.3 Matlab MD script

% LJ MD calculation
clear all; clf;
L = 10; % Number of atoms = L^2
N = L*L;
rho = 0.8; % reduced density
Temp = 0.1; % reduced temperature
nsteps = 10000;
dt = 0.02;
printfreq = 1;
% Initial coordinates on cubic grid
r = zeros(N,2);
v = zeros(N,2);
[x y] = meshgrid((0:L-1),(0:L-1));
r(:,1) = x(:); r(:,2) = y(:);
% Rescale to wanted rho
L = L*(1.0/rho^2); r = r*(1.0/rho^2);
% Initialize with wanted T
v = sqrt(Temp)*randn(N,2);
% Internal variables
dt2 = dt*dt;
force = zeros(N,2);
9.8 Old Stuff 369

epot = zeros(N,1); ekin = epot; t = epot;

% Integrate motion
for i = 1:nsteps
% Velocity-Verlet - part 1
r = r + v*dt + 0.5*force*dt2;
r = r + (r<0.0)*L - (r>L)*L; % Periodic
v = v + 0.5*force*dt;
% Find forces/accelerations
[force,energy] = LJforce(r,L);
% Velocity-Verlet - part 2
v = v + 0.5*force*dt;
% Store energies
epot(i) = energy/N;
ekin(i) = 0.5*sum(sum(v.*v))/N;
t(i) = i*dt;
% Plot
if (mod(i,printfreq)==0)
tit = sprintf(’Timesteps = %d’,i);
plot(r(:,1),r(:,2),’o’);
title(tit);
axis equal, axis([0 L 0 L])
drawnow
end
end
%%
figure
ii = (1:i-1);
plot(t(ii),ekin(ii),’-r’,t(ii),ekin(ii)+epot(ii),’:k’);
xlabel(’t’)
ylabel(’E’);
legend(’K’,’E_{TOT}’);

function [force,energy] = LJforce(r,L);

% Calculate the force on each particle and the
% potential energy of a Lennard-Jones system
% with potential energy for each pair:
% V(r) = 4*U0*((sigma/dr)^12-(sigma/dr)^6))
% Variables in: r(1:N,3) coordinates
% L system size
% Notice: lengths measured in units of sigma
% energies measured in units of U0
s = size(r);
npart = s(1); dim = s(2);
L2 = L*0.5; mL2 = -L2;
ff = zeros(npart,dim); % forces
en = 0;
for i = 1:npart
ri = r(i,:);
for j = i+1:npart
rj = r(j,:);
rij = (ri-rj);
rij = rij + (rij<mL2)*L - (rij>L2)*L;
r2 = sum(rij.*rij);
370 9 Chemical potential and Gibbs Distribution

ir2 = 1.0/r2;
ir6 = ir2*ir2*ir2;
ir12 = ir6*ir6;
% Calculate force from i-j interaction
fij = (2*ir12-ir6)*rij*ir2;
ff(i,:) = ff(i,:) + fij;
ff(j,:) = ff(j,:) - fij;
% Calculate energy from i-j interaction
enij = (ir12-ir6);
en = en + enij;
end
end
en = en*4;
energy = en;
ff = ff*24;
force = ff;
return

9.9 Summary

9.10 Exercises

Exercise 9.1: Maxwell relations

a) Find the three Maxwell relations for Gibbs free energy.

Exercise 9.2: Compressibility of the van der Waals gas

ams 12: Show problems with vdW gas and compressibility when T < Tc .

9.11 Projects
 Fermi and Bose gases
10

10.1 Gibbs factor and Gibbs sum

We have already developed a microscopic theory for systems where the

temperature, volume and number of particles are constant (T, V, N ).
Can we develop a similar theory for systems where particles can diffuse,
so that N is no longer constant, but where the chemical potential is
constant (in equilibrium), that is, for systems described by (T, V, µ). We
call such systems grand canonical systems. Such as system is illustrated
in Fig. 10.1. The system consists of two parts, the system S and a large
reservoir R. The whole system, consisting of parts S and R, is isolated,
but the system S may exchange both energy and particles with the
reservoir.
This system is similar to the system we introduced when we discussed
the canonical system, but we have now opened for particle transport in
addition. For the canonical system we found that the probability for the
system to be in state i with energy i could be expressed as
1 −i /kT
P (i) = , Z(T, V, N ) = e−i /kT . (10.1)
X
e
Z i

where the partition function Z(T, V, N ) was introduced as a normalization

constant. We could use this theoretical prediction for the probabilities of
states i to calculate average values for the energy, and we were able to
relate the partition function to Helmholtz free energy.

371
372 10 Fermi and Bose gases

How can we generalize this microscopic theory to a system S in contact

with a reservoir R, but where the contact allows both exchange of energy
and exchange of particles? Since the whole system in Fig. 10.1 is isolated,
the total energy and the total number of particles are conserved, that is

E = ES + ER = const. , N = NS + NR = const. , (10.2)

are both constants. Let us now at a particular case where system S has
NS particles and is in a state i with energy i . (Notice that the states
and the energies of the states depend on the number of particles in the
system). How can we find the probability for this state?

TR

ΔE ΔN

TS

Fig. 10.1 Illustration of a system S in contact with a reservoir R, where the system is
allowed to exchange both energy and particles with the reservoir.

We know that in the isolated system, consisting of both the system

S and the reservoir R, the probability of every microstate is the same
— this is the fundamental assumption of statistical mechanics. We can
therefore find the probability of a particular macrostate in this system
by counting the number of microstates in the macrostate. We call ΩS
the multiplicity of system S, ΩR , the multiplicity of the reservoir. The
probability for system S to be in a state i is therefore
ΩS ΩR
P (NS , i) = P , (10.3)
ΩS ΩR
where the sum is the total number of microstates in the whole system
(R+S). If we now address a situation where the number of particles
in system S is NS and the system is in a specific state i, there is only
10.1 Gibbs factor and Gibbs sum 373

one way this can be achived, and the multiplicity ΩS of system S is

1. However, the energy of the reservoir is then ER = E − i , and the
number of particles is NR = N − NS , and there are many states with
this combination.
How can we estimate the multiplicity for the reservoir? We use the
same trick as we did for the canonical system. First, we look at the
logarithm of the multiplicity, SR /k = ln ΩR , and then we realize that the
deviations NS and i are small compared with the energy and number of
particles in the reservoir, therefore we can Taylor expand the entropy of
the reservoir around the value for N, E:
∂SR ∂SR
   
k ln ΩR = SR (N −NS , E−i ) ' SR (N, E)−NS −i .
∂NR E,V ∂E E,N
(10.4)
We recognize the partial derivatives from Chap. 6:
∂SR µR ∂SR 1
   
=− and = . (10.5)
∂NR ER ,VR TR ∂E ER ,NR TR

We therefore get
NS µR i
k ln ΩR = SR (N − NS , E − i ) ' SR (N, E) + − , (10.6)
TR T
and therefore the multiplicity is

ΩR = C e(NS µR −i )/kT , (10.7)

where C is a constant. We insert this back into expression in (10.3) for

the probability for a state (NS , i) :

ΩS ΩR 1 × C e(NS µR −i )/kT

P (NS , i) = P ' P = C 0 e(NS µR −i )/kT ,
ΩS ΩR ΩS ΩR
(10.8)
where the constant C 0 can be determined by normalization of the prob-
ability, where we need to sum over all possible states for the system S:
This means summing over all possible values for the number of particles
NS in system S, and for each value of NS we need to sum over all the
states i(S) of the system given that the system has NS particles:

C 0 e(NS µR −i )/kT = 1 ⇒ (1/C 0 ) = e(NS µR −i )/kT . (10.9)

XX XX

NS i NS i
374 10 Fermi and Bose gases

We call this normalization constant Gibbs sum or the Grand partition

function, denoted by ZG :

The probability for a system in diffusive and thermal equilibrium

with a thermal bath to be in a state (N, i) is given by
1 (N µ−i )/kT
P (N, i) = e . (10.10)
ZG
where Gibb’s sum is

ZG (T, V, µ) = (10.11)
XX
e(N µ−i )/kT .
N i

It is important to realize how this sum is performed: over all particle

numbers N , and for each particle number, N , over all the possible states
for the systems when it has N particles. Its use become clearer through
the following examples.

10.1.1 Averages using Gibbs sum

When we know the probability for all the possible states of the system,
we can use this to find the average of any macroscopic quantity using
the definition of the average:

Average of a macroscopic variable in a grand canonical system

described by (T, V, µ):

hXi = X(N, i)P (N, i) . (10.12)

XX

N i

Notice that both the energy, E, and the number of particles, N , now
are fluctuating quantities, but for macroscopic systems the values will
typically be sharp.
For example, the average number of particles is:
1 XX
hN i = N e(N µ−i )/kT . (10.13)
ZG N i
10.1 Gibbs factor and Gibbs sum 375

To simplify this expression, we can use the “derivative trick” we used

earlier, but now take the derivative with respect to µ:
d (N µ−s )/kT
N e(N µ−s )/kT = kT e , (10.14)
dµ
and therefore
1 d ∂ ln ZG
hN i = kT ZG = kT . (10.15)
ZG dµ ∂µ
This shows that we again find macroscopic quantities directly from the
grand partition function, ZG , just like we did for the partition function
for canonical systems.

10.1.2 Gibb’s sum for systems with several species of

particles
What if we have more that one species of particles in the system? For
example a gas like air may consist of both N2 and O2 molecules. In
this case, the complete system (reservoir and system), is described by
(T, V, N1 , N2 , . . . , Nk ) for the k different species. Similarly, we can intro-
duce the chemical potential for each species as
!
∂F
µj = . (10.16)
∂Nj V,T,Ni 6=Nj

Following exactly the same arguments are above, the probability for a
state with Nj particles of each of the species and for a state i of this
system is
1 (
P
Nj µj −i )/kT
P (i, N1 , N2 , . . . , Nk ) = (10.17)
X
e j ,
ZG (N1 ,N2 ,...,Nk )

where the Gibb’s sum now is

P
( Nj µj −i )/kT
ZG (T, V, (N1 , N2 , . . . , Nk )) = (10.18)
X
e j .
(N1 ,N2 ,...,Nk )

We demonstrate the use of Gibb’s sum for both a single species and for
several species through the following example.
376 10 Fermi and Bose gases

10.1.3 Example: CO poisoning

A hemoglobin molecule is illustrated in Fig. 10.2. Each hemoglobin
molecule has four independent adsorption sites, each consisting of a Fe2+
ion, and each site can couple to one O2 molecule. Find the probability
that the site is occupied with an O2 molecule, when you assume that
the hemoglobin in the blood is in equilibrium with O2 in the air in your
lungs, at a temperature of T = 310 K, and where the partial pressure of
O2 is pO = 0.2 atm.

(a) (b) (c) (d)

O O
O C

Fe2+ Fe2+ Fe2+

NO= 0 NO= 1 NO= 0

NCO= 0 NCO= 0 NCO= 1
εi= 0 εi= εO εi= εCO
Fig. 10.2 a Illustration of a hemoglobin molecule, b,c,d Illustration of the three states
of the molecule.

Analysis of one species system. We adress a system consisting of a

single Fe2+ ion in contact with a gas of O2 molecules. The system therefore
has two possible states, as illustrated in Fig. 10.2, either occupied by
oxygen (N = 0) or not occupied by oxygen (N = 1). Here, N , is the
number of oxygen molecules attached to the site, and i is the energy
of the system. The possible energy states are: For N = 0 there is only
one possible state of the system with energy i = 0. For N = 1 there is
only one possible state of the system with energy  = −0.7 eV. (Notice
that we could also have studied excited states, which would have added
more possible states for the N = 1 case, but we disregard this possibility
here.)
To find the average number of attached O2 molecules, we first find the
probability, and then calculate the average number of N for the system.
To find the probability, we first find the grand partition function:

ZG = e| (0 µ−0)/kT
{z } + e| {z } .
(1 µ−)/kT
(10.19)
(N =0) (N =1)
10.1 Gibbs factor and Gibbs sum 377

We see that this function includes the chemical potential, µ. But we have
not been given a value for µ. How can we find this? We assume that in
the lungs, the blood is in diffusive equilibrium with the air in the lungs,
and we can therefore use the chemical potential for an ideal gas, where
we know that the density of O2 molecules can be related to the partial
pressure of O2 using the ideal gas law: pV = N kT , and N/V = p/kT ,
hence

µ = −kT ln(nQ /n) = −kT ln(nQ (T ) kT /p) ' −0.6eV , (10.20)

when T = 310K, which is the temperature in your body, and p = 0.2 atm,
which is the partial pressure for oxygen. This gives

e−(−µ)/kT ' e0.1 eV/kT ' 40 , (10.21)

which means that the probabilty to be occupied is

40
P (N = 1) = ' 0.98 . (10.22)
1 + 40

Table 10.1 Elements for the partition sum.

NO NCO i

0 0 0
1 0 O
0 1 CO

Analysis of two-species system. Now, what happens if CO is also

present, which also can be adsorbed at the same site. Now, there are
three possible states: unoccupied, occupied by O2 or occupied by CO.
And there are two species, the number of oxygen molecules, NO , and the
number of CO molecules at the site, NCO . We must therefore include
both species in the grand partition function. All the possible particle
number and the states of the system — all the terms that must be
included in the grand partition function — are listed in Table ??. The
partition function is therefore

ZG = e(0 µO +0 µCO −0)/kT + e(1 µO +0 µCO −O )/kT + e(0 µO +1 µCO −CO )/kT .
(10.23)
378 10 Fermi and Bose gases

Now we need numbers. CO is more strongly bound, so CO = −0.85 eV.

But what is the chemical potential? We could still use the expression for
the ideal gas, but with the concentration nCO of CO in air. If CO is x
times less abundant that oxygen, we would find that
nQ nQ nQ
µCO = −kT ln = −kT ln = −kT ln + kT ln x , (10.24)
nCO x nO nO
where kT ln 100 = 0.12 eV, so that µCO = 0.72 eV. This gives for the new
Gibbs factor:
e(1 µCO −CO )/kT = 120 , (10.25)
and therefore
40
P (O) = = 0.25 . (10.26)
1 + 40 + 120
So just a small amount of CO is devastating!

10.1.4 Example: Diffusive equilibrium in the Einstein crystal

10.1.5 Example: Vacancies
10.2 Fermi-Dirac and Bose-Einstein distribution

When we discussed the ideal gas we assumed that quantum effects were
not important. This was implied when we introduced the term 1/N ! for
the partition function for the ideal gas, because this term only was valid
in the limit when the number of states are many compared to the number
of particles.
Now, we will address the general case of a quantum gas. The concepts
and methods we introduce here will be important for your understanding
of applied problems in quantum mechanics, such as electrical conducitiv-
ity, solar power cells, thermo-electric materials, neutron stars, and many
other processes.
The system we will study is a system of non-interacting particles in a
box of size L × L × L = V . For a single particle in a box, we know that
the possible translational states of the particle have energies
 2 
~2 π 
(nx , ny , nz ) = n2x + n2y + n2z
2m L (10.27)
 
=a n2x + n2y + n2z = an ,
2

where
10.2 Fermi-Dirac and Bose-Einstein distribution 379

~2 π 2 h2
 
a= = , (10.28)
2m L 8mV 2/3
and m is the mass of the particle. The three positive integers nx , ny ,
nz enumerates all the possible states of the system. That is, the state
of one particle in a box is given by (nx , ny , nz ), where nx = 0, 1, 2, . . .,
ny = 0, 1, 2, . . ., nz = 0, 1, 2, . . ..
This describes a the states of a single particle in a box. However, if
we have many particles in the same box, we need to address whether
two particles can occupy the same quantum state in the system. This
depends on the type of particle:
• For Fermions (1/2 spin) Only 0 or 1 fermion can be in a particular
state.
• For Bosons (integer spin) Any number of bosons can be in a particular
state.
Given this condition, we will assume that there are no other interactions
between the particles and that they can occupy any of the possible
quantum states of the system.
How can we find the thermodynamics of this system using a statistical
physics approach? We will apply a simplification that allows us to address
each state separately: Let us assume that we study a grand-canonical
system, that is, we will assume a system at constant temperature, T ,
volume, V , and chemical potential µ. We will then use a “trick” – we
will look only at a single state in the system, find the probability that
this state is occupied, and then sum the occupation numbers of all states
to find the total number of particles in the system and the total energy
of the system.
Let us first look at a system of Fermions with spin 1/2, such as
the electrons in an electron gas. In this case, there can only be one
particle in each state. However, for each translational state given by
(nx , ny , nz ) we will have 2 spin states, σ = ±1/2. Let us pick one such
state, (nx , ny , nz , σ) with energy (nx , ny , nz ) ( For example, the state
(1, 0, 1, −1/2) with energy  = a(12 + 02 + 12 ) = 2a). We study the
occupancy of this state using the Gibbs sum approach. The Gibbs sum
is the sum over the possible number of particles in this state, and for
each number of particles, we sum over all the possible states of the
system given this number of particles. In this case these sums are simple.
The system may have either N = 0 particles or N = 1 particle. For
N = 0 the system consisting of the state (nx , ny , nz , σ) can only have
380 10 Fermi and Bose gases

one possible energy – zero. For N = 1, the energy of the system is simply
(nx , ny , nz , σ), which we write at  for short. The Gibbs sum is therefore
1 X
ZG = e−(s(N ) −N µ)/kT
X

N =0 s(N )
(10.29)
= e−(0−0µ̇)/kT + e−(−1µ̇)/kT
= 1 + e−(−µ)/kT .

In the Gibbs sum formalism, the probability to have N particles in the

system and for the system to be in the state s(N ) is
1 −(s(N ) −N µ)/kT
P (s(N ) , N ) = e ., (10.30)
ZG

The average number of particles in the state s = (nx , ny , nz , σ), N̄s is

then
1
N̄(nx ,ny ,nz ,σ) = P (s(N ) , N )N
X

N =0
1 −(−µ̇)/kT
=0·1+1· e
ZG (10.31)
e−(−µ)/kT
= −(−µ)/kT
e +1
1
= (−µ)/kT .
e +1
This quantity – the average number of particles in a state (nx , ny , nz , σ)
– is called the Fermi-Dirac distribution function:
1
N̄(nx ,ny ,nz ,σ) = fF D (; µ, T ) = . (10.32)
e(−µ)/kT + 1
Because the average number of particles in a state (nx , ny , nz , σ) only
depends on the energy of this state, and not on any other details of the
state, we write the average number as a function of the energy  of the
state alone.
What does the average number of particles – the Fermi-Dirac distri-
bution function – look like? First, let us look at the functional form. We
see that if we introduce x = ( − µ)/kT , the functional form is
1
f (x) = , (10.33)
ex + 1
10.2 Fermi-Dirac and Bose-Einstein distribution 381

This is a function that provides a smooth transition from a value of 1 when

x  −1 to 0 when x  +1. We have plotted fF D (; µ, T ) as a function of
( − µ)/kT in fig. 10.3. The temperature effectively determines the range
of the transition region. We can illustrate this by introducing an energy
scale in the plot. Let us use the chemical potential µ as the energy scale1 ,
that is, we plot fF D (; µ, T ) as a function of /µ for different values of T
corresponding to kT = 0.2µ, kT = µ and kT = 5µ. We see that when
the temperature goes towards zero, the function approximates a step
function, and when the temperature increases the transition becomes
broader. These plots were generated by the following short script:
% Figure a
subplot(2,1,1)
x = linspace(-10,10);
f = 1.0./(exp(x)+1);
plot(x,f)
xlabel(’(\epsilon-\mu)/kT’);
ylabel(’f(\epsilon;\mu,T)’)
% Figure b
ktvalues = [5.0 1.0 0.2];
legarray = [];
subplot(2,1,2)
for i = 1:length(ktvalues)
kt = ktvalues(i);
x = linspace(-20,20);
f = 1.0./(exp((x-1)/kt)+1);
plot(x,f); hold all
legarray = [legarray; sprintf(’$kT=%3.1f\\cdot \\epsilon$’,kt)];
end
xlabel(’$\epsilon/\mu$’);
ylabel(’$f(\epsilon, \mu,T)$’)
legend(legarray);

What happens when T = 0? We see from the plots that the function
approaches a step function that goes from 1 to 0 at the chemical potential.
We call the chemical potential at T = 0 the Fermi energy:

F = µ(T = 0) . (10.34)

At T = 0 all the states up to the level F are occupied – and none of the
levels above F are occupied. Later on we will see how we can relate F
to the number of particles N in the gas.
1
Notice that here the chemical potential is a given, constant, whereas further on, we
will assume that the number of particles is constant instead, and the calculate a chemical
potential that will depend on T , V , and N .
382 10 Fermi and Bose gases

0.8

f ( ϵ; µ , T )
0.6

0.4

0.2

0
−10 −8 −6 −4 −2 0 2 4 6 8 10
( ϵ − µ ) /k T

1 kT = 5. 0 · ϵ
kT = 1. 0 · ϵ
kT = 0. 2 · ϵ
0.8
f ( ϵ, µ , T )

0.6

0.4

0.2

0
−20 −15 −10 −5 0 5 10 15 20
ϵ/µ
Fig. 10.3 Plot of Fermi-Dirac distribution.

10.2.1 Bose-Einstein distribution

Bosons are particles of integer spins. For Bosons, more than one particle
can be in the same state. Examples of bosons are photons, phonos, and
particles such as Helium-4. The Gibbs sum and the average number of
particles per state is different for Bosons than for Fermions because there
is no limit on the number of particles in a given state for Bosons. This
means that the number of particles, N , may run from 0 to ∞. The energy
s(N ) of the system when there are N particles in a state s (where each
particle has an energy s ) is N s . The Gibbs sum is therefore
∞ X
ZG = e−(s(N ) −N µ)/kT
X

N =0 s(N )
∞
= e−(N s −N µ)/kT (10.35)
X

N =0
∞
1
= e−N (s −µ)/kT =
X
,
N =0
1− e−(s −µ)/kT
10.2 Fermi-Dirac and Bose-Einstein distribution 383

where the sum is the well-known geometric series. The average number
of particles in state s is
XX N
N̄s = e−(s(N ) −N µ)/kT
N
Z
s(N ) G
1 X
= N e−(N s −N µ)/kT
ZG N
1 X
= N e−N (s −µ)/kT (10.36)
ZG N
1 X ∂ −(s −µ)N/kT
= kT e
ZG N ∂µ
∂
= kT ln ZG ,
∂µ
where we have used a usual “trick” by introducing the derivative and
taking it outside the sum. The result is
1
N̄s = fBE (s ; µ, T ) = . (10.37)
e(s −µ)/kT −1
Because the distribution only depends on the energy s of the state s of
the particle-in-box system, it is common to simply write the distribution
is a function of . We have plotted the Bose-Einstein distribution along
with the Fermi-Dirac distribution in figure 10.4.
% Plot FD
kt = 1.0;
legarray = [];
x = linspace(-10,10);
f = 1.0./(exp(x/kt)+1);
plot(x,f)
hold all
legarray = [legarray; ’FD’];
x = linspace(0.4,10);
f = 1.0./(exp(x/kt)-1);
plot(x,f)
legarray = [legarray; ’BE’];
x = linspace(-1.0,10);
f = exp(-x/kt);
plot(x,f)
legarray = [legarray; ’C ’];
xlabel(’(\epsilon-\mu)/kT’);
ylabel(’f(\epsilon, \mu,T)’)
legend(legarray);
ax = axis();
ax(4) = 2.0;
384 10 Fermi and Bose gases

axis(ax);

2
FD
BE
C

1.5

f (ϵ, µ, T )
1

0.5

0
−10 −5 0 5 10
(ϵ − µ)/k T
Fig. 10.4 Plot of Bose-Einstein distribution.

We see that the distribution diverges as  approaches µ, that is, as

 − µ approaches 0. This means that as the temperature goes towards
zero, all the particles will be in the ground state.

10.2.2 Classical limit

A gas is in the classical limit when the number of particles in a given
state is much smaller than one. For a gas at room temperature and
atmospheric pressure, the typical number of particles in each state is less
than 10−6 . In this case, the differences between Fermions and Bosons are
not important – these differences are only important when the number
of particles is one or larger This means that the distribution function
f (; µ, T ), which gives the number of particles in a state with energy ,
also must be very small, f  1. This implies that the exponential term
must be very large. In that case, we can assume that it is much larger
than 1 and the ±1 in the denominator does not matter. In the classical
limit we therefore get that the behavior of a Boson gas and a Fermion
gas are identical. The classical distribution function is

fclassical (, µ, T ) ' e(−µ)/kT , (10.38)

10.2 Fermi-Dirac and Bose-Einstein distribution 385

We have plotted all the distributions functions in the same plot in fig. 10.4,
where we see that all the distribution functions are indistinguishable for
large values of ( − µ)/kT .
Let us now compare this result with our results for a classical ideal
gas in a canonical system, where the chemical potential is a function of
T, V, N : µ = µ(T, V, N ). Since we have now addressed the system using
the Gibbs formalism, the (average) number of particles in the system
is a function of T , V , and µ, N̄ = N̄ (T, V, µ). How can we compare the
two results? We can calculate the average total number of particles, N̄ ,
as a function of the chemical potential in the T, V, µ system, and then
solve the resulting equation to find the chemical potential as a function
of the average total number of particles. (We may not always be able
to solve this equation analytically, but we may then resort to numerical
methods and still save the day).
For a given chemical potential, µ, the total average number of particles,
N̄ , is the sum of the average number of particles in each state, s, summed
over all possible states s:

N̄ = N̄s = f (s , µ, T ) , (10.39)

X X

s s

In the classical limit, we have argued that the distribution function

f (; µ, T ) has a simple form, so that the sum is:

N̄ = exp((s − µ)/kT = eµ/kT (10.40)

X X
es /kT .
s s

The last sum is the sum over all states, s, for a particle in a box.
We recognize this as the one-particle partition function, Z1 , for the
translational motion of one particle in a box:

N̄ = eµ/kT Z1 , (10.41)

and the chemical potential therefore is:

µ = kT ln(N̄ /Z1 ) . (10.42)

For an ideal gas - that is for a particle in a box - we found that the
one-particle partition function is

Z1 = nQ V , (10.43)

where
386 10 Fermi and Bose gases

mkT 3/2
 
nQ = , (10.44)
2πh2
was called the quantum concentration. The chemical potential is therefore

µ = kT ln(N̄ /(nQ V )) = kT ln(n̄i/nQ ) , (10.45)

where n̄ = N̄ /V .

10.3 Fermi gases

We can use a similar approach to determine the number of particles in a

Fermi gas: We find the average number of particles in each state for a
particle in a box, and then find the total average number of particles as
the sum of the average number of particles in each state:

N̄ = f (s ; µ, T ) , (10.46)
X

where the sum is over all the states for a single particle in a box.
However, we are usually instead interested in understanding the be-
havior in a system with a given number, N , of particles and not a given
chemical potential. How can we transform the results we have for a given
chemical potential to a system with a given number of particles? Since
we know how to calculate the (average) number of particles, N̄ , in a
system, we can simply find the chemical potential, µ(V, T, N ), for which
the average number of particles, N̄ , is equal to the number of particles,
N : N̄ = N . From a more practical point of view, this means that we
first calculate N̄ = N̄ (T, V, µ) for a given µ, and then find µ from the
equation
N = N̄ = N̄ (T, V, µ) . (10.47)

10.3.1 Zero temperature limit

Let as start at the simplest case. At very low temperatures, where T = 0,
the Fermi distribution is a step function. This means that all the states
below the chemical potential at T = 0 – the Fermi energy – are occupied
by one particle, and none of the states above the chemical potential are
occupied. (We recall that the chemical potential at T = 0 is called the
Fermi energy, F = µ(T = 0). )
10.3 Fermi gases 387

In this case, it is simple to find the number of particles in the system.

Since all the states below F have one particle in them, we simply need
to count the number of states with energies below F and this will be
the (average) number of particles in the gas.
In general, we find the average number of particles in the gas from

N̄ = N̄s = f (s ; µ, T ) , (10.48)

X X

s s

where the sum is over all the states s of the system. The states of a
particle in a box are given as (nx , ny , nz , σ). We can sketch the states in
the nx , ny , nz space as illustrated in fig. 10.5. We see that for each set of
integers nx , ny , nz we have two states. If we look at a “volume” ∆V =
∆nx ∆ny ∆nz in this space, we know that ∆nx = ∆ny = ∆nz = 1 we will
have 2 states in this volume (as long as the volume is in octant where nx ,
ny , and nz all are positive). We can therefore introduce a density of
states in the nx , ny , nz -space: D(nx , ny , nz ) so that the number of states,
∆N , in the volume from nx , ny , nz to nx + ∆nx , ny + ∆ny , nz + ∆nz is

∆N = D(nx , ny , nz )∆nx ∆ny ∆nz . (10.49)

The density of states for this problem – a particle in a box – is uniform

in nx , ny , nz -space:
(
2 when nx > 0, ny > 0, nz > 0
D(nx , ny , nz ) = . (10.50)
0 otherwise

The number of states in a volume V is therefore simply D(nx , ny , nz )V =

2V in nx , ny , nz -space.

Fig. 10.5 Illustration of the states of a particle in a box. Each vertice of a voxel represents
a single state. The three figures illustrates the states that are below nF for nF = 3, 7, 11.
388 10 Fermi and Bose gases

For T = 0 we wanted to count all states with energies below F . The

energy of a state is
 
(nx , ny , nz ) = a n2x + n2y + n2z = an2 , (10.51)

which means that the energy only depends on the distance n from the
origin in nx , ny , nz -space. The condition that  ≤ F therefore corresponds
to
 ≤ F
an2 ≤ F (10.52)
r
F
n≤ = nF .
a
To find the number of states with  ≤ F we must therefore count the
number of states in the nx , ny , nz -space with n ≤ nF , that is, we must
count all the states inside a sphere with radius nF .
Let us follow two approaches to find the number of states in this
volume. First, we simply use that the denstiy of states in nx , ny , nz -space
is uniform, and that the number of states therefore is the volume of the
sphere with radius nF multiplied with the density of states which is 2,
taking into account that only positive values of nx , ny , nz . Using this
argument, the number of states inside nF is
1 4π 3 π 3
N̄ = 2 · VF = |{z}
2 · · nF = n . (10.53)
8 |3{z } 3 F
2 spins |{z}
nx ,ny ,nz >0 volume of sphere

However, it is instructive to calculate N̄ following a more formal

approach, since this approach can be generalized to the case where T > 0.
The total average number of particles in the system is the sum of the
average number of particles in each state:

N̄ = N̄s = f (s ; µ, T ) , (10.54)

X X

s s

where the states now are given as (nx , ny , nz , σ):

N̄ = f ((nx , ny , nz ); µ, T ) . (10.55)
XXXX

nx ny nz σ

If we assume that the sum will be over many states, we can approximate
this sum by an integral over (nx , ny , nz ), where we also must include the
10.3 Fermi gases 389

density of states:
ZZZ
N̄ = f ((nx , ny , nz ); µ, T )D(nx , ny , nz )dnx dny dnz . (10.56)

For T = 0, f () becomes a step function, and we only need to include

the nx , ny , nz values that satisfy n ≤ nF :
ZZZ
N̄ = D(nx , ny , nz )dnx dny dnz . (10.57)
n≤nF

Now, we can use our knowledge from calculus to spherical coordinates

in the nx , ny , nz space. The integral transformation needed to go from
Cartesian to spherical coordinates is to replace dnx dny dnz = 4πn2 dn.
However, when we integral in spherical coordinates, we must remember
that we only include 1/8 of the sphere since we only allow positive
coordinates. The integral transform is therefore
nF 4π 2
ZZZ Z
N̄ = D(nx , ny , nz )dnx dny dnz = D(nx , ny , nz ) n dn .
n≤nF 0 8
(10.58)
We insert D(nx , ny , nz ) = 2, getting:
nF 4π 2 nF
Z Z
N̄ = 2 n dn = πn2 dn = . (10.59)
0 8 0

From this equation, we see that we can interpret πn2 as the density of
states in n-space (which here is different from nx , ny , nz -space): D(n) =
πn2 .
This integral can be easily solved:
nF πn3F
Z
N̄ = πn2 dn = . (10.60)
0 3
This is, of course, the same as we found in (10.53).
Solving for the chemical potential. We insert nF = F /a to find the
p

average number of particles in terms of the chemical potential at T = 0:

3/2
π 3 π F
N̄ = nF = . (10.61)
3 3 a3/2
We can solve for F , getting:
390 10 Fermi and Bose gases

3N̄
!
3/2
a3/2 = F , (10.62)
π

and therefore !2/3

3N̄
a = F . (10.63)
π
We recall that a also includes the volume V of the system:

h2
a= , (10.64)
8mV 2/3
which gives
!2/3 !2/3
3N̄ 3
 2/3
h2 h2 N̄
F = = . (10.65)
π 8mV 2/3 8m π V

We repeat the argument that lead to this conclusion: At T = 0 all the

states up to the chemical potential are occupied by one particle. We
can therefore find the average number of particles, N̄ , in the gas when
the chemical potential is given, by summing up how many staes we
have below the chemical potential. We reverse this argument, and find
the chemical potential given the number of particles, which gives us
µ(T = 0) = F as a function of N̄ (and V ) - which is what we found
above.
Average energy at T = 0. We can also find the average energy of the
gas at T = 0 using a similar approach. The average energy of a state
(nx , ny , nz , σ) is N (nx , ny , nz , σ) · (nx , ny , nz , σ). We can therefore find
the total average energy by summing the average energies for all the
states:

Ē = N (nx , ny , nz , σ) · (nx , ny , nz , σ) . (10.66)

X

(nx ,ny ,nz ,σ)

The energy only depends on the magnitude of (nx , ny , nz ),  = an2 , we

can therefore replace the sum with an integral over n:
Z ∞
Ē = an2 f (; µ, T )D(n)dn , (10.67)
0

where D(n) is the density of states in the n-space and D(n)dn gives the
number of states in the interval from n to n + dn.
10.3 Fermi gases 391

How can we solve this integral? We will work more with this in general
in the following, but for the T = 0 case, the integral reduces to a simple
form because f (; µ, T = 0) = 0 when n > nF and 1 otherwise. The
integral therefore becomes:
Z nF
Ē = an2 D(n)dn , (10.68)
0

where D(n)dn = πn2 dn:

Z nF
Ē = an2 πn2 dn
0
Z nF
= πa n4 dn
0
π
= an5F
5 (10.69)
 π 2
 
= an2F · n3F ·
| {z } | 3{z } 5
=F
N̄
2
= N̄ F .
5
(Notice that we used the result N̄ = (π/3)n3F from (10.61)). This result
is surprisingly simple! This simplicity is not that surprising: Since we
are at T = 0 we have filled up all the states up to the level F , starting
from the states with lowest energy. Because of the Pauli principle we
can only have one Fermion in each state, and the particles will therefore
be distributed across a range of states with a range of energies. You
can gain some inituition about these states from the illustration of the
states in nx , ny , nz -space in fig. 10.5. However, since the energies of the
states increases rapidly with distance from the origin,  = an2 , most of
the energy comes from the states far from the origin. The result is that
Ē = (2/5)N F , which means that many of the states are near F . We
will gain more intuition into this in the next section.
The result we have found also give us useful physical insights into
the quantuum effects on the behavior of a quantum gas. We see that
if we keep N constant for the gas, but compress the system (decrease
V ), then F will increase (from (10.65) we know that F ∝ (N̄ /V )), and
therefore the energy will increase. This effect gives rise the a repulsion
due to Fermi-exclusion.
392 10 Fermi and Bose gases

10.3.2 Density of states

We introduced the density of states in both nx , ny , nz -space and in n-
space in order to describe the number of states in a small interval around
a point (nx , ny , nz ) or n. The density of states in nx , ny , nz -space was
D(nx , ny , nz ). By this we meant that the number of states between
(nx , ny , nz ) and (nx + dnx , ny + dny , nz + dnz ) is

N = D(nx , ny , nz )dnx dny dnz , (10.70)

where we found that D(nx , ny , nz ) = 2 when nx , ny , and nz all are

positive, and zero otherwise.
We have similarly introduced a density of states in n-space - so that
the number of states with magnitude n between n and n + dn is
1
D(n)dn = 2 · · 4πn2 dn = πn2 dn , (10.71)
8
where the factor 2 again comes from the two spin states per n value and
the factor of 1/8 is included because only positive values of nx , ny , and
nz are used, hence only 1/8 of the volume of a sphere with radius n is in
the first octant. We therefore call D(n) = πn2 the density of states for
the three-dimensional ideal gas. Notice that the density of states would
be different for a one- or a two-dimensional gas - the n-dependence would
be different!
However, the density of states in n-space is not that useful since the
occupation number - f (, µ, T ) - is given in terms of . If we want to
calculate average values - such as the average number of particles or
the average energy - we perform the “sums” over (nx , ny , nz ) which we
convert to integrals of n (since all quantities only depend on the length of
(nx , ny , nz ) - and since the number of states is large so that the integral
is a good approximation for the sum):
Z ∞
N̄ = 2f ((nx , ny , nz ); µ, T ) = f ((n), µ, T )D(n)dn .
X

(nx ,ny ,nz ) 0

(10.72)
Similarly, we find the average energy from a similar integral:
Z ∞
Ū = 2(nx , ny , nz )f ((nx , ny , nz ); µ, T ) = (n)f ((n); µ, T )D(n)dn ,
X

(nx ,ny ,nz ) 0

(10.73)
10.3 Fermi gases 393

Now - we can evaluate this integral by inserting (n) = an2 . Or we can

change integration variable in the integral, and instead integrate over .
That is, we introduce the variable (n) and the differential:
d
d(n) = dn , (10.74)
dn
giving the integrals:
∞ 1
Z
N̄ = f (; µ, T )D(n()) d , (10.75)
0 d/dn
and ∞ 1
Z
Ū = f (; µ, T )D(n()) d . (10.76)
0 d/dn
From these expressions we see that we can interpret
1
D(n()) d = D()d , (10.77)
d/dn

as the density of states in  space. The quantity D()d gives the number
of states with energies between  and  + d.
Let us find the density of states in energy space for the three-
dimensional gas. We find it using the integral transformation we in-
troduced above:
D(n)dn = D()d , (10.78)
and therefore
1
D(n()) = D() , (10.79)
d/dn
where we now use that
 2
~ π
(n) = n2 = an2 , (10.80)
2m L
and therefore we find:
n = (/a)1/2 , (10.81)
and that
d
= 2an , (10.82)
dn
which we insert in (10.79) to get
394 10 Fermi and Bose gases

1
D() = D(n)
d/dn
1
= πn2
2an
π
= n
2a r
π 
= (10.83)
2a a
π
= 3/2 1/2
2a
π(8m)3/2 √
= V 
2h3
3N √
= 3/2  .
2F
The nice thing about this expression is that it can be interpreted in
the same plot - in the same space as we say here - as the distribution
function f . This is illustrated in fig. 10.6, where we have plotted both
the density of states D(), f (; µ, T ) and their product in the same plot.

2
f (ϵ)
f ( ϵ, µ , T ) , D ( ϵ)

D (ϵ)
1.5 f ·D

0.5

0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
ϵ/ϵ F
Fig. 10.6 Plot of the density of states, D(), the number of particles in a state with
energy , f (; µ, T ), and their product for kT = 0.25 · F .

10.3.3 Behavior at T = 0
We can use fig. 10.6 to gain more insight into the system at T = 0. In
this case, the distribution function f (; µ, T ) is a step function with the
step at F . We can therefore think of what happens at T = 0 in a slightly
different way: We can assume that we have a given number of particles
10.3 Fermi gases 395

N . These particles will fill up all the states, starting with the states with
the lowest energy first. Since two particles cannot occupy the same state,
we fill in the density of states, D(), function starting from  = 0 and
then up to F , as illustrated in fig. 10.7. In this case, it is simple to find
the average number of particles – it is the area under the D() curve
from 0 to F :
Z ∞
N̄ = f (; µ, T )D()d
0
Z F
= D()d
0
F
r (10.84)
π 
Z
= d
0 2a a
π 1 2 3/2
= √  ,
2a a 3 F

where we have inserted D() = (π/2a) /a. This result corresponds

p

to what we found previously. However, it is simple to see from the 1/2

dependence of D() that the average number of particles will depend on
3/2
F .

10.3.4 Qualitative behavior at low temperatures

We have found the chemical potential, F , at T = 0. How does the
chemical potential depend on temperature at low temperatures? We can
gain insights just from the behavior of D() itself. If we look at a gas with
a given number N of particles (such as a given number of free electrons
in an electron gas), then we can solve the equation
Z ∞
N = N̄ = f (; µ(T ), T )D()d , (10.85)
0

for µ(T ). What does this integral represent? It represents the area
under the curve f (; µ, T ) · D() as illustrated in fig. 10.7. Since the
area corresponds to the number of particles, and we assume that the
number of particles is conserved, the area must be conserved when the
temperature is increased from T = 0.
What happens as T is increased? Some of the particles in the states
below the chemical potential (area A) are moved into states above the
chemical potential (area B) as illustrated in fig. 10.7. However, the
number of particles removed from states below the chemical potential
396 10 Fermi and Bose gases

f ( ϵ, µ , T ) , D ( ϵ)
2.5

1.5

0.5

0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
ϵ/ϵ F

3
f ( ϵ, µ , T ) , D ( ϵ)

2.5

2 A

1.5

0.5
B
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6
ϵ/ϵ F
Fig. 10.7 (Top) At T = 0 all the states up to F are filled. (Bottom) For T > 0 the
particles in the states in area A have been moved into the the states B with higher energies
for kT = 0.25F .

(area A), must be equal to the number of particles added to states above
the chemical potential (area B) in order for the number of particles to
remain constant.
Now, if we assume that the chemical potential µ does not change when
T is increased, we see that area B will be larger than area A because
D() is an increasing function and f (; µ, T ) is symmetric around  = µ.
This is illustrated in the magnification shown in fig. 10.8. The top figure
shows what happens if we assume that the chemical potentia µ(T ) = F
also when T > 0. In this case the area B is larger than the area A. This
is not that easy to see from the figures, since the differences are rather
small.
How should µ(T ) depend on T in order to increase the area A and
decrease the area B? To get this argument right, it is important to realize
what areas we really are to compare. We have illustrated this in the
bottom figure in fig. 10.8. Here we have illustrated the chemical potential
by the dotted line. The chemical potential is smaller than F in this
10.3 Fermi gases 397

figure. We have also illustrated the relevant areas: It is the area below
F and above F we should compare. (Not the area below and above µ –
can you see the difference?). At T = 0 the area below D() corresponds
to the number of particles in the system.
In the top figure in fig. 10.8 we have argued that area B is slightly
larger than area A. In the bottom figure in fig. 10.8 we see that area A
is clearly larger than area B. This means that the correct value for µ lies
somewhere between µ = F in the top figure and µ = 0.99 · F in the
bottom figure. This means that µ must be smaller than F , but not as
small as illustrated in this figure.
Conclusion: µ(T ) must be a decreasing function of T near T = 0.
f ( ϵ, µ , T ) , D ( ϵ)

0
0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1
ϵ/ϵ F
f ( ϵ, µ , T ) , D ( ϵ)

2 A

1
B
0
0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1
ϵ/ϵ F
f ( ϵ, µ , T ) , D ( ϵ)

2 A

1
B
0
0.9 0.92 0.94 0.96 0.98 1 1.02 1.04 1.06 1.08 1.1
ϵ/ϵ F
Fig. 10.8 Plot of f (; µ, T = 0) · D() (top), f (; F , T = 0.1TF ) · D() (middle), and
f (; µ, T = 0.1TF ) · D() (bottom) where TF = F /k.
398 10 Fermi and Bose gases

10.3.5 Quantitative description of low temperature behavior

We can also devise a simplified quantitative argument to estimate how
the energy Ē depends on the temperature and then use this argument to
find how µ depends on T near T = 0. The average energy of the system
is Z ∞
Ē = f (; µ, T )D()d . (10.86)
0
Let us now assume that the number of particles, N , is given so that
µ = µ(T ). We can then find the heat capacity (at constant volume, since
we still work in a T, V system) from
!
dĒ
Z ∞ d
CV = = D() f (; µ(T ), T )d , (10.87)
dT 0 dT

where there are two T -dependencies in f – both a direct dependency and

an indirect dependency through µ(T ):

df (; µ(T ), T ) ∂f ∂f dµ
= + . (10.88)
dT ∂T ∂µ dT
This gives
Z ∞ ∂f dµ
Z ∞ ∂f
CV =  D()d +  D()d . (10.89)
0 ∂T dT 0 ∂µ

We can do the same for N̄ :

Z ∞
N̄ = f (; µ(T ), T )D()d , (10.90)
0

dN̄ ∞ ∂f Z
= D()d , (10.91)
dT 0 ∂T
Z ∞ ∂f dµ ∞ ∂f
Z
0= D()d + D()d . (10.92)
0 ∂T dT 0 ∂µ
(Where we have used that (dN̄ /dT ) is constant, since the number of
particles in the system does not change when we change the temperature
if the number of particles is constant). Up to now we have not done any
approximations. Now, we see that f is a rapidly varying function of 
around µ:
∂f e(−µ)/kT
=
2 . (10.93)
∂µ e(−µ)/kT + 1
10.3 Fermi gases 399

We can therefore approximate the second integral in (10.89) by

Z ∞ ∂f
Z ∞ ∂f
D() d ' µ D() d , (10.94)
0 ∂µ 0 ∂µ
where the right hand side now corresponds to the second integral in
(10.92). We can therefore replace
∂f µ
Z ∞ ∂f
µint∞
0 D()d = − D()d + . (10.95)
∂µ dµ/dT 0 ∂T

We replace this into the equation for CV in (10.92) getting

Z ∞ ∂f
Z ∞ ∂f ∂f
Z ∞
CV '  D()d − µ D()d =
D()d . ( − µ)
0 ∂T 0 0 ∂T ∂T
(10.96)
We will use the same approximation once more, but this time argue that
∂f /∂T falls off rapidly with  away from µ, but that D() varies slowly,
so that we can assume that D() is approximately constant and equal to
D(µ), simplifying the integral to:
Z ∞ ∂f
Z ∞ ∂f
CV ' ( − µ) D()d ' D(µ) ( − µ) d . (10.97)
0 ∂T 0 ∂T
Here, we need ∂f /∂T :

∂F ( − µ) e(−µ)/kT
=
. (10.98)
∂T kT 2 e(−µ)/kT + 1 2

Now, we are near an integral we can solve. We introduce a new integration

variable, x = ( − µ)/kT , so that d = kT dx, which gives:
∞ x2 ex
Z
CV ' D(µ)k T 2
dx , (10.99)
−µ/kT (ex + 1)2
where again the integral varies rapidly around x = 0, and the lower
bound therefore can be set to be approximately −∞, and we can solve
the integral exactly:
∞ x2 ex 1
Z
CV ' D(µ)k 2 T dx = k 2 T D(µ) π 2 . (10.100)
0 (ex + 1) 2
3

We may also approximate D(µ) ' D(F ), which gives

400 10 Fermi and Bose gases

3N 1 3 1
D(µ) ' D(F ) = = , (10.101)
2 F 2 N kTF
and therefore
1 T
CV ' π 2 N k = aN T , (10.102)
2 TF
where kTF = F , that is, TF = F /k, and a is a constant that does
not depend on N or T . A typical values for TF for an electron gas is
TF ' 5 · 104 K.
This long derivation shows that the heat capacity due to an electron
gas (or another Fermi gas) is proportional to T at low temperatures (T
small compared with TF ).

10.4 Photon Systems

Let us now use the theory we have developed to address the behavior of
a photon gas - a system with electromagnetic waves inside a container –
in equilbrium with the container walls. In this case the walls will absorb
and emit electromagnetic wave (packets) - so the number of photons is
not conserved.
For a container of length L, we assume that the field only consists of
standing waves (in equilbrium). These waves can therefore have frequen-
cies f = c/λ and corresponding energies  = nhf = n~ω, where n is an
integer corresponding to the state.
(The integer n comes from the number of half-wavelengths that make
up the length L: L = (n/2)λ, where n is an integer.)
This corresponds to the harmonic oscillator we have already studied.
We will now study the occupancy probability for these states. We can
assume that the system has constant volume V and temperature T , and
that since photons are continually created and destroyed - the chemical
potential for the system is therefore zero.
We can then use the Bose-Einstein distribution law:
1
fBE (, µ, T ) = , (10.103)
e(−µ)/kT −1
which we can simplify since µ = 0 to
1
fBE (, µ = 0, T ) = , (10.104)
e(/kT −1
10.4 Photon Systems 401

which is the same as we found for the harmonic oscillator in the canonical
ensemble.
This tells us the occupancy of a given energy level s . In addition we
need to know the density of states for the photon gas.
For particles (fotons) in a box, the possible (standing wave) solutions
are
2L hn
λ= , , (10.105)
n 2L
where n is a positive integer. The energy of the photon is
hcn
 = pc = ~ω = , (10.106)
2L
instead of  = p2 /2m for classical particles.
This means that the energy of a photon generally is proportional to n
while the energy of a classical moving particles (in an ideal gas / Fermi
gas) is proportional to n2 .
This is also true in three dimensions, where the momentum is inde-
pendent in the three direction, and equal to hc n/2L in each direction:
hc
p= (nx , ny , nz ) , (10.107)
2L
and the energy still is  = pc, where p is the magnitude of p:
hc  2 1/2 hcn
= nx + n2y + n2z = . (10.108)
2L 2L
In order to use the distribution function, we need to sum over all
possible states (and their corresponding energies) to find the number of
photons and the total energy of the photon gas.
Let us first look at the total energy - which we find by summing over
all possible n-values - including the effect of two possible polarizations:

U =2 (nx , ny , nz )fBE (, µ = 0, T ) , (10.109)

XXX

nx ny nz

where  = hcn/L. Instead of summing, we integrate over n in n-space:

∞ 4πn2 hcn 1
Z
U =2 dn . (10.110)
0 8 L ehcn/2LkT −1
We can now transform to a variable that simplifies the exponent, choosing
x = hcn/2LkT , which gives n = 2LkT x/hc and dn = 2LkT /hc dx:
402 10 Fermi and Bose gases

8π(kT )4 L3 ∞ x3
Z
U= dx , (10.111)
(hc)3 0 ex − 1

where the integral is π 4 /15. Energy per volume is therefore then

U 8π 5
= (kT )4 . (10.112)
V 15h3 c3
This is called the Stehan-Boltzmann law of radiation.
We can also find the frequency (or energy, since they are propotional
 = hν) distribution of the photon gas / blackbody radiation.
What does that mean? We can find for example how many photons are
in the range from ν to ν + dν, how many photons are in the range from
 to  + d - or preferrably – how much energy is in the corresponding
range of frequencies / energies: We find this as the number of photons in
the range multiplied by the energy per photon, .
We can read this directly from the integral, realizing that the integral
for the total energy can be written as
Z ∞ Z ∞
U= n D(n)dn = D()d , (10.113)
0 0

or if we are interested in the energy density (U/V) instead:

U
Z ∞ n
Z ∞ 
= D(n)dn = D()d , (10.114)
V 0 V 0 V
which is what we just found above.
Starting from
∞ 4πn2 hcn 1
Z
U =2 dn . (10.115)
0 8 L e hcn/2LkT −1
we can instead insert  = hcn/2L, getting
U ∞ 8π
Z
= d , (10.116)
V 0 (hc)3 e/kT −1
and we see that the energy density in the range  to  + d is

8π 3
u() = . (10.117)
(hc)3 e/kT − 1
We can express it instead using ν where  = hν:
10.4 Photon Systems 403

8πhν 3 1
u(ν) = dν . (10.118)
c3 ehν/kT − 1
This is called the Planck radiation law.
We can plot this distribution in dimensionless form by plotting x3 /(ex −
1) as a function of x = /kT .
This function has a maximum at x = 2.82, which corresponds to
 = 2.82 kT .
This shows that the maximum energy (or frequency) increases with
temperature. This law is called Wien’s law. (Wien’s displacement law).
This law implies that the temperature can been seen from the most
prominent color - since the frequency of this this color is proportional to
the temperature. (If we can consider the body we examine as blackbody
radiation).

10.4.1 Entropy of the photon gas

We can find the entropy, since for constant volume we have

dU 4aT 3
T dS = dU , dS = = , (10.119)
T T
which we integrate from 0 to T , getting:

1 32π 5
3
kT

S(T ) = 4a T 3 = V k. (10.120)
3 45 hc
(We can find the total number of photon using the same formula, but
with a different prefactor).

10.4.2 Radiation from a black body

What if we have a photon gas and we study the radiation emitted from
a small whole in the container.
We say that such a hole radiates as a black body.
The energy flux is the rate of energy emission per unit area.
How much radiates out in a time dt?
The energy in the volume dV which is cdt times the opening area A
times some geometric factor g (which is 1/4 - see textbook and problems).
The flux is therefore
404 10 Fermi and Bose gases

dE udV uAcdtg
J= = = = ucg . (10.121)
Adt Adt Adt
and
cU 2π 5 (kT )4
J= = = σT 4 , (10.122)
4V 15 (hc)3
where σ is called the Stefan-Boltmann constant.
And the law is called Stefan’s law.
This is from a black body - non-reflecting! - body with surface tem-
perature T .
Applications: Cosmic black body background radiation, Emission and
absorption (Kirchoffs law).

10.5 Phonon Systems

We have previously introduced the Einstein model - or the ideal crystal -

as our model for a crystal. But in this system there are no interactions
between the atoms/particles. Does this have any consequence?
For the Einstein crystal we found that the heat capacity was:
 2 /kT


e
CV = 3N k kT

2 . (10.123)
e/kT − 1

where N now is the number of atoms/particles, so that 3N is the number

of oscillators.
What happens when   kT for this model? In that range the heat ca-
pacity approaches a constant: 3N k. This corresponds to the equipartition
theorem.
What happens for small temperatures, when   kT ? In this range the
heat capacity goes to zero exponentially. This is not in correspondence
with experiments, which show that the heat capacity goes to zero at T 3
in the low temperature limit.
What went wrong?
We have ignored interactions among the atoms - which probably may
be important for lattice vibrations.
Mainly because there are different vibration modes in systems with
many atoms - the atoms may vibrate together to form low frequency
modes. At low temperatures the high frequencies (high energy) modes
are frozen out, but then the low frequency (low energy) modes become
more important. By only looking at single atoms we have overlooked
10.5 Phonon Systems 405

collective modes that have lower energy. Therefore the heat capacity goes
to zero slower than predicited by the Einstein model.
The vibration modes in a crystal resembles electromagnetic waves:
They are waves, but they have smaller velocities (much smaller of
course). We will here assume that the speed is a constant cs - even if it
acutally depends on the wave length in a crystal.
They have three polarizations. The transverse and one longitudinal.
The polarizations really have different velocities. At first we will ignore
this effect.
The waves cannot have all possible wavelengths, because the atoms
are on a lattice with a given lattice spacing, and we cannot resolve waves
with wavelength shorter than (two times) the lattice spacing.
Let us assume that we can otherwise describe a phonon - a lattice
vibration mode energy packet - just as we have described photons: with
uniformly spaced energy levels:
hcs hcs n
s = hν = = , (10.124)
λ 2L
where L is the system size. Again n is the magnitude of the n vector in
n-space.
We also assume that phonons are bosons with µ = 0, so that the
distribution function is given by the Bose-Einstein distribution:
1
fBE = . (10.125)
e/kT − 1
We find the energy and the number of phonons by summing over all
possible n values:

U =3 (n)f (, µ = 0, T ) . (10.126)

XXX

nx ny nz

The main difference with the crystal lattice is that not all values are
possible - we can only have some values for n.
Let us look at the x-direction.
Along this direction we have Nx = (N )1/3 atoms as shown in the
figure.
This puts at limit on the maximum upper number of n.
This should correspond to a cube in n-space.
The Debye approximation is to assume that we instead include all
modes up to a radius nD in n-space, but so that we ensure that the total
number of modes is equal to 3N :
406 10 Fermi and Bose gases

3 nD
Z
3 = 4πn2 dn = 3N , (10.127)
XXX

nx ny nz 8 0

This determines nD . Since the integral is simply the volume of a sphere

of radius nD we find that
3 4π 3
3N = n , (10.128)
8 3 D
and therefore that
6N 1/3
 
nD = . (10.129)
π
Now, we have a theory we can use to find the energy of the phonon
gas using the approach we now are used to:

U= n f (n , µ = 0, T )
XXX

nx ny nz
(10.130)
3 nD 2 hνn
Z
= n dn ,
8 0 exp(hνn /kT ) − 1
where
hcs n
hνn = . (10.131)
2L
The integral is therefore

3π nD hcs n3
Z
U= dn . (10.132)
2 0 2L exp(hcs n/2LkT ) − 1

We introduce the new integration variable x = hcs n/2LkT getting:

6N
1/3
hcs nD hcs TD

xD = = = , (10.133)
2LkT 2kT πV T
where we call TD the Debye temperature.
This gives - after some algebra:

9N kT 4 TD /T x3
Z
U= dx . (10.134)
TD3 0 ex − 1
This integral cannot be solved analytically, but it is easy to solve numer-
ically.
However, we can also find the high and low temperature limits directly
by approximations.
10.5 Phonon Systems 407

In the high temperature limit the upper bound of the integral is much
smaller than 1 and in this limit we can approximate ex = 1 + x, and the
integral becomes

9N kT 4 TD /T x3
Z
U= dx
TD3 0 x
9N kT 1 (10.135)
4 3
TD

=
TD3 3 T
= 3N kT .

This corresponds to the equipartition principle.

In the lower limit, T  TD , the upper limit is so large that we can
replace it by infinity (since the ex will go very quickly to zero, this is not
a problem). The integral is therefore the same as we did for the photon
gas - with value π 4 /15, and the total energy is

3π 4 N kT 4
U= , (10.136)
5 TD3
and the heat capacity in this limit is

12π 4
3
T

CV = Nk , (10.137)
5 TD
which agrees very well with experimental measurements.
We find the intermediate values by numerical integration.
For metals we need to include both the contribution from the phonos
and the contributions from the electrons, so that the heat capacity has
the behavior:
12π 4 N k 3
CV = γT + T , (10.138)
5TD3
when T  TD and γ = π 2 N k 2 /2F .
If we now plot CV /T as a function of T 2 we can check both constants
in the resulting plot, which should be linear in T 2 with an intercept
corresponding to γ.
What are typical values for TD ?
For lead 88K
For diamond 1860K
Above TD you can get away with using the equipartition theorem since
the heat capacity by then has reached 95% of its maximum value.
408 10 Fermi and Bose gases

10.6 Boson Gas and Einstein Condensation

We have now looked at photons and phonons. For these particles we do

not have a conservation law. What about other bosons, such as Helium-4?
In this case the chemical potential is not zero - we will need to determine
the chemical potential from the number of particles, just as we did for
Fermions.
We start in the limit of small temperatures.
When T = 0 all the atoms will be in the lowest energy state.
For atoms in a box of volume V = L3 the energy states are

h2  2 
= n + n2
+ n 2
, (10.139)
8mL2 x y z

so that the lowest energy state has energy

3h2
0 = , (10.140)
8mL2
which is a very small value for realistic (macroscopic) values of L.
The Bose-Einstein distribution gives the average number of atoms in
this state:
1
N0 = , (10.141)
exp((0 − µ)/kT ) − 1
As the temperature approaches zero, we know that N0 will be large,
which means that exp((0 − µ)/kT ) − 1 must be close to 1. This means
that the exponent is close to zero. In this limit, we can expand the
exponent using exp(x) = 1 + x, getting:
1 kT
N0 = = , (10.142)
1 + (0 − µ) /kT − 1 0 − µ
and therefore that µ = 0 when T = 0 and then just a bit larger when T
is small.
To make life simpler, let us change energy scales, so that 0 = 0. The
result is then
kT kT
N0 = − , µ=− . (10.143)
µ N0

10.6.1 Spacing of levels

What is really a realistic value for the spacing of energy levels?
10.6 Boson Gas and Einstein Condensation 409

The energy is
h2  2 
= n + n2
+ n2
(10.144)
8mL2 x y z

the energy difference between the lowest and the second lowest is therefore

3h2
∆ = (10.145)
8mL2
Now if we look at Helium-4, m = 6.6 × 10−24 g and L = 1cm, then
∆
= 1.8 × 10−14 K . (10.146)
k
This is a small splitting!
How could this play an important physical role at temperatures which
are Kelvin or at best a thousands of a Kelvin?

10.6.2 Approaching T = 0
What happens if we have a constant particle number, N , and we lower
the temperature? Then the number of particles in the lowest leverl, N0 ,
approaches N - all the particles are in the state with the lowest energy.
This means that
1
N0 = , (10.147)
exp((0 − µ)/kT ) − 1

approaches N (which is very large) when µ ' 0 .

For low temperatures we separate out the number of particles in the
lowest state, and write
∞
1
N = N0 + (10.148)
X
,
j=1
exp((j − µ)/kT ) − 1

where we have removed the singularity in the lowest energy level.

We can change this to an integral (since the energy levels are very
close), but then we need the density of states, D():
Z ∞ g()
N = N0 + d , (10.149)
1 exp(( − µ)/kT ) − 1

Since 1 also is very small ( 1 ' 0) and g(0) = 0, we can instead put
0 as the lower bound for the integral. In addition, we still assume that
410 10 Fermi and Bose gases

µ = 0: Z ∞ g()d
N = N0 , (10.150)
0 e/kT − 1
where we now insert for g():

2 2πm
3/2
√

g() = √ V , (10.151)
π h2
and we introduce the variable x = /kT . The integral is then reduced to

2 2πmkT
3/2 ∞ x1/2 dx
 Z
N = N0 + √ V , (10.152)
π h2 0 ex − 1
where the integral is
√
∞ x1/2 π
Z
dx = · 2.612 = 2.315 , (10.153)
0 e −1
x 2
and therefore we find
2πmkT
 3/2
N = N0 + 2.612 V . (10.154)
h2
This expression is only valid for low temperatures, but the second term
increases with temperature. What is the upper limit of validity of this
expression? That is when N0 is zero and all the particles are in higher
energy levels, this occurs at a temperature TC given by:

2πmkTC
 3/2
N = 2.612 V , (10.155)
h2
which gives
1
2/3
h2 N

kTC = . (10.156)
π(2.612)2/3 2m V
The critical temperature TC therefore depends both on details of the
particles, through m, and on the density N/V of the system. (for low
temperatures).
We can also use TC to rewrite the expression for N :
3/2
T

N = N0 + N , (10.157)
TC
which gives
10.6 Boson Gas and Einstein Condensation 411

3/2 !
T

N0 = N 1 − . (10.158)
TC
In the range T < TC we have Bose-Einstein condensation. At T = 0 all
the particles are in the lowest energy state.
What happens when T < TC ? In this case our calculation is no longer
valid. Instead we must include how the chemical potential varies with
temperature. We can do this by solving the equation
Z ∞ g()
N= d , (10.159)
0 e(−µ)/kT −1
numerically. We introduce new variables, x = /kTC , t = T /TC and
c = µ/kTC , and get the integral
∞ x1/2 dx
Z
2.315 = , (10.160)
0 e(x−c)/t − 1
which you now know how to solve.
 References

[1] N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, and A. H. Teller.

Equations of state calculations by fast computing machines. Journal
of Chemical Physics, 21, 1953.
[2] C. N. Yang. The spontaneous magnetization of a two-dimensional
Ising model. Physical Review, 85, 1952.

413
 Index

addition rule, 67 diffusive equilibrium, 302

algorithmic model, 20 dilute solutions, 350
arrow of time, 20
atomic potential, 15 Einstein crystal, 130, 193
attractive interactions, 339 engines, 274
average, 68 entropy of expansion, 163
average energy, 204 entropy of mixing, 164
average value, 202 environment, 271
equation of state, 272
Bernoulli process, 85 equation of state for an ideal gas,
binomial distribution, 85 117
Boltzmann statistics, 193 evaporation, 328
event, 61
canonical system, 193 excluded volume, 338
Central limit theorem, 98 exponential distribution, 100
chemical potential, 302 extensive, 272
Clausius’ law, 338 extensive properties, 206
Clausius-Clapeyron’s equation, 329 extensive property, 165
correlated, 61 extreme value statistics, 99
critical point, 328
cumulative distribution, 99 face-centered cubic lattice, 34
fair, 61
data-collapse, 146 fcc, 34
detailed balance, 137 fluctuations, 205

415
416 INDEX

free energy, 215 microstate, 26, 79

free expansion, 163 microstates, 122
frequency, 63 minimum free energy principle, 317
fundamental assumption, 137 mixture, 350
Molecular Dynamics, 32
Gaussian distribution, 92 molecular dynamics, 13
Gaussian function, 154 Monte Carlo methods, 138
Gibb’s paradox, 165 multiplicity, 78, 126
Gibbs free energy, 315 multiplicity function, 144
heat capacity, 205 negative rule, 67
heat engines, 274 non-interacting particles, 114
Helmholtz free energy, 215 Normal distribution, 98
normalization, 65, 66
ideal crystal, 130
normalization condition, 66
ideal gas, 114
normalized probability, 65
ideal gas law, 116, 117
numerical statistical experiment,
ideal mixture, 350
61
ideal solution, 350
independent, 67 outcome, 61
intensive, 272 Ovito, 48
interacting systems, 212
irreversible, 273 partition function, 199, 200
irreversible process, 273 periodic boundary conditions, 35
Ising model, 212 phase diagram, 328
isolated system, 193, 271 Pizza.py, 43
probability, 60, 63
LAMMPS, 41, 42 probability density, 95
lammps trajectory file, 49 probability distribution, 95
latent heat, 326 processes, 273
law of corresponding states, 344
Legendre transform, 317 quasi-static, 274
Lennard-Jones potential, 15, 33 quasi-static processes, 274
London interaction, 15
random number, 61
macrostate, 26, 79 random variable, 67
macrostates, 122 Raoult’s law, 356
MD, 32 repulsive interactions, 338
mean-field theory, 339 reversible, 273
melting, 328 reversible process, 273
microcanocical system, 193
sample space, 61, 67
INDEX 417

second law of thermodynamics, 154

sequence of random numbers, 61
sharp multiplicity function, 154
sharpness of canonical system, 206
simple cubic lattice, 34
solute, 350
solution, 350
solvent, 350
span sample space, 67
standard deviation, 74
state variable, 272
statistical experiment, 61
Stirling’s approximation, 89
sublimation, 328
system, 271

thermal contact, 271

thermal equilibrium, 272
thermally insulated, 271
thermodynamic limit, 146
thermodynamics, 271
triple-point, 328
two-state system, 203

uncorrelated, 61
uniform distribution, 100
unit cell, 34, 38
universal distributions, 98

van der Waals equation of state,

341
van der Waals force, 33
van der Waals interaction, 15
van der Waals interactions, 33
van der Waals model, 341
Vapor pressure, 314
variance, 74
Velocity Verlet, 36
VMD, 48