The Origin Story

Long before physicists came around to the idea, chemists believed in the theory of atomism, the idea that all matter is comprised of indivisible units (atoms). This theory suggests that, by studying the motions and behavior of these individual, “indivisible” units (we usually work with molecules, which very much can be divided, but that’s neither here nor there), we should be able to arrive at the understanding of a macroscopic system.

The problem with this approach is that there are too damn many particles for this to be remotely tractable. As an example, at standard atmospheric pressure (\(1~\text{atm} = 101325~\text{Pa}\)), one cubic centimeter of an ideal gas at \(T = 300~\text{K}\) contains roughly \(4.06 \times 10^{-5}\) moles of gas, which is roughly \(4.90 \times 10^{13}\) particles. Multiply by 6 (3 position components, 3 velocity or momentum components), and you wind up with over \(10^{14}\) variables that you need to solve for. That’s not happening! For perspective, even with modern state-of-the-art supercomputers, it’s rare to run simulations with more than 10,000,000 (i.e., \(10^{7}\)) atoms (let alone molecules), though simulations on the billion-atom scale (i.e., \(\sim 10^{9}\)) are possible[^1].

We need another way. We need another technique to understand macroscopic systems based on microscopic behavior. We need… statistical mechanics.

[^1] See, e.g., Jung et al., DOI: 10.1002/jcc.25840

Microstates, Macrostates, and Ensembles

We begin our journey into statistical mechanics by developing language and concepts with which to characterize (chemical, physical) systems.

A microstate is a description of the instantaneous state of every sub-component of a system. For a box of particles, a microstate would include all of their positions, momenta, energy levels, etc. But we can also describe other types of systems in terms of microstates. For instance, let’s say that we have a chessboard or go board (if you don’t like chess or go, pick your favorite board game). A microstate in a game of chess would be the position of every piece on the board. Similarly, in go, a microstate would be the occupation of every point on the board – is it occupied by a black stone, a white stone, or neither?

A macrostate is a description of a system overall in terms of macroscopic variables (go figure). In our box of particles example, a macrostate might describe the total energy, the number of particles, the temperature, the pressure, etc. In our board games, the macrostate could be the number (and, for chess, type) of pieces each player has on the board. A macrostate can be thought of an an abstraction (greatly reducing the number of variables used to define a system’s state), or as an equivalence class. That is, if two microstates can be described by the same macrostate, then they are in some way equivalent (specifically, in physical/chemical systems, they are equivalent in the sense that they cannot be distinguished by a macroscopic measurement).

Once we’ve figured out our microstates and our macrostates, the last puzzle piece that we need for statistical mechanics is the idea of an ensemble. An ensemble is basically a thought experiment; in general, no one can actually construct a statistical ensemble. In this experiment, we imagine that we have a collection of copies of some system of interest. Represented among these copies are instances of systems in every possible microstate. For the purposes of this class, we’ll be working with thermodynamic ensembles, which are ensembles where all of the systems are in equilibrium under some fixed conditions. For instance, we’ll talk about the microcanonical ensemble, where the energy \((U)\), number of particles \((N)\), and volume of each system \((V)\) are fixed, and we’ll talk about the canonical ensemble, where the tempertature \((T)\), \(N\), and \(V\) are fixed.

Though ensembles are fictitious idealizations, they are meant to model the idea that a scientist/experimenter/researcher/observer/being, performing the same experiment multiple times, cannot perfectly control the microscopic details of the system under study, and so will observe a range of possible outcomes.

Postulates and Approach of Statistical Mechanics

Like most good physical theories, the postulates of statistical mechanics are remarkably brief. There are a few different ways to formalize these postulates, but usually there are only two or three different assumptions that we need to make. With those assumptions in place, we can develop the whole theory of (equilibrium) stat. mech.

For this class, I’ve chosen to describe the postulates as follows:

1. Extension of microscopic laws to macroscopic systems: basically, we assume that the behavior of any macroscopic system can be described (at least in principle) using microscopic theories (e.g., quantum mechanics). If this postulate weren’t true, then the whole point of stat. mech. is moot; we wouldn’t be able to study systems of microscopic particles to understand macroscopis systems at all!
2. Principle of equal a priori probabilities: a system is equally likely to be in any microstate consistent with its macrostate. This postulate is perhaps less obviously true (and less obviously necessary) than the extension principle. There are a few different ways to justify equal a priori probabilities, but here, I’ll stick to the most basic. I just wrote above that two microstates that belong to the same macrostate (or, if you prefer, two microstates that can be described by the same macrostate) cannot be differentiated by a macroscopic measurement. And if they can’t be differentiated – if we can’t know by observing a system what microstate it’s in – we can’t really do any better than assuming they’re all equally likely, can we? This is known as the indifference principle. It’s one of the few examples in science of a useful, mostly-correct theory emerging out of shrugging your shoulders and embracing ignorance, which I find delightful.

That’s it! With those two tools, we can do stat. mech. Though there will be some variations, our general approach will be the following:

1. Define an ensemble appropriate to the system’s boundary conditions (we’ll go over that more through some examples over the next couple of lessons)
2. Within your chosen ensemble, determine the relationship between some known (or knowable) microscopic properties and your macroscopic quantities of interest (by the extension principle, such a relationship will always exist)
3. Take an average over all of the members of your ensemble

Doesn’t sound too bad, right?

Entropy

When you learn (macroscopic) thermodynamics, entropy is this strange, mysterious beast. Usually, we’re taught that it has something to do with disorder – more disorder, more entropy. But what is “disorder”? It doesn’t sound like a physical quantity. We learn that entropy is somehow related to heat, and we learn that entropy has a certain directionality to it (that is, according to our Second Law, the entropy of the universe is never decreasing).

Statistical mechanics allows us to pin down entropy and give it a (more) clear, more directly physical meaning. It has to do with (micro)states, specifically, the number of states available to a system.

It’s possible to rigorously derive an expression for the entropy, but I’m going to save you all of that math and keep it conceptual.

We know that a macroscopic system tends to move towards the maximum entropy consistent with its constraints. By our equal a priori probability postulate, we can also say that a system will tend to sample as many microstates as possible, i.e., the number of microstates consistent with the macrostate. This connection provides us with a link between the number of microstates available to a system and the system’s entropy.

Now, entropy is an additive quantity. If I have two systems, \(A\) and \(B\), the total entropy is the sum of the entropies of the two systems (\(S_{total} = S_A + S_B\)). But the number of states available is multiplicative. If there are three microstates available to \(A\) and three microstates available to \(B\), I don’t have six possible microstates for the total system; I have nine.

Remember our combinatorics discussion from Lesson 1! Verify, using this example of another of your choosing, that the number of possible combinations is multiplicative. For this example, where we’re trying to figure out “How many combinations of one microstate from \(A\) and one microstate from \(B\) are possible?” it might help to label the \(A\) states and \(B\) states differently (i.e., \(A_1\), \(B_2\), and so on).

From these considerations, we can hypothesize that the entropy has the form

\[ S = k_B ln(\Omega), \]

where \(k_B\) is a constant of proportionality (\(k_B = R / N_A\), where \(R\) is the ideal gas constant and \(N_A\) is Avogadro’s number), and \(\Omega\) is the number of microstates available to the system.

The Microcanonical Ensemble

The microcanonical ensemble, also known as the NVE ensemble, is an ensemble where each system has fixed energy, number of particles, and volume. It’s not the most practically useful ensemble (that title would probably go to the canonical, grand canonical, or Gibbs ensemble), but it is in many ways the simplest to work with.

In our microcanonical ensemble, there’s only one macrostate available, so our entropy is exactly the expression above (\(S = k_B ln(\Omega)\)). We also have the constraint that, because the total energy is fixed, \(\sum_i \varepsilon_i = E\), where \(E\) is the total energy, \(\varepsilon_i\) is the energy of sub-system \(i\), and the sum is over all components of the system. If we can quantify the number of states available (often based on the total energy and the energies available to sub-systems), we can quantify the entropy, and from there, many other properties of interest.

The “Polymer Model”

In class, we went over the two-state model using the microcanonical ensemble. Here, we’ll work through another example. It’s a bit artificial – despite the name, the “polymer model” doesn’t really help much with modeling real polymers – but it’s an illustrative example for educational purposes, at least.

Imagine a “polymer” chain made up of \(N\) monomers, each of unit length \(a\) (for simplicity, we’ll here say \(a = 1\).The polymer lies in a 2D plane, and each monomer can be oriented up, down, left, or right. The left end of the polymer is attached to a wall. and the right end is being pulled with some applied tension force \(\mathcal{T}\). Monomers facing left or right have an energy \(\varepsilon_{\text{left}} = \varepsilon_{\text{right}} = 0\), and monomers facing up or down have an energy \(\varepsilon_{\text{up}} = \varepsilon_{\text{down}} = \varepsilon\).

Some things to keep in mind as you work through this example:

• The total number of monomers is fixed, so \(N_{\text{left}} + N_{\text{right}} + N_{\text{up}} + N_{\text{down}} = N\).
• The total length in the \(x\) direction is \(L_x = N_{\text{right}} - N_{\text{left}}\)
• The total length in the \(y\) direction is \(L_y = N_{\text{up}} - N_{\text{down}}\)
• The energy \(\frac{U}{\varepsilon} = N_{\text{up}} + N_{\text{down}}\)

Exercise: Solve for \(N_{i}\) \((i \in \{\text{left, right, up, down}\})\) in terms of \(U\), \(\varepsilon\), \(L_x\) and \(L_y\).

The number of microstates available to the “polymer” chain can be thought of as a combination. How many combinations of \(N\) monomers, facing left, right, up, or down, produce a chain of dimensions \(L_x\) and \(L_y\) with energy \(U\)?

This is a somewhat more complicated example than the combinations we saw in class, but the form is similar:

\[ \Omega = \frac{N!}{N_{\text{left}}!N_{\text{right}}!N_{\text{up}}!N_{\text{down}}!} \]

Using this expression for \(\Omega\) and the expressions for \(N_i\) that you derived above, try calculating the entropy \(S\) for this “polymer”. Hint: you’ll definitely want to use Stirling’s approximation for this.

Now, for our final trick (for now), let’s see what the average lengths \(L_x\) and \(L_y\) are. Hint: \(\frac{\mathcal{T}_i}{T} = -\frac{\partial S}{\partial L_i}\), where \(i\) is \(x\) or \(y\).

(back to course home)