Let’s talk about a small question as a way of introducing a big question.

## How thick is the atmosphere?

How far does Earth’s atmosphere extend into space?  In other words, how high can you go in altitude before you start to have difficulty breathing, or your bag of chips explodes, or you need to wear extra sunscreen to protect your skin from UV damage?

You probably have a good guess for the answer to these questions: it’s something like a few miles of altitude.  I personally notice that my skin burns pretty quickly above ~10,000 feet (about 2 miles or 3 km), and breathing is noticeably difficult above 14,000 feet even when I’m standing still.

Of course, technically the atmosphere extends way past 2-3 miles.  There are rare air molecules from Earth extending deep into space, becoming ever more sparse as you move away from the planet.  But there’s clearly a “typical thickness” $h$ of the atmosphere that is on the order of a few miles.  Altitude changes that are much smaller in magnitude aren’t noticeable, and altitude changes that are much larger give you a much thinner atmosphere.

What physical principle determines this few-mile thickness?

At a conceptual level, this is actually a pretty simple problem of balancing kinetic and potential energy.  Imagine following the trajectory of a single air molecule (say, an oxygen molecule) for a long time.  This molecule moves in a sort of random trajectory, buffeted about by other air molecules, and it rises and falls in altitude.  As it does so, it trades some of its kinetic energy for gravitational potential energy when it rises, and then trades that potential back for kinetic energy when it falls.  If you average the kinetic and potential energy of the molecule over a long time, you’ll find that they are similar in magnitude, in just the same way that they would be for a ball that bounces up and down over and over again.

There is actually an important and precise statement of this equality, called the virial theorem, which in our case says that

$2 \langle \textrm{K.E.}_z \rangle = \langle \textrm{P.E.} \rangle$

where $\langle \textrm{K.E.}_z \rangle$ is the average potential kinetic energy in the vertical direction and $\langle \textrm{P.E.} \rangle$ is the average potential energy.

The gravitational potential energy of a particle of mass $m$ is just

$\textrm{P.E.} = mgh$

and the typical kinetic energy of the air molecule is related to the temperature, (this is, in fact, the definition of temperature):

$\langle \textrm{K.E.}_z \rangle = \frac{1}{2} k_B T$,

where $k_B$ is Boltzmann’s constant and $T$ is the absolute temperature (i.e., measured from absolute zero).  On the earth’s surface, $k_B T$ is about 25 milli-electronvolts, or $\approx 4 \times 10^{-21}$ Joules.

Using these equations to solve for $h$ gives $h \sim k_B T /(m g)$, which is about 5 miles.

Everything makes sense so far, but let’s ask a more interesting question: What is the function that describes how the thickness of the atmosphere decays with altitude?  In other words, what is the probability density $p(z)$ for a given air molecule to be at altitude $z$?

Let’s take a God-like perspective on this question [insert joke here about typical physicist arrogance].  Imagine that you could choose some function $p(z)$ from the space of all possible functions, and in order to make your choice you must first ask the question: which function is best?

“Best” may seem like a completely subjective word, but in physics we often have optimization principles that let us define the “best solution” in a very specific way.  In this case, the best solution is the one with the highest entropy.  Remember that saying “this state has maximum entropy” literally means “this state is the one with the most possible ways of happening”.  So what we are really searching for is the function $p(z)$ that is most probable to appear from a random process.

The entropy of a probability distribution $p(z)$ is

$S = - k_B \int_0^\infty p(z) \ln p(z) dz$,

This is a generalization of the Boltzmann entropy formula $S = k_B \ln W$ (which is a sufficiently big deal that Boltzmann had it engraved on his tombstone).

Now, there are two relevant constraints on the function $p(z)$.  First, it must be normalized:

$\int_0^\infty p(z) dz = 1$.

Otherwise, it wouldn’t be a proper probability distribution.

Second, the distribution must correspond to a finite average energy.  In particular, the average potential energy of an air molecule must be $k_B T$.  Since the energy of a molecule with altitude $z$ is $m g z$, we have the second constraint

$\int_0^\infty (m g z) \times p(z) dz = k_B T$.

Now, for those of you who read the previous post, this kind of problem should start to look familiar.  To recap, we want

• a function $p(z)$ that maximizes some quantity $S$
• and is subject to two constraints

This is a job for Lagrange multipliers!

To optimize the quantity $S$ using Lagrange multipliers, we start by writing the Lagrange function

$\Lambda = S - \lambda_1 [\int_0^\infty p(z) dz - 1] - \lambda_2 [mg \int_0^\infty z p(z) dz - k_B T]$.

Here, the two quantities in brackets represent the constraints.  Putting in the expression for $S$ and then taking the derivative $\partial \Lambda/\partial p$ and setting it equal to zero gives

$-k_B [\ln p(z) + 1] - \lambda_1 - \lambda_2 m g z = 0$

Since $p$ appears only in a logarithm, rearranging and solving for $p(z)$ gives something like

$p(z) = \textrm{const.} \times e^{-\textrm{const.} \times z}$

Now we can use the two constraint (normalization and having a fixed expectation value of the energy) to solve for the values of the two constants.  This procedure gives

$p(z) = \frac{1}{h} e^{-z/h}$,

where $h = k_B T /(m g)$ is the same “typical thickness” that we estimated at the beginning.

Maybe this seems like a funny little exercise in calculus to you, but what we just did is actually a big deal.  We started with very little knowledge of the system at hand: we didn’t know anything about the composition of Earth’s atmosphere, or how air molecules collide with each other, or any principles of physics at all except for the high-school level formula for gravitational potential energy and the understanding that temperature is a measure of kinetic energy.  But that was enough to figure out the precise formula for atmospheric density, just by demanding that such a formula must be the most likely one, in the sense of having the highest entropy.

And, it turns out, our derivation is pretty good.  Here’s data from the Naval Research Laboratory:

Notice that the density of the atmosphere looks very much like an exponential decay (a straight line on this plot) up until about 80 km of altitude.  At higher altitude there’s a sort of crazy increase in temperature (probably due to direct heating from solar radiation and an absence of equilibration with the thicker atmosphere below it) that slows down the decay of atmosphere density.

## The Boltzmann Distribution

With a relatively small amount of work, we figured out how thick Earth’s atmosphere is, and how the that thickness depends on altitude.

But it turns out that what we really just did is something much bigger.  We found a way to relate energy — in that last problem, expressed through altitude — to probability.

So let’s take a step back, and look over what we did while thinking of a much bigger, more general problem.  Suppose that some system (it could be a single particle, or it could be a set of many particles) has many different configurations that it can take.  Let’s say, generically, that the energy of some configuration $i$ has energy $E_i$.  Now let’s ask: what is the best probability distribution $p_i$ for describing how likely each configuration is?

Despite knowing literally nothing about the specifics of this problem, we can still approach it in exactly the same way as the last one.  We say that the distribution $p_i$ must maximize the entropy:

$S = - \sum_i p_i \ln p_i$,

while it is subject to the normalization constraint

$\sum_i p_i = 1$

and the constraint of having a finite average energy $k_B T$:

$\sum_i E_i p_i = k_B T$.

These equations all look identical to the ones we wrote down when talking about the atmosphere.  So you can more or less just write down the answer now by looking at the previous one, without doing any work:

$p_i = \textrm{const.} \times e^{-E_i/(k_BT)}$.

Now this formula is a really big deal. It is called the Boltzmann distribution.

The Boltzmann distribution allows you, very generically, to say how likely some outcome is based only on its energy.  The only real assumption behind it is that the system has time to evolve in a sort of random way that explores many possibilities, and that its average quantities are not changing in time.  (This set of conditions is what defines equilibrium.)

It’s a formula that rears its head over and over in physics, turning seemingly impossible problems into easy ones, where all the details don’t matter.  I’m pretty confident that, if I had discovered it, I would put it on my tombstone also.

### Footnote:

• While I, personally, have used the Boltzmann formula countless times in my life, my favorite application of it was to study pedestrian crowds.  It turns out that humans have a very well-defined analogue of “interaction energy” with each other that dictates how they move through crowds.  The Boltzmann distribution is what enabled us to figure out how that interaction worked!

One of the most useful concepts I learned during my first year of graduate school was the method of Lagrange multipliers. This is something that can seem at first like an obscure or technical piece of esoterica – I had never even heard of Lagrange multipliers during my undergraduate physics major, and I would guess that most people with technical degrees similarly don’t encounter them.  When I was first taught Lagrange multipliers, my reaction was something like “okay, I’m guessing this is just a mathematical trick used by specialists in a few specific circumstances. After all, I’ve done just fine without it so far.”

But, like many mathematical tools, Lagrange multipliers are one of those things that open doors for you.  Once you understand how to do optimization using them, whole worlds of problems open up that you would have previously thought were too hard, or had no good solution. I personally have found myself using Lagrange multipliers for everything from statistics to quantum mechanics, from electron gases to basketball.

My goal for the next few posts is to derive some of the most important equations in physics: the “distribution functions” that relate energy to probability.  But before we get there it’s worth pausing to appreciate the power of Lagrange multipliers, which will be one of the major tools that enable us to understand how nature maximizes probability.

## A simple example

The basic use of Lagrange multipliers is fairly simple: they are used to find the maximum or minimum of some function in situations where you have constraints.  For example, the standard introductory problem to Lagrange multipliers is usually something like this:

Suppose that you are living on an inclined plane described by the equation z = -2x + y, but you can only move along the circle described by x2 + y2 = 1.  What is the highest point (largest z) that you can reach? What is the lowest point?

What makes this problem tricky, of course, is the relationship between x and y.  If x and y were independent of each other, then you could simply maximize the function with respect to each variable independently.  But the constraint that x2 + y2 = 1 means that you have to work harder.

If you haven’t learned the method of Lagrange multipliers, your first instinct will probably be to try and reduce the number of variables in the problem.  For example, you could try to use the constraint equation x2 + y2 = 1 to solve for y in terms of x, and then plug the solution for y into the equation that you’re trying to maximize or minimize.  Then you can hope to get the maximum or minimum by taking the derivative of z with respect to your one remaining variable, x. If you try this method, however, you’ll find that it gets messy really quickly. And heaven help you if you have a problem with many variables or many constraints – you’ll have to do a whole lot of messy solving and substituting before you get the equation down to a single variable.

The key idea behind the method of Lagrange multipliers is that, instead of trying to reduce the number of variables, you increase the number of variables by adding a set of unknown constants (called Lagrange multipliers).  What you get in exchange for increasing the number of variables, however, is a new function (commonly denoted Λ), for which all the variables are independent.  With this magic new function you can do the optimization simply by taking the derivative of Λ with respect to each variable one at a time.  This function (called the Lagrange function) is:

$\Lambda(x,y, ..., \lambda_1, \lambda_2 ...) = (\textrm{function you're trying to optimize}) ...$

$- \lambda_1 (\textrm{first constraint equation}) - \lambda_2 (\textrm{second constraint equation}) - ...$

[Here when I write “constraint equation”, I really mean “the left-hand side of a constraint equation, written so that the right-hand side is zero”.]  You can find the maximum or minimum of this function by setting all of its derivatives to zero:

$\frac{\partial \Lambda}{\partial x} = \frac{\partial \Lambda}{\partial y} = ... = \frac{\partial \Lambda}{\partial \lambda_1} = \frac{\partial \Lambda}{\partial \lambda_2} = ... = 0$

So in our example problem, the Lagrange function is

$\Lambda = -2x + y - \lambda(x^2 + y^2 - 1)$.

The first part, $-2x + y$, is the function z that we’re trying to maximize/minimize, and the part in parentheses, ($x^2 + y^2 -1$), is the constraint.  The three equations that come from taking the derivatives of  are

$\frac{\partial \Lambda}{\partial x} = -2 -2 \lambda x = 0$

$\frac{\partial \Lambda}{\partial y} = 1 - 2 \lambda y = 0$

$\frac{\partial \Lambda}{\partial \lambda} = x^2 + y^2 - 1 = 0$.

This last equation is just a repetition of the constraint equation, but the other two are really useful.  You can manipulate them pretty easily to find that

­­­$x = -1/\lambda$,    $y = 1/(2 \lambda)$

Using the constraint equation allows you to solve for $\lambda$, and after a relatively painless bit of plugging and chugging you’ll arrive at two solutions:

$x = -2/\sqrt{5}$,    $y = 1/\sqrt{5}$,     $z = \sqrt{5}$

$x = 2/\sqrt{5}$,    $y = -1/\sqrt{5}$,    $z = -\sqrt{5}$.

These are the maximum and the minimum that we’re looking for.

The real power of Lagrange multipliers

What’s really great about Lagrange multipliers is not that they can solve rinky-dink little problems like the one above, where you’re looking for the best point on some function.  What’s amazing is that Lagrange multipliers can find you an optimal function.

Let’s imagine, as an example, the following contrived problem.  Suppose that there is an outdoor, open-air rock concert, and music fans crowd around the stage to hear.  In general, the density of the crowd will be highest right next to the stage, and the density will get lower as you move away.

In choosing where to stand, the audience members have to weigh the tradeoff between their desire to be close to the band and their desire to avoid a very dense crowd.  Suppose that there is some “happiness function” that weighs both of these factors together.  For the purposes of our contrived example, let’s say it’s

$h = \frac{1}{1+x} - c \rho^2$.

Here, h is the happiness of a person at a distance x (in some units) from the stage, c is some constant, and $\rho$ is the density of the crowd around them.  The term $1/(1+x)$  is supposed to represent the enjoyment that people get from being close to the band, which decays as you move away, while the negative term  represents a person’s discomfort at being in an extremely dense crowd.  The interesting question is: what distribution of crowd density, $\rho(x)$, maximizes the total happiness of everyone at the concert?  In other words, what is the very best function $\rho(x)$?

While h(x) represents the happiness of a particular person at position x, the total happiness of everyone in the crowd is

$H = \int h(x) \rho(x) dx$.

That is, H is equal to the number of people $\rho(x) dx$ in any small interval $(x, x+dx)$ of position, multiplied by the happiness of those people, and summed over all positions.  This is the function that we will try to maximize.

The constraint on this function is that there is some fixed total number N of people in the crowd:

$\int \rho(x) dx = N$.

Now, using the recipe outlined above, we can write down a Lagrange function

$\Lambda = H - \lambda ( \int \rho(x) dx - N)$.

In the previous problem, we were only trying to find optimal values of two specific variables: x and y.  Here, we are trying to find the optimal value of $\rho(x)$ at every value of x.  So you can think that our goal is to optimize the function $H$ with respect to infinitely many variables: one value of  for every possible position.  Beyond that conceptual generalization, however, the recipe for solving the problem is the same.  If it helps, you can imagine dividing up the set of all possible positions into discrete points: $x_1, x_2, x_3$, etc.  Each position $x_i$ has a corresponding value of $\rho_i$ and a corresponding value of the local happiness function $h_i = 1/(1+x_i) - c \rho_i^2$.  The function to be optimized is then just

$H = h_1 \rho_1 + h_2 \rho_2 + ...$

while the constraint condition is

$\rho_1 + \rho_2 + ... = N$.

The optimality of the Lagrange function says that

$\frac{\partial \Lambda}{\partial \rho_1} = \frac{\partial \Lambda}{\partial \rho_2} = ... = 0$

Let’s consider some particular point $\rho_i$.  The Lagrange equation

$\frac{\partial \Lambda}{\partial \rho_i} = 0$

gives

$\frac{\partial}{\partial \rho_i} (h_i \rho_i) - \lambda = 0$

$\frac{1}{1 + x_i} - 3 c \rho_i^2 - \lambda = 0$.

Drop the subscript i, and you’ll see that this equation is actually telling you about the functional dependence of the density  on the position x.  In particular, solving for $\rho$ gives

$\rho(x) = \sqrt{ \frac{1}{3c} ( \frac{1}{1 + x} - \lambda) }$.

The value of $\lambda$ depends on the number of people N in the crowd – a larger crowd means $\lambda$ gets closer to zero.  You can go back and solve for its value by doing the integral of $\rho(x)$, but in the interest of not being too pedantic I’ll spare you the details.   The final solution for $\rho(x)$ looks something like this:

The takeaway from this funny exercise is that Lagrange multipliers allow you to solve not just for the optimal point on some function, but for the optimal kind of function for some environment.  This is the kind of problem that I didn’t even realize was well-posed until I got to graduate school, and the ability to solve such problems is an extremely powerful tool in physics.  Indeed, it is one of the recurring themes of physics that when we want to know which laws govern nature, we start by asking “which laws would give the smallest (or largest) total amount of X?”

When it comes to asking those kinds of questions, Lagrange multipliers are like a math superpower.

### UPDATE:

A couple people have commented (on Twitter) that there is a simple pictorial way to think about Lagrange multipliers and why they work, and there’s no reason for me to make them seem like black magic.  This is true, of course, so let me try and give a quick recap of the intuitive explanation for the method.

Consider the first example in this post, where you are constrained to move along the circle $x^2 + y^2 = 1$.  Imagine an arrow pointing in the direction of your motion as you walk around the circle.  And now imagine also an arrow that represents the gradient of the function $f$ you are trying to maximize (remember, the gradient of a function points in the direction of greatest increase of that function).  If the arrow for the direction of your motion points in the same direction as the gradient, then you are moving directly uphill.  If the arrow of your motion points in the opposite direction as the gradient, then you are moving directly downhill.

Most of the time, there will be some particular angle between the direction of your motion and the gradient.  This means you are moving “somewhat uphill” or “somewhat downhill”.  But at the very peak height (or at the very lowest point) of your trajectory, your motion will be exactly perpendicular to the gradient, meaning that for that instant you are moving neither uphill nor downhill.

The key idea is to imagine a function $g(x,y)$ that represents the constraint — in our example $g(x,y) = x^2 + y^2 - 1$.  The constraint (the definition of the circle you are constrained to walk along) represents the contour $g(x,y) = 0$.  The gradient of the function $g(x,y)$ always points perpendicular to the direction of your motion along the circle, since by definition moving along the circle does not increase the value of $g(x,y)$.

So, putting all the pieces together, we arrive at the conclusion that at a maximum or minimum, the gradient of $f$ points parallel to the gradient of $g$.

In equation form, this is

$\partial_{x} f = \lambda \partial_{x} g$,

$\partial_{y} f = \lambda \partial_{y} g$,

where $\lambda$ is some constant.

This is exactly the Lagrange multiplier equation $\partial_{x} \Lambda = latex \partial_{y} \Lambda = ... = 0$, with $\Lambda = f – \lambda g$.

If this all still feels pretty opaque, there is a very nice video series from Khan academy on this subject.

There is a common conception that physics is a business of writing and solving exact equations.  This idea is not untrue, in the sense that physicists generally prefer to produce exact solutions when they can.  But precise equations can be slow: they are often cumbersome to work with and can obscure important concepts with a tedium of error-checking and term-collecting.  For these reasons, physicists often figure things out (at least in the initial stages of problem solving) using a kind of semiquantitative reasoning that doesn’t make use of exact equalities.

In this kind of reasoning, all (or most) equations are downgraded from having an equals sign, $A = B$, which means “A is equal to B”, to having a “squiggle” sign, $A \sim B$, which means “A is equal to B up to some numeric factor that I don’t particularly care about.”

This may seem kind of dumb to you. Why reason with squiggles when you can write exact equations instead?  But the truth is that “squiggle reasoning” often allows you to figure things out much more quickly and easily than you would ever be able to if you insisted on writing only exact equations.  And as long as you are willing to live with some ignorance about exact numerical values, you sacrifice very little in terms of conceptual clarity.

As it happens, I designed and taught a short course last year for high school students that introduces basic ideas in quantum mechanics using squiggle reasoning. (I am teaching the course again this year.)  As an introduction, I gave the students the following problem:

If a bunch of animals of different sizes all jump out of an airplane together, how fast do they each fall?

In this post I’ll take you through the answer to this problem, which can perhaps serve as a gentle introduction to quantitative reasoning in situations where you don’t know how to (or don’t want to) write down exact equations.

### Gravitational Force

The starting point in solving this problem is to forget that animals have particular shapes.  That is, simplify the geometry of a given animal down to a single number: its “size” $L$.  Now, obviously for any real animal you will get a different number for the “size” depending on which direction you choose for the measurement.  For example, I personally am something like 1.8 meters tall, 0.6 meters wide, and 0.3 meters thick.  But if you just want a number that is in the right ballpark, it is fair to say that I am ~1 meter in size, as opposed to 1 centimeter or 1 kilometer.

To connect to an old trope, this kind of thinking isn’t really “assuming a cow is a sphere” so much as it is “not caring about the difference between a cow and a sphere”.

Now you can ask: what is the force of gravity acting on an animal of size $L$?  Well, the force of gravity is $F_g = m g = \rho V g$, where $g \approx 10 \textrm{ m/s}^2$ is the acceleration due to gravity, $m = \rho V$ is the animal’s mass, $\rho$ is the density of the animal, and $V$ is its volume.

Since we have decided to forget about all specifics of the animal’s shape, making an estimate for the animal’s volume is actually very easy:

$V \sim L^3$.

In fact, in squiggle reasoning, every three-dimensional shape has volume $\sim (\textrm{size})^3$, unless you have decided to look at some shape that is especially long and skinny.  This means that we can easily write an approximate equation for the force of gravity acting on the animal:

$F_g \sim \rho g L^3$.

### Drag force

Immediately after jumping out of the airplane, the $L$-sized animal in question is in freefall, and accelerates downward at a rate $\sim g$.  However, after falling for a little while its acceleration is halted by the force of all the air rushing back against it.  The animal will eventually reach a steady downward velocity determined by the two forces being in balance:

So how big is the drag force $F_D$?

Of course, the exact answer to this question depends on the shape of the animal.  If you really wanted to know, with numeric accuracy, the value of the drag force, then you would need to understand the air flow pattern around the animal.  This would presumably require you to stick the animal in a wind tunnel and make careful measurements. (And you would get different answers depending on which way the animal was facing).

But at the level of squiggle reasoning, we can figure out the drag force using a simple thought exercise.  Imagine the process of throwing a big block of air at the animal:

This block is taken to have the same cross-sectional size as the animal (area $L^2$) , and some length $w$.  The mass of the air block is therefore something like $m_\textrm{air} \sim \rho_\textrm{air} L^2 w$.  If the block is thrown with a speed $v$, then it has a kinetic energy $KE \sim m_\textrm{air} v^2 \sim \rho_\textrm{air} v^2 L^2 w$.  (I’m sure you learned that first equation as $KE = \frac{1}{2} mv^2$, but when you’re doing squiggle reasoning there’s no reason to fuss about $\frac{1}{2}$’s.)

In order to stop the block of air, the animal applies a force that does work on the block equal to $KE$.  The work is equal to the drag force of the air multiplied by the distance over which the force is applied.  That distance is $\sim w$; you can think that the force is applied continuously as the air block smooshes into the animal’s side.  Thus, we have $F_D w \sim KE$, and therefore

$F_D \sim \rho_\textrm{air} v^2 L^2$.

Of course, when the animal is falling through the air, this drag force is applied continuously, as the animal finds itself continuously colliding with “blocks of air” that move toward it with speed $v$.

### Final Answer: never skydive in the rain

Now we are ready to get an answer: equating $F_g$ with $F_D$ and solving for $v$ gives us

$v \sim \sqrt{\rho g L / \rho_\textrm{air}}$.

Thus we arrive very quickly at an important semi-quantitative conclusion: larger animals fall faster, with a terminal velocity that grows as the square root of the animal’s size.

In fact, you can use this equation to get a pretty good order-of-magnitude estimate for the terminal velocity $v$, using the fact that pretty much all animals have the same density as water, $\rho \sim 1 \textrm{ g/cm}^3$, while air is about 1000 times less dense.

In particular, the squiggle equation for $v$ suggests that a meter-sized human has a terminal velocity on the order of $\sim 100 \textrm{ m/s}$.  (For reference, one m/s is about 2 mph — within the accuracy of our squiggle reasoning you can take a meter-per-second and mile-per-hour to be roughly the same thing.)  A centimeter-sized cockroach has a terminal velocity of $\sim 10 \textrm{ m/s}$, and a 10-meter-sized whale falls at about $300 \textrm{ m/s}$; three times faster than you do.

Thus, you can see pretty quickly why falling off a building is deadly for you (hitting the ground at ~ 100 mph is worse than just about any car accident) but not deadly for insects (hitting the ground at a couple mph is no big deal).

In fact, there is a pretty practical implication of this result (besides “don’t fall off a building”): You should never go skydiving in the rain.  You might think (as I initially did) that it would be a sort of magical and pleasant experience, wherein you fall together with the raindrops like an astronaut playing with zero-gravity water droplets.  But the truth is much more unpleasant: the meter-sized you will be falling at ~1o0 mph, while the millimeter-sized raindrops fall at a slow ~3 mph.  So, from your perspective, you’ll be getting stabbed by raindrops that blast you in the face at ~97 mph.

Highly unpleasant, and just a small amount of squiggle reasoning before you jump can save you the trouble.

Lately I have seen an increasingly honest, and increasingly public discussion about the feelings of inadequacy that come with trying to be a scientist.

For example, here Anshul Kogar writes about the “Crises in Confidence” that almost invariably come with trying to do a PhD.

In this really terrific account, Inna Vishik tells the story of her PhD in physics, and the various emotional phases that come with it: from “hubris” to “feeling like a fraud”.

I might as well add my own brief admissions to this discussion:

• More or less every day, I struggle with feeling like I am insufficiently intelligent, insufficiently hardworking, and insufficiently creative to be a physicist.
• These feelings have persisted since the beginning of my undergraduate years, and I expect them to continue in some form or another throughout the remainder of my career.
• I often feel like what few successes I’ve had were mostly due to luck, or that I “tricked” people into believing that I was better than I actually am.

I have gradually come to understand that these kinds of feelings, as dramatic as they seem, are relatively normal.  Some degree of impostor syndrome seems to be the norm in a world where intellect is (purportedly) everything, and where you are constantly required to “sell” your work.  And I should probably make clear that I am not a person who lacks for confidence, in general.  (If you asked my wife, she might even tell you that I am an unusually, perhaps frustratingly, confident person.)

I have also come to understand that there is a place for a person like me in the scientific enterprise.  I have very real shortcomings as a scientist, both in talent and in temperament.  But everyone has shortcomings, and in science there is room for a great variety of ability and disposition.

There is one practice that I have found very helpful in my pursuit of a scientific career, and which I think is worth mentioning.  It’s what I call fostering a “culture of tolerating ignorance.”

Let me explain.

As a young (or even old) scientist, you continually feel embarrassed by the huge weight of things you don’t know or don’t understand.  Taking place all around you, among your colleagues, superiors, and even your students, are conversations about technical topics and ideas that you don’t understand or never learned.  And you will likely feel ashamed of your lack of knowledge.  You will experience some element of feeling like a fraud, like someone who hasn’t studied hard enough or learned quickly enough.  You will compare yourself, internally, to the sharpest minds around you, and you will wonder how you were allowed to have the same profession as them.

These kinds of feelings can kill you, and you need to find a way of dealing with them.

I have found that the best strategy is to free yourself to openly admit your ignorance.  Embrace the idea that all of us are awash in embarrassing levels of ignorance, and the quickest way to improve the situation is to admit your ignorance and find someone to teach you.

In particular, when some discussion is going on about a topic that you don’t understand, you should feel free to just admit that you don’t understand and ask someone to explain it to you.

If you find yourself on the other side of the conversation, and someone makes such an admission and request, there are only two acceptable responses:

1. Admit that you, also, don’t understand it very well.
2. Explain the topic as best as you can.

Most commonly, your response will be some combination of 1 and 2.  You will be able to explain some parts of the idea, and you will have to admit that there are other parts that you don’t understand well enough to explain.  But between the two of you (or, even better, a larger group) you will quickly start filling in the gaps in each others’ knowledge.

A culture where these kinds of discussions can take place is a truly wonderful thing to be a part of.  In such an environment you feel accepted and enthusiastic, and you feel yourself learning and improving very quickly.  It is also common for creative or insightful ideas to be generated in these kinds of discussions.  To me, a culture of tolerating ignorance is almost essential for enjoying my job as a scientist.

The enemies of this kind of ideal culture are shame and scorn.  The absolute worst way to respond to someone’s profession (or demonstration) of ignorance is to act incredulous that the person doesn’t know the idea already, and to assert that the question is obvious, trivial, and should have been learned a long time ago.  (And, of course, someone who responds this way almost never goes on to give a useful explanation.)  An environment where people respond this way is completely toxic to scientific work, and it is, sadly, very common.  My suggestion if you find yourself in such an environment to avoid the people who produce it, and to instead seek out the company of people with whom you can maintain enthusiastic and non-scornful conversations.

I have personally benefited enormously from those kinds of people and that kind of culture. At this point in my career, I would hope that I could tolerate a colleague admitting essentially any level of scientific ignorance, and that I would respond with a friendly explanation of how I think about the topic and a declaration of the limits of my own understanding.

As I see it, ignorance to essentially any degree is not a crime.  There is simply too much to know, and too many perspectives from which each idea can be understood, to shame someone for admitting to ignorance.  The only crime is professing to understand something that you don’t, or making claims that are not supported by your own limited understanding.

Today, April 16, is the one day in the year when I use this blog for very personal purposes.  In particular, I reserve the day for remembering Virginia Tech and my time there.  (Past years’ writings are here: 1, 2, 3, 4, 5).

If you’re here for physics-related content, just hold on; a new post should be up within a few days.

On the afternoon of May 12, 2007, I almost did something terrible.

That particular Saturday was the day of my college graduation.  The physics department was holding a warm and enthusiastic ceremony for the seventeen of us who were graduating, with plenty of food and lots of cheer spread among the hundred or so people in attendance.

The dangerous part was that our valedictorian was an unusually generous person, and had offered to split the valedictory speech with me.  I probably should have declined, but I was apparently neither sufficiently polite nor sufficiently humble to do so.  And so I was slated to give a short speech as part of the ceremony.

What made this dangerous was that late April and early May of 2007 were confusing times for those of us at Virginia Tech.  During the week or so before the ceremony, as I sat down to try and draft my graduation speech, I found that I kept coming back to the themes we were all facing after the Virginia Tech shooting: loss, grief, anxiety, community, etc.

With those themes ever-present in my mind, I wrote something that was predictably awful.  Most of the specifics of what I wrote have been (graciously) lost to my memory, but you can probably imagine it easily enough: a painfully over-earnest speech that betrayed a deficit of self-awareness.  It would have been the sort of thing that drips with a sense of how moved the speaker is by himself.

To this day I still have nightmares where I find that I have become like “Mike”, the guy who threw me into an unreasonable rage by writing a terrible poem.  I guess I almost did the same thing.

But a very fortunate thing happened to me on the morning of Saturday, May 12:
I woke up feeling happy.

As it so happens, on the day of my graduation, surrounded by my family members and friends, I was happy.  I wasn’t “confused” or fragile or maudlin.  It was much more simple.  I was just happy.

And so I made the fortunate decision to ditch that terrible speech in favor of something more straightforward.

I decided to sing a song.

During college I had actually made a minor habit of writing parody songs about being a physics major and about the VT physics department. So I guess I was sufficiently well-practiced to be able to put together a song pretty quickly, in time for the ceremony.

The lyrics are reproduced below.

Now, I should probably warn you in advance that this is not a good song.  It’s full of overzealous dorkiness and now-incomprehensible inside jokes.  But I treasure the memory of standing in front of that audience and singing this song.  Because it is a memory of being happy; of feeling yourself surrounded by people who like you and care about you; and of being unashamed of who you are, and unafraid of the future.

I should mention, by the way, that our valedictorian’s half of the speech was awesome.  It was more or less entirely made up of jokes and impressions of our professors, and the whole afternoon was baked in geekish enthusiasm.

Good enough for me

A song for the Virginia Tech physics class of 2007

[sung to the tune of “Me and Bobby McGee”, as performed, for example, by Janis Joplin]

Standing in my cap and gown
waiting to hear my name
I’m feeling near as divided as a triplet state.
So many things I never learned,
so many tests where I got burned,
but at least I beat the high physics dropout rate.

Well my education has served me well.
It taught me some important skills,
and it taught me to avoid what I can’t do.
From Tauber’s quantum purgatory
to Mizutani’s rambling stories
I’ve mislearned more science than most people ever knew.

Well a diploma’s just a way of saying
“you’re good enough to leave
but hey, we’re not making any guarantees.”
And I may never solve a single problem
in a rotating reference frame…
But inertial frames are good enough for me.
Good enough for me to get my degree.

From the sub-basement physics lounge
to our campouts in the woods
just think of all the nerdy things we’ve done.
Text Twist games that last for months,
telling awful science puns,
yeah we’ve invented a language of our own.

Some of us can obfuscate with pictures,
but all of us speak math,
And if you say it sounds like Greek,
then I’ll have to agree.
And while we may sound pretty smart,
I’ll tell you a secret truth:
none of us know what quantum mechanics means.

Well a diploma’s just a way of saying
“you’re good enough to leave
no matter what score you got on the GRE.”
I may not know how to solve the time-dependent Schrodinger equation…
But time-independent is good enough for me.
Good enough for me to get my degree.

Well a diploma’s just a way of saying
“you’re good enough to leave
but if you want respect you’ll still need a PhD.”
And though my time here has seemed short
and it’s hard for me to leave…
Well, I guess five years were good enough for me.
Good enough for me to get my degree.

There used to exist a really wonderful webcomic called Pictures for Sad Children.  A few years ago its creator, John Campbell, grew tired of the project and removed all of it from the internet.  But the comic was hugely influential, and you can find most of its pieces reproduced online if you do a Google search.

Lesser known is the author’s smaller follow-up comic that was (somewhat bizarrely) themed around a fictionalized recounting of the life of the actor Michael Keaton.  This comic has also been taken down completely, and it is much harder to find any of its pieces online.

There was one of the Michael Keaton comics that I loved in particular, though, and which I managed to find using a lot of patience and the Internet Archive site.  I am reproducing it here not because I have any right to do so, but because it was too sad for me to think that it might get lost to humanity.

How unreasonable is it to not vaccinate your children?

I ask this not as a rhetorical question, but as a mathematical one. How do we describe, mathematically, the benefits and risks of vaccination? What does this description tell us about the reasonableness (or unreasonableness) of not vaccinating?

These days, most of the debate about vaccination is centered around questions of misinformation, misunderstanding, delusion, and conspiracy. But all this shouting obscures an interesting and very real mathematical question.

So let’s consider the dilemma of a perfectly well-informed and perfectly rational person faced with the decision of whether to vaccinate their child against some particular disease. Making this decision involves weighing issues of risk and reward, and thinking about selfishness and altruism.

Luckily for us, there is an entire mathematical science devoted to addressing these kinds of questions: the science of Game Theory.

In this post I want to take a game-theoretical look at the problem of vaccination. In particular, we’ll ask the questions: under what conditions is a disease dangerous enough that you should vaccinate? And is doing what’s best for your child different from doing what’s best for society as a whole?

## Risk and Reward

The key idea in this analysis is as follows. When you vaccinate your child, you provide them with the benefit of immunity against a disease that they might encounter in the future. This benefit is potentially enormous, and life-saving.

However, if your child lives in a population where nearly everyone already has the vaccine, then the benefit of the vaccine to your child is greatly reduced. After all, if everyone around is effectively immune to the disease already, then the group’s “herd immunity” will greatly reduce the chance that your child ever gets exposed to the disease in the first place.

You might therefore be tempted to decide that even a very small risk inherent in the vaccine would make it not worthwhile. And, certainly, such risks do exist. For example, there is a very small chance that your child could have a serious allergic reaction to the vaccine, and this reaction could lead to things like deafness or permanent brain damage. If your child is already getting “herd immunity” from everyone else’s vaccination anyway, then why risk it?

Let’s consider this question in two steps. First, we’ll ask what is the optimal vaccination rate. This is the rate that maximizes the safety and well-being of the whole population. Then, we’ll ask the more pointed question: which is the decision that is best for your child alone, given that as a parent your concern is to minimize the chance of harm to your child, and not to the world as a whole.

Let’s discuss these ideas in a completely theoretical sense first, and then we’ll put some numbers to them to see how the real world compares to the theoretical ideal.

## The optimal behavior

Imagine, first, a population where everyone is vaccinated against a particular disease except for some fraction x of non-vaccinators. Now suppose that a randomly-chosen individual gets exposed to the disease.

If the vaccine is highly effective, then the chance that this person will contract the disease is the same as the chance that they are not vaccinated: x. In the event that this person does contract the disease, then they will expose some number n of additional people to the disease. This wave of second-hand exposures will lead to a wave of third-hand exposures, and so on. At each wave there is a multiplication by n in the number of potentially exposed people, and a (hopefully small) probability x of the disease being communicated.

You can diagram the spread of the disease something like this:

This picture illustrates the case n = 4 (i.e., every infected person exposes an average of four other people). Each branch labeled “x” shows the probability of the disease being spread at that step.

If you add up the expected number of infected people, you’ll get

$(\text{total infected}) = x( 1 + n x (1 + nx(1 + ... ) ) )$

$(\text{total infected}) = x \sum_{i = 0}^{\infty} (n x)^i$

$(\text{total infected}) = x/(1 - n x)$

This last equation already suggests an important conclusion. Notice that if the rate of non-vaccination, x, gets large enough that $n x \geq 1$, then the total number of infected people blows up (it goes to infinity).

In other words, if $n x \geq 1$ then the population is susceptible to epidemics. There is a very simple way to interpret this condition: $n x$ is the average number of new people to whom a given sick person will pass their infection. If each sick person gets more than one other person sick, then the disease will keep spreading and you’ll get an epidemic.

If this condition is met, then there is no question about vaccinating. A population that is susceptible to epidemics is one where you need to get vaccinated. End of story.

But let’s assume that you live somewhere where this particular disease doesn’t cause epidemics anymore. (Like, say, mumps in the USA – more on this example below.) An absence of epidemics generally implies $n x < 1$, and any flare-up in the disease will be relatively small before it dies off.

Let’s say that every so often someone within the population is exposed to the disease. We’ll call this the rate of exposure, E, which can be defined as the number of initial exposures in the population per year. Combining this rate with the equation above means that

$E x/(1 - n x)$

people will be infected per year.

This rate of disease-induced sickness should be compared with the rate of vaccine-induced sickness. If a fraction x of people are not vaccinated, that means that $N (1- x)$ people do get vaccinated, where $N$ is the total number of people in the population. As a yearly rate, $N(1-x)/T$ people are vaccinated per year, where $T$ is the average lifetime of a person (or, if the vaccine requires periodic boosters, $T$ is the time between successive vaccinations). Let’s suppose, further, that the vaccine makes a child sick with some probability $v$.

What this all means is that there are

$(1-x) N v/T$

vaccine-induced illnesses in the population every year.

(If you’re getting lost keeping track of all these variable names, don’t worry. Only two will matter in the end.)

From a population-wide standpoint, the optimal rate of vaccination is the one that minimizes the total amount of illness in the population per year:

$F = E x/(1 - nx) + (1-x) N v/T$.

Taking the derivative $dF/dx$ of the function $F$ and setting it equal to zero gives a solution for the optimal non-vaccination rate:

$x = 1/n - \sqrt{R}/n$.                                                                    (1)

Here, the variable $R$ can be called the “relative disease risk”, and it is a combination of the variables introduced above:

$R = ET/Nv$.

You can think of $R$ as the relative risk of the disease itself, as compared to the risk associated with getting the vaccine.

(The variable $v$ should be considered to be the probability of getting sick from the vaccine, multiplied by its relative severity, as compared to the severity of the disease itself. More on this below.)

You can notice two things about the theoretically optimal non-vaccination rate, equation (1). First, the non-vaccination rate x is always smaller than $1/n$. This guarantees that there are no epidemics.

Second, the rate of non-vaccination declines as the relative disease risk $R$ increases, and at $R > 1$ the optimal non vaccination rate goes to zero. In other words, if the risk of the disease is large enough, and the risk of the vaccine is small enough, then the optimal thing is for everyone to get vaccinated.

## Rational self-interested behavior

The analysis in the previous section was only concerned with the question “what is best for the world at large?” If you are asking the more limited question “what is best for my child?”, then the answer is slightly different. For this decision, you only need to weigh the probability of getting the disease against the probability of getting sick from the vaccine. The risk of conveying the illness to others doesn’t enter the analysis.

To figure out the probability of your child getting the disease, you can repeat a similar analysis to the one above: drawing out the tree of possibilities for each instance of infection. That analysis looks a lot like the picture above, except that there is one possible branch (representing your unvaccinated child) that has no protection against infection, and the rate of contracting the disease upon exposure is $1$ instead of $x$.

The corresponding probability of your child being infected after a given initial exposure is therefore

$\frac{1}{N} \sum_{i = 1}^{\infty} (x n)^i = 1/[n(1-xn)]$.

Since we have assumed that there are $E$ initial exposures per year, the probability of your child getting the disease in their lifetime is $E T/[N(1-xn)]$.

As a rational, self-interested parent, you should only vaccinate if this probability is greater than the probability $v$ of your child getting sick from the vaccine. This means the condition for vaccination is

$E T/[N (1-xn)] > v$.

You can call this condition a “Nash equilibrium”, using the language of game theory. When the inequality is satisfied, vaccination is a good idea. When it is not satisfied, vaccination is a bad idea, and self-interested individuals will not do it. As a consequence, a population of rational, self-interested people will settle into a situation where the inequality is just barely satisfied, which is equivalent to

$x = 1/n - R/n$.                                                                           (2)

This result actually has a lot of features in common with the optimal result for vaccination. For one thing, it implies that you should always vaccinate if $x > 1/n$, which is the same lesson that has been repeated above: always vaccinate if there is any chance of an outbreak.

More pointedly, however, you should also always vaccinate any time the relative risk of the disease, $R$, is larger than 1.

In this sense the self-interested behavior is pretty closely aligned with the globally optimal behavior. The disagreement between them is a relatively mild quantitative one, and exists only when the relative disease risk $R < 1$.

## Confident self-interested behavior

Now, it’s possible that you don’t accept one of the central premises of the analysis in the preceding section. I assumed above that an essentially healthy population is subject to occasional, randomly-occurring moments of “initial exposure”. In such moments it was assumed that a person is chosen at random to be exposed to the disease. Presumably this exposure has to do with either traveling to a foreign location where the disease is endemic, or with meeting someone who has just come from such a location.

You might think, however, that it is very unlikely that your child will ever be such a “primary exposure point”. Perhaps you know that your child is very unlikely to travel to any place where the disease is endemic, or to meet anyone who has come directly from such a place. If you have this kind of confidence, then the calculation changes a bit. Essentially, one needs to remove the probability of being the initial exposure point from the analysis above.

Under these assumptions, the resulting risk of contracting the disease becomes $E T x n/[N(1-xn)]$, which is smaller than the one listed above by a factor $x n$. Consequently, the Nash equilibrium shifts to a higher rate of non-vaccination, given by

$x = 1/[n(1+R)]$.                                                                           (3)

This equation satisfies the same “no epidemics” rule, but it is qualitatively different in the way it responds to increased disease risk $R$. Namely, there is never a point where the population achieves complete vaccination.

In other words, a population of “confident” self-interested individuals will always have some finite fraction $x$ of vaccination holdouts, no matter how high the disease risk or how low the vaccine risk. If enough of their fellow citizens are vaccinated, these individuals will consider that the herd immunity is enough to keep them safe.

The three possible non-vaccination rates can be illustrated like this:

## Real data: the MMR vaccine

The above discussion was completely theoretical: it outlined the ideal rate of vaccination according to a range of hypothetical decision-making criteria. Now let’s look at where the present-day USA falls among these hypotheticals. As a case-study, I’ll look at one of the more hotly-discussed vaccines: the measles-mumps-rubella (MMR) vaccine.

First of all, it is sadly necessary for me to remind people that there is absolutely no evidence for any link between MMR (or any other vaccine) and autism.

But that’s not to say that there is zero risk inherent in the MMR vaccine. In very rare cases, a vaccination can lead directly to a runaway allergic reaction, which can produce seizures, deafness, permanent brain damage, or other long-term effects. The CDC estimates these side effects to occur in at most one person per million MMR vaccinations. (In terms of the variables above, this means $v = 10^{-6}$.

Compare this to the combined rate of measles, mumps, and rubella infections in the USA. The average rate of occurrence of these diseases during the past five years has been something like 1200 cases per year. Given that the MMR vaccine coverage in the United States is about 92%, this implies a rate of “initial exposure” of something like $1200/0.08 = 15000$/year across the entire US. (Most exposures do not lead to infection.)

Of course, most people who contract measles, mumps, or rubella recover without any permanent side effects – they just have to suffer through an unpleasant illness for a few weeks. So to make a fair comparison, I’ll discount the exposure rate by a factor that approximates only the risk of acquiring a permanent disability due to the disease. For example, about 0.3% of measles cases are fatal. For mumps, about 10% of cases lead to meningitis, and something like 20% of those result in permanent disability (such as hearing loss, epilepsy, learning disability, and behavioral problems). Finally, the main danger of rubella is associated with congenital rubella syndrome, a terribly sad condition that affects infants whose mothers contract rubella during the middle trimester of pregnancy.

Even discounting this last one, a low-side estimate is that about 1.7% of people who get measles, mumps, or rubella will suffer some form of permanent disability as a consequence. So I’ll discount the “primary exposure” rate to only $E = 0.017 \times 15000 \approx 260$/year.

This number should be compared to the rate of vaccine-induced disability, which is something like $4$ instances/year (given that about 4 million people get the MMR vaccine per year, and about one per million gets a permanent disability from it).  Comparing these rates gives an estimate for the relative disease risk:

$R \approx 70$.

Pause for a moment: this is a large number.

It implies that the risk associated with actually contracting measles, mumps, or rubella is at least 70 times larger than the risk from the vaccine.

This is true even with the relatively low incidence of cases in the US, and even with the relatively robust “herd immunity” produced by our 92% vaccine coverage. $R = 70$ is also a low-side estimate – there are a number of other disease-related complications that I haven’t taken into account, and I haven’t made any attempt to account for the unpleasantness of getting a disease that you eventually recover from without permanent disability.

Given this large value of relative risk, we can safely conclude the current non-vaccination rate in the USA, $x \approx 8$%, is way too high. At such a large value of $R$, both the altruist and the self-interested person will agree that universal vaccination is the right thing to do.

Even the “confident self-interested” person, who believes that their child has no chance of being a point of primary exposure to the disease, will agree that the current vaccine coverage is too low to justify non-vaccination. Only at $x$ less than $1$% could such a calculation possibly justify non-vaccination in the present-day USA.

## Conclusion

I went through this analysis because I believe that, at a theoretical level, there is room for a conversation about weighing the risks of vaccination against the benefits. It is true that in a relatively healthy population that is herd-immunized against outbreaks, a vaccine’s side effects can be a more real risk than the disease itself. It is also worth understanding that in such situations, the incentives of the altruist (who wants to minimize the risk to the world at large) are not perfectly aligned with the incentives of individual parents (who want to minimize the risk to their own child).

But in the present-day USA, these choices do not appear to be at all difficult, and there are no thorny theoretical issues to worry about. Our vaccines remain safe enough, and the disease risks remain large enough, that any level of rational quantitative thinking, self-interested or altruistic, leads to the same conclusion.