A useful metaphor for explaining MCMC

I work in an interdisciplinary institute, and you should see the fun when mathematicians, statisticians, and physicists try to discuss models and methods for health metrics each using the dialect of their specific fields. And then throw doctors and epidemiologists into the mix, with the understanding that doctors secretly think scientists might not be smart enough to be doctors and vice-versa.

It’s here where I think this metaphor my officemate and I were just trying out will be really useful. Markov Chain Monte Carlo (MCMC) is this foundational technique in my work lately, the central algorithm I have been using for sampling from the posterior distribution of all of my models. But “how does it work?”, my non-MCMC colleagues sometimes dare to ask me. (Or more frequently lately, “why doesn’t it work?”)

To explain by way of analogy, imagine that the posterior probability density of the model is a mountain, with higher probability parameters corresponding to points of higher elevation. Our goal is to summarize the topography of the mountain. Many of my colleagues are familiar with “hill-climbing algorithms”, wherein the algorithm looks for the mountain peak by taking the steepest path up from wherever it currently stands. (Familiar because they have using algorithms that do this, and often, since this is the pacific northwest, because they spend their weekends doing this themselves on actual mountains.)

MCMC is an approach that explores the mountain with a “drunken walk”, one carefully designed to stand at points of a given elevation for an amount of time proportional to the elevation. I love the visual, drunken mountain climbing.

Then, as Nate and I were just discussing, the “why do/doesn’t it work” question has an analogical answer summarized by these pictures:


Which mountain are you trying to climb drunk?