25 | April | 2011 | Healthy Algorithms

Daily Archives: April 25, 2011

April 25, 2011 · 6:59 pm

MCMC in Python: Part II of PyMC Step Methods and their pitfalls

I had a good time with the first round of my Step Method Pitfalls: besides making some great movies, I got a tip on how to combine Hit-and-Run with Adaptive Metropolis (together they are “the H-RAM“, fitting since the approach was suggested by Aram). And even more important than getting the tip, I did enough of a proof-of-concept to inspire Anand to rewrite it in the correct PyMC style. H-RAM lives.

Enter the GHME, where I had a lot of good discussions about methods for metrics, and where Mariel Finucane told me that her stress test for new step methods is always the banana (from Haario, Saksman, Tamminen, Adaptive proposal distribution for random walk Metropolis algorithm, Computational Statistics 1999):

The non-linear banana-shaped distributions are constructed from the Gaussian ones by ‘twisting’ them as follows. Let f be
the density of the multivariate normal distribution $N(0, C_1)$ with the covariance again given by $C_1 = {\rm diag}(100, 1, ..., 1)$ . The density function of the ‘twisted’ Gaussian with the nonlinearity parameter $b > 0$ is given by $f_b = f \circ \phi_b$ , where the function $\phi_b(x) = (x_1, x_2 + b x_1^2 - 100b, x_3, ..., x_n)$ .