Tag Archives: MCMC

The one before that

Jake Vanderplas’s comparison of Python MCMC modules was preceded by a Bayesian polemic. In general, I find the stats philosophy war old-timey and distracting, but his comparison of confidence intervals and credible intervals is something I need to understand better.

http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/

Comments Off

Filed under statistics

MCMC in Python: a bake-off

While I’m on a microblogging spree, I’ve been meaning to link to this informative comparison of pymc, emcee, and pystan: http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bayesianism-4-bayesian-in-python/

Comments Off

Filed under statistics

MCMC in Python: Another thing that turns out to be hard

Here is an interesting StackOverflow question, about a model for data distributed as the sum of two uniforms with unknown support. I was surprised how hard it was for me.

http://stackoverflow.com/questions/24379868/estimate-the-parameters-of-a-random-variable-which-is-the-sum-of-uniform-random/24397044#24397044

I think the future of probabilistic programming should be to make a model for this easy to code.

Comments Off

Filed under statistics

MCMC in Python: Never… no… always check for convergence

I’ve had no teaching responsibilities over the last quarter, and I must miss it. I’ve found myself responding to PyMC questions on StackOverflow more than ever before. It is an interesting window into what is hard in Bayesian computation. Checking (and achieving) MCMC convergence is one thing that is hard. Here are some questions and answers that include it:

http://stackoverflow.com/questions/24294203/difference-between-bugs-model-and-pymc/24347102#24347102

http://stackoverflow.com/questions/24242660/pymc3-multiple-observed-values/24271760#24271760

http://stackoverflow.com/questions/24402834/fitting-power-law-function-with-pymc/24413323#24413323

Comments Off

Filed under statistics

MCMC in Python: sampling in parallel with PyMC

Question and answer on Stack Overflow.

Comments Off

Filed under software engineering

MCMC in Python: How to make a custom sampler in PyMC

The PyMC documentation is a little slim on the topic of defining a custom sampler, and I had to figure it out for some DisMod work over the years. Here is a minimal example of how I did it, in answer to a CrossValidated question.

Comments Off

Filed under MCMC

MCMC Convergence Diagnostics

I have revisited my approach to deciding if MCMC has run for long enough recently, and I’m collecting some of the relevant material here:

Last time I thought about it: http://healthyalgorithms.com/2010/04/19/practical-mcmc-advice-when-to-stop/

Original paper for R_hat approach: http://www.stat.columbia.edu/~gelman/research/published/itsim.pdf

Presentation comparing several approaches: http://www.people.fas.harvard.edu/~plam/teaching/methods/convergence/convergence_print.pdf

Published comparison: http://www.jstor.org/stable/2291683

Blog about a cool visual approach: http://andrewgelman.com/2009/12/24/visualizations_1/

Discussion on cross-validated: http://stats.stackexchange.com/questions/507/what-is-the-best-method-for-checking-convergence-in-mcmc

Book with a chapter on this referenced there: http://www.amazon.com/dp/1441915753/?tag=stackoverfl08-20 (available as an eBook from UW Library, how convenient!)

Another related blog: http://xianblog.wordpress.com/2012/11/28/mcmc-convergence-assessment/

Comments Off

Filed under statistics

ML in Python: Naive Bayes the hard way

A recent question on the PyMC mailing list inspired me to make a really inefficient version of the Naive Bayes classifier. Enjoy.

Comments Off

Filed under machine learning

Stan in IPython: reproducing 8 schools

Continuing my experiment using Stan in IPython, here is a notebook to do a bit of the eight schools example from the RStan Getting Started Guide.

Comments Off

Filed under software engineering

Stan in IPython: getting starting

There has been a low murmur about new MCMC package bouncing through my email inbox for a while now. Stan, it is. The project has reached the point where the developers are soliciting Python integration volunteers, so I decided it is time to check it out.

Good news, it installed and ran the example without frustration! I don’t take that for granted with research software.

IPython Notebook here.

Comments Off

Filed under software engineering