The Edward library http://edwardlib.org/ has made the modeling approach of G.E.P. Box sound appealing to me. Here is some reading on it:
https://www.jstor.org/stable/1266570?seq=1#page_scan_tab_contents
https://www.jstor.org/stable/1266125?seq=1#page_scan_tab_contents
https://www.jstor.org/stable/1266318?seq=1#page_scan_tab_contents
https://www.jstor.org/stable/2286841?seq=1#page_scan_tab_contents
https://www.jstor.org/stable/2982063?seq=1#page_scan_tab_contents
Comments Off on G.E.P. Box on Model Building
Filed under Uncategorized
Here is a github issue and solution that I saw the other day. I think it’s a nice pattern.
def generate_model(values={'mu': true_param, 'm': None}):
#prior
mu = pymc.Uniform("mu", lower=-10, upper=10, value=values['mu'],
observed=(values['mu'] is not None))
# likelihood function
m = pymc.Normal("m", mu=mu, tau=tau, value=values['m'],
observed=(values['m'] is not None))
return locals()
Comments Off on MCMC in Python: sim and fit with same model
Filed under statistics
Here is a recent q&a on stack overflow that I did and liked.
Comments Off on MCMC in Python: Fit a non-linear function with PyMC
Filed under statistics
Jake Vanderplas’s comparison of Python MCMC modules was preceded by a Bayesian polemic. In general, I find the stats philosophy war old-timey and distracting, but his comparison of confidence intervals and credible intervals is something I need to understand better.
http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/
Comments Off on The one before that
Filed under statistics
While I’m on a microblogging spree, I’ve been meaning to link to this informative comparison of pymc, emcee, and pystan: http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bayesianism-4-bayesian-in-python/
Comments Off on MCMC in Python: a bake-off
Filed under statistics
Here is an interesting StackOverflow question, about a model for data distributed as the sum of two uniforms with unknown support. I was surprised how hard it was for me.
I think the future of probabilistic programming should be to make a model for this easy to code.
Comments Off on MCMC in Python: Another thing that turns out to be hard
Filed under statistics
I’ve had no teaching responsibilities over the last quarter, and I must miss it. I’ve found myself responding to PyMC questions on StackOverflow more than ever before. It is an interesting window into what is hard in Bayesian computation. Checking (and achieving) MCMC convergence is one thing that is hard. Here are some questions and answers that include it:
http://stackoverflow.com/questions/24294203/difference-between-bugs-model-and-pymc/24347102#24347102
http://stackoverflow.com/questions/24242660/pymc3-multiple-observed-values/24271760#24271760
http://stackoverflow.com/questions/24402834/fitting-power-law-function-with-pymc/24413323#24413323
Comments Off on MCMC in Python: Never… no… always check for convergence
Filed under statistics
Comments Off on MCMC in Python: sampling in parallel with PyMC
Filed under software engineering
The PyMC documentation is a little slim on the topic of defining a custom sampler, and I had to figure it out for some DisMod work over the years. Here is a minimal example of how I did it, in answer to a CrossValidated question.
Comments Off on MCMC in Python: How to make a custom sampler in PyMC
Filed under MCMC
I have revisited my approach to deciding if MCMC has run for long enough recently, and I’m collecting some of the relevant material here:
Last time I thought about it: https://healthyalgorithms.com/2010/04/19/practical-mcmc-advice-when-to-stop/
Original paper for R_hat approach: http://www.stat.columbia.edu/~gelman/research/published/itsim.pdf
Presentation comparing several approaches: http://www.people.fas.harvard.edu/~plam/teaching/methods/convergence/convergence_print.pdf
Published comparison: http://www.jstor.org/stable/2291683
Blog about a cool visual approach: http://andrewgelman.com/2009/12/24/visualizations_1/
Discussion on cross-validated: http://stats.stackexchange.com/questions/507/what-is-the-best-method-for-checking-convergence-in-mcmc
Book with a chapter on this referenced there: http://www.amazon.com/dp/1441915753/?tag=stackoverfl08-20 (available as an eBook from UW Library, how convenient!)
Another related blog: http://xianblog.wordpress.com/2012/11/28/mcmc-convergence-assessment/
Comments Off on MCMC Convergence Diagnostics
Filed under statistics
Healthy Algorithms 