My new job is in a den of Bayesians! This sort of philosophical trouble is something I avoided for years when I worked on random graphs. In combinatorial probability, I just said “assume the axioms of probability” and got to look for all the interesting facts that follow logically. People want these probability calculations to say something about the “real world”? That’s not my thing; it’s up to them to go from math to science. Well, now it is my problem.
When I first started planning this transition, I got a book recommendation from Russ Lyons, who said I should read Statistics by Freedman, Pisani, and Purves. This really is a gem of a textbook. However, everything in the book fits into the philosophically risk-adverse framework of frequentist probability. (Russ told me that he thinks Bayes Theorem is all well and good, but if you go applying it like the Bayesians do, then it’s your responsibility to explain what, if anything, the results have to do with reality.)
So far I only know the kind of Bayesian statistics you learn on the streets. Does anyone know a good book? It seems like Principles of Statistical Inference by Cox is a favorite, so if I don’t hear otherwise, I’ll have a look at that.
But enough philosophical dilemmas; let me tell you some news! After spending 8 years analyzing the theoretical performance of MCMC algorithms, for the first time I actually want to run some. And it looks like it won’t be too hard. I’ll get back to you on that.
Healthy Algorithms 
You’ve read Bayesian Data Analysis, I assume? I consider that a pretty nice book.
I didn’t realize you were in Seattle now. Actually, I didn’t even realize you’d finished grad school! Congrats. I’ll probably be visiting Seattle in November, btw.
Hi Rif! As I continue blogging, you’ll learn not to assume that I have any relevant knowledge for this stuff.
Is this the Bayesian Data Analysis book you’re thinking of? I haven’t read it, but I recently started reading Gelman’s new book Red State, Blue State, Rich State, Poor State. It has great pictures like this one. IHME folks are very inspired by techniques from quantitative political science.
Let me know when you’re in Seattle!
It would be interesting to hear your thoughts on the first couple chapters of ‘Probability theory: the logic of science’ by ET Janes, http://omega.albany.edu:8008/JaynesBook.html .
Anand: I’ve only had time to read the first chapter so far, but it looks good so far. I’m a big fan of this “axiomatic approach”; it seems very honest.