It’s been a busy two weeks since I got back in town. The PBFs who went to “the field” for their summer abroad have returned with lots of fun and interesting stories. A new batch of PBFs and PGFs has arrived, bringing IHME to it’s planned capacity of around 100 heads. And I’ve been getting deeply into experimental analysis of a gaussian process regression technique, much like the one we used for estimating child mortality rates.
Maybe I’ll work on it publicly here on healthy algorithms. I’ll see if that seems too boring as I proceed.
For the moment, I’m just looking for reading suggestions. I was very inspired by David Johnson’s papaer A Theoretician’s Guide to the Experimental Analysis of Algorithms when I read it, but that was years ago. I’m going to have to read it again. What else do you recommend like this?
Healthy Algorithms 
Google scholar suggests quite a list of papers referencing this: http://scholar.google.com/scholar?cites=869217855495422023
Toby Walsh has a webpage of Empirical Methods in CS and AI.
http://www.cse.unsw.edu.au/~tw/empirical.html
@Abner: Thanks, that’s a nice collection. It’s all contemporaneous to Johnson’s article, though. Is there anything new out there I should know about?
I’ve done some more web-searching (partially inspired by David Johnson’s Principle 2, paraphrased here as do your freaking homework), and here is my shortlist to read of recent papers on the subject:
http://www.springerlink.com/content/1n2507380750vr44/
http://portal.acm.org/citation.cfm?id=1830911
Click to access Dalal_Streaming_final.pdf
http://portal.acm.org/citation.cfm?id=1297818
Click to access ed3.pdf
Catherine McGeoch is the researcher who’s been publishing on this the longest, it seems, and her webpage says that she has a book-in-progress on this exact topic. Maybe that is what I need.
Another to-be-published book, this one with a publisher’s webpage: http://www.springer.com/computer/ai/book/978-3-642-02537-2
Pingback: MCMC in Python: Part IIb of PyMC Step Methods and their pitfalls | Healthy Algorithms
Catherine McGeoch: A Guide to Experimental Algorithmics, Cambridge Press 2012.