I just went through the classic paper on WinBUGS, which might or might not be called probabilistic programming. It is listed on the probabilistic programming resource page, and it is certainly interesting. The WinBUGS “hello, world” is a linear regression model:
Healthy Algorithms 