Prof Darren Wilkinson Professor of Stochastic Modelling
School of Mathematics & Statistics
Newcastle University

Metropolis-Hastings sampling

This week we will look at how to construct Metropolis and Hastings samplers for sampling from awkward distributions, and how to carry out a basic analysis of the output. First read carefully through the following examples, trying them out as you go along, then tackle the exercises below.


Try out the R code for the simple Metropolis example. See how different choices of alpha affect the mixing of the chain. The example used alpha=1 which isn't too bad. Try alpha=200 - this gives a chain which is far too "hot". Very few candidate points are accepted, but when they are, they often represent quite considerable moves within the sample space. Also try alpha=0.1 - this gives a chain which is far too "cold". Most candidates are accepted, but they represent rather conservative moves within the sample space.
As we have seen, high acceptance probabilities tend to mean that the chain is too cold, whereas very low acceptance probabilities correspond to chains which are too hot. Modify the R function so that it records and then prints the overall acceptance rate of the chain, as well as returning the vector of simulated values. "Tune" the Metropolis sampler by finding a value of alpha which results in an acceptance probability of around 0.3.
Try out the R code for the Metropolis-Hastings independence sampler. Try it for different shape and scale parameters. In particular, try it for a=0.1, b=0.01. Notice how there is a bit of a problem with the sampler getting "stuck" at very small and very large values. Again, modify the code to keep track of acceptance probabilities. When does the sampling scheme do worst? How might you modify this sampling scheme to be more efficient?
4. (for postgrads)
Get the C code for the two samplers compiled and loaded into R. Make sure you get similar results (but faster) using the compiled code.

Darren J Wilkinson Book: Stochastic Modelling for
	      Systems Biology