PIRU aims to carry out methodological projects, either on its own or as part of a particular evaluation, which will contribute to the development of better methods for policy evaluation. Almost every evaluation of pilots of a new approach has to tackle the potential for selection bias when comparing the new with the previous arrangements. This raises issues at each stage of the evaluation, including how to select pilots and controls, what to measure, for how long, which analytical methods to choose, and how to present the results. Our work aims to illustrate the methodological alternatives faced at each stage, and to consider the potential impact on the results of alternative methodological choices.
We have also looked more widely at the specification, design, execution and use of findings from evaluations of a number of high profile pilot programmes commissioned by the Department of Health in recent years in order to examine how, and for what purpose, recent pilots were initiated and how decisions about their organisation have influenced opportunities for evaluation.