Project, Programme and Policy Evaluation

In this module you will learn about evaluation, and specifically about the types of evaluation that assesses the achievements of public interventions and uncover the lessons learned which have wider application. Often termed 'summative', 'ex-post', or 'post-hoc' this sort of evaluation looks back over what has changed over the course of an intervention by a public institution, and tries to answer questions such as:

• What difference has our project, programme or policy made?
• Who has benefitted and who has been left out?
• Did our intervention deliver what we expected and hoped it would?
• Were the resources spent on it justified by the results achieved?
• What else might account for the changes that have happened?
• Should we continue doing this sort of thing, and what have we learned to help us do better in the future?
• Are there lessons from this experience which can help others do better?

Ex-post evaluation is of growing importance for governments, donors, international bodies and those involved in delivering projects and programmes 'on the ground'. It has spawned an industry of researchers, evaluators and consultants who specialise in designing methodologies to answer the sort of questions posed above, and who sell their services to public bodies who want to know the answers. In many cases, the provision of public funds for an endeavour bring with them a requirement for an eventual ex-post evaluation to be carried out – and a concomitant need for data to be collected right from the beginning.

You will hear the term evaluation used in many different ways and applied at different times. This module focuses on the type of evaluation that takes stock and forms judgements. It can be contrasted with other types of evaluation that you may encounter, and which are covered to a greater or lesser degree in a number of other CeFiMS Public Policy and Management courses.

Learning outcomes

When you have completed this module and its readings, you should be able to:

• outline how ex-post evaluation informs public policy and management processes
• set the meaning of ex-post evaluation in the wider context of different ways to assess public intervention
• distinguish between ex ante, interim, and ex post evaluation, and set ex post evaluation in its proper temporal context
• specify the differences between public policies, programmes and projects, and the implications of these for their evaluation
• define a wide variety of evaluation terms and organise them into their purpose, scale, period in time and type of study design
• choose an evaluation methodology that is relevant to the institutional context and resources available
• design and carry out evaluations of different types, with an understanding of likely challenges, including identifying and validating impact, disentangling impacts from other influences; the likelihood of absent baselines, inadequate monitoring data, and insufficient research capacity
• critique evaluation studies in relation to the quality of their design, their conduct and the value of conclusions drawn
• fit evaluation practice into the real world of politics and the question of what those commissioning really want to know, and want to hear
• practically assess timescales, changing questions, and when to look for results
• form a judgement as to whether you yourself would like to consider developing a career as an evaluator
• join in with networking and exchange amongst the evaluation 'community', and identify differences from broader research.

Tuition and assessment

Students are individually assigned an academic tutor for the duration of the module, with whom you can discuss academic queries at regular intervals during the study session.

You are required to complete two Assignments for this module, which will be marked by your tutor. Assignments are each worth 15% of your total mark. You will be expected to submit your first assignment by the Tuesday of Week 6, and the second assignment at the end of the module, on the Tuesday after Week 10. Assignments are submitted and feedback given online. In addition, queries and problems can be answered through the Virtual Learning Environment.

You will also sit a three-hour examination on a specified date in September/October, worth 70% of your total mark. An up-to-date timetable of examinations is published on the website in July each year.

Study resources

• Study guide: The module study guide is carefully structured to provide the main teaching, defining and exploring the main concepts and issues, locating these within current debate and introducing and linking the assigned readings.
• Key texts: 
  • Palfrey C, P Thomas & C Phillips (2012) Evaluation for the Real World. The Policy Press.
  • Bamberger M, J Rugh & L Mabry (2012) RealWorld Evaluation: Working Under Budget, Time, Data, and Political Constraints. 2nd Edition. Sage.
• Readings: Throughout the module you will be directed to study a selection of readings, including journal articles, book extracts and case studies that are of particular relevance and interest to the topics covered in the module.
• Virtual learning environment: You will have access to the VLE, a web-accessed study centre. Via the VLE, you can communicate with your assigned academic tutor, administrators and other students on the module using discussion forums. The VLE also provides access to the module Study Guide and assignments, as well as a selection of electronic journals available on the University of London Online Library.

Study calendar 2022/23

Study calendars are subject to change.

Module overview

Unit 1 History, Types and Uses of Evaluation

• 1.1 Introduction
• 1.2 Evaluation in Public Policy
• 1.3 Judgement: Evaluation, Audit, Inspection, Journalism and Academic Research
• 1.4 Different Types of Evaluation
• 1.5 Conclusion

Unit 2 Designing an Ex-Post Evaluation: Programme Theory and the Importance of Context

• 2.1 Ex-Post Evaluation
• 2.2 Programme Theory: How Interventions are Supposed to Work
• 2.3 The Importance of Context
• 2.4 Sources of Information
• 2.5 Conclusion

Unit 3 Project Evaluation I

• 3.1 Micro Evaluation – Policies at the Coalface
• 3.2 Pilots and the Mainstream – Projects as Experiments or Examples of Basic Delivery
• 3.3 Projects in Context
• 3.4 Drawing Conclusions and the Question of Attribution
• 3.5 Project Evaluation as a Laboratory – the Question of Replicability
• 3.6 Time to Get Going: Scoping the Evaluation
• 3.7 Conclusion

Unit 4 Project Evaluation II

• 4.1 The Case Studies
• 4.2 Purpose and Context
• 4.3 Theory of Change
• 4.4 Evaluation Design and Methodology
• 4.5 Results and Conclusions in the Reports
• 4.6 Conclusion

Unit 5 Evaluation Methods: Quantitative, Qualitative and Mixed

• 5.1 Introduction
• 5.2 Quantitative Evaluation Methods
• 5.3 Qualitative Evaluation Methods
• 5.4 Mixed Method Evaluation
• 5.5 Conclusion

Unit 6 Programme Evaluation I

• 6.1 Programmes and Their Evaluation
• 6.2 Evaluation Criteria
• 6.3 The Specification of Impacts
• 6.4 Conclusion

Unit 7 Programme Evaluation II

• 7.1 The Case Studies
• 7.2 End-of-Life Care in Nursing Homes
• 7.3 The Tuungane Community-Driven Reconstruction Project
• 7.4 Conclusion

Unit 8 Policy Evaluation, and Being an Evaluator

• 8.1 Introduction
• 8.2 Policy Evaluation
• 8.3 Policy Evaluation in a Development Context
• 8.4 Multinational Policy Evaluation
• 8.5 Maximising the Usefulness of Evaluation
• 8.6 Conclusion

Module samples

• M479 Module Introduction (PDF)
• M479 Specimen Examination (PDF)
• M479 Unit 1 (PDF)

Disclaimer

Important notice regarding changes to programmes and modules