Conduct outcome evaluation
What is it?
Examining and reviewing the overall impact of the intervention, and assessing what was achieved at key milestones (short, medium, long).
Why do this?
- There are many reasons to evaluate the outcomes. You will need to be able to judge and demonstrate the success (or otherwise) of your intervention in order to make judgements about its contribution to the issue you set out to address, the value for money achieved, and what future actions are needed
How might you do this?
- Make an assessment of the outcomes achieved using indicators across short, medium and long-term timescales. More >
Short-term outcomes – who was aware of the intervention during the evaluation period?
Medium-term outcomes – which determinants of behaviour were changed?
Long-term outcomes – the intended (and unintended) effects on behaviour and health.
- It is helpful to develop an ‘outcomes roadmap’ (or logic model). This sets out the underlying logic of your thinking about how the intervention will work to change people’s behaviour in different ways over time. More >
Logic models convey not only the activities that comprise the intervention, but the link between those components and the outcomes. Over time, evaluation, research and day-to-day experiences will deepen understanding of what does and does not work, and the model will change and develop further
- Be as objective as possible, for example by using several data sources to check and validate your evaluative findings. More >
This is often called ‘triangulation’, a process in which you do not rely solely on one data source, but compare it with others to see if you are getting a recurring and reliable picture of the impact. For example, the target audience may report that it is eating more fresh fruit and veg. Does this equate with what local shopkeepers are saying and with data on food sales? Similarly, trends in sexual health clinic attendance can be compared with behavioural trend data
- The cost-effectiveness, or the return on investment (ROI), of the intervention can be calculated as part of the outcome evaluation. More >
There are various ways to calculate the cost-effectiveness of an intervention. The NHS Centre for Reviews and Dissemination (2002), ‘Cost Effectiveness Matters’, provides a short, useful overview of economic evaluation http:/www.york.ac.uk/inst/crd/EM/em61.pdf
- You need clear and measurable behavioural goals from the outset so that evaluation can assess if they have been achieved
- ‘Awareness’ and ‘audiences’ views’ are important, but not as important as what actually happens to the behaviours that you are concerned with
- Try and link your evaluation into a behaviour change model. More >
For example, using Prochaska’s Stages of Change (1984) model to illustrate, it may be that a survey looks to measure the level of the population in the ‘preparation’ stage based on the assumption that they will, in time, move into the ‘action’ stage
- There are numerous national and local surveys which provide baseline data and tested questions for use in your own surveys. More >
The National Social Marketing Centre has produced a resource guide which provides a description and access details for more than 80 social surveys and market research studies covering a range of topics. A copy is available in the Tools for this task. Another useful resource is the Economic and Social Research Council (ESRC) question bank of attitudinal and behavioural questions available at: www.surveynet.essex.ac.uk/sqb/qb. Both sources are a good starting point for developing a questionnaire
- Showing that the intervention has achieved a positive return on investment can help you make a strong case for continued funding
- Determining the impact on the behavioural goals set for the intervention
- The information required to enable you to report on the outcomes and impacts of the intervention, including reporting on return on investment