Monitor and evaluate the process
What is it?
Gathering information to check that the intervention is being delivered well and is on track to meet objectives.
Building in evaluation measures that allow you to monitor how well it is being delivered and measure progress against objectives.
You should also include a regular ‘health check’ on your intervention as it rolls out to ensure that the process and/or the outputs are not breaching ethical guidelines.
Why do this?
Monitoring information provided by the evaluation can provide a powerful management tool because it highlights how well things are progressing, and spots problems early enough for them to be addressed.
Some interventions to promote positive behavioural change may inadvertently lead to negative outcomes. You need to be able to identify if this is happening and respond.
How might you do this?
- Process evaluation should have been considered alongside other evaluative elements (such as formative and outcome evaluation) at an earlier stage and appropriate plans firmed up as part of the overall evaluation plan
- You can start your evaluation during implementation to check that you are delivering the intervention in the most effective way
- Collect information on how the intervention is actually being delivered, compare that against your original implementation plan, and adjust your actions accordingly. The Guide to Evaluation/Monitoring Indicators provides guidance on what and how you can measure progress in your intervention
- As soon as the intervention is launched, activities should be documented and spot-checked that they adhere to the plan
- Have in place a system for recording target-audience feedback on the intervention activities, communication materials, services, and products
- For more complex, multi-faceted interventions, process evaluation will often need to be piecemeal, as the data is collected separately for each part of the intervention.
Process evaluation works when it draws together the results of all the activities so that you can systematically evaluate overall performance. This allows an overall picture to emerge and highlights which part of the intervention might be refined and improved
- Consult with and involve key stakeholders and staff throughout the implementation process and collect their views on all aspects of the intervention’s progress
- Look on evaluation as an integral part of the implementation process and something that allows you to improve what you are doing.
Don’t think of evaluation as a retrospective activity, only performed once the intervention is finished
- Think carefully about the data that will best help you identify how the intervention is performing and what refinements are needed.
For example, tracking the number of leaflets distributed may be meaningless if they are picked up by people but not read, or if they are read but fail to change behaviour
- Remember that your evaluation methods need to be in place before the intervention is launched
- The collection of information on relevant specific process indicators
- An interim evaluation of how well the intervention is doing and how it might be strengthened