What difference does training make?

A theory of change approach to evaluation
What difference does training make?

Monitoring and evaluation plays a vital role in conservation. It tells us how much progress we are making towards our goals and ultimately if our actions are making a difference or not. Equipped with this information, we can make better decisions about what works and what we need to do differently. The importance of evaluation to training interventions is no exception, however, it is uniquely challenging!

Unlike the outcomes of many other conservation actions, the outcomes of training can be much more difficult to define and measure. Trainees can experience not only a change in their skills and knowledge but also in their emotions, attitudes and perceptions. The time it takes for this change to manifest can also vary widely from one individual to the next. It can be years before some trainees apply what they learned and it can take even longer for their actions to make a difference. On top of this, there are a huge number of factors that can influence how much progress an individual makes.  

Part of my role at Durrell has been to develop and test a system that attempts to overcome these challenges and systematically evaluate the difference our training programme makes over time. To do this, we first created a theory of change, i.e. a model that describes the pathway of change we expect from an individual completing a training course to achieving conservation goals. This theory of change acts as a step-by-step guide to evaluation, from which we can identify what we need to measure in order to monitor progress and evaluate success. 

We use a series of online surveys to gather information throughout our theory of change, each incorporating both quantitative and qualitative data collection methods. The quantitative data tells us if we are achieving what we expected to, and the qualitative data gives us an insight into the unexpected and unmeasurable change an individual might experience. The more time that passes, the more challenging it becomes to attribute outcomes to training. In response, we rely more on qualitative methods over time and measure the extent to which training contributes to progress made. The more we learn, the more we are able to refine our theory of change and improve our understanding of what we can achieve.

Despite its challenges, this new evaluation system has provided us with a wealth of information, enabling us to explore new ways of maximising our training impact and better communicate the difference we make to stakeholders. 


Go to the profile of Adam Barlow
almost 2 years ago

Hi Rachel, thanks so much for sharing this as it is something that we also are trying to tackle. In case it helps others by building on what you have mentioned already -  what also helped us a lot was:

1. Splitting impact into what was in our control (basically the number of people we trained) and what was out of our control (what they did with our training and what impact and change in efficiency that helped them to achieve)

2. Only setting objectives for what was inside our control but still (as you do already) capture the impact/change that the training contributed to but that was outside of our control.

This helped us a lot to simplify and better report on our training impact. The process we used to determine what was inside and outside our control is here and here is our plan in which we document our monitoring approach - any suggestions on improving it very welcome! 

Go to the profile of Rachael Gerrie
over 1 year ago

Hi Adam - apologies I just spotted your response!

It would be great to chat some more and share lessons learned during development and implementation. Maybe we can set something up on here?

We also found it helpful to clearly define which results were within our control before developing an evaluation system. In our theory of change, this includes knowledge, self-confidence and motivation which we measure immediately after each training course.