Evaluation is the process of thinking about what impact you would like your project to have, devising indicators that would demonstrate that this has (or hasn’t) happened, collecting this information, analysing and reporting these findings (either internally or externally), and learning from the findings in order to inform future projects, research, services, etc. (of your own or others).
The Evaluation Pathway
- Setting outcomes & indicators
- Collecting evidence
- Analysing and Reporting
- Learning from and acting on your findings
As an example, an outcome might be that the quality of life and mental wellbeing of people using a service is improved; evidence might be baseline collection (i.e. before the intervention takes place) and 9 month follow-up of EQ5D and WEMWBS; a statistical analysis can be carried out once enough data has been gathered (this could be as complicated as regression modelling or as simple as creating bar charts and tables), and then reported on; briefings may be created for different audiences and stakeholders, such as: GPs, health professionals, policy makers, MSPs, etc. and if any aspects of the intervention were not useful, then they may be removed from future services.
CES Planning Triangle
The Big Lottery Fund (BIG) recommend using a Planning Triangle1 when preparing projects, in order to think about the overall aim, the outcomes that will get you there, and the activities that the project will deliver that will contribute to these. (More information)
Evaluation Support Scotland
Evaluation Support Scotland works with third sector organisations and funders so that they can measure and report on their impact. They aim to make evaluation valuable, relevant and proportionate. Their website gives access to evaluation tools and support.
- A Stitch in Time (Final report) – This three year programme ran to March 2015 and supported the third sector to collect and present evidence about its contribution to Reshaping Care for Older People (RCOP). It focused on third sector organisations working with older people and carers in Lothian.
- Threading the Needle – Scottish Government funded programme to support health and social care commissioners in Glasgow, Fife, North Ayrshire and Perth and Kinross to use third sector evidence to commission outcomes for health and social care.
Inspiring Impact is a programme that aims to change the way the UK third sector thinks about impact. It is a collaboration of eight voluntary sector organisations. You can sign up to the mailing list on their website.
Scottish Co-production Network – Co-production: how we make a difference together – Produced in partnership with the Joint Improvement Team, the ALLIANCE and Governance International, this suite of resources uses videos, case studies to spread understanding of co-production approaches and draw together the principles and practicalities of working this way.(April 2015)
Joint Improvement Team – Readiness for Integration Tool (April 2015) – This is an updated version of the Joint Improvement Team’s existing Readiness for Integration tool. It is designed to be used as part of a self-evaluation or as a stand-alone assessment by organisations aiming to fulfill the requirements of the Public Bodies (Joint Working) (Scotland) Act 2014.
Outcomes Toolbox – IRISS and CCPS have developed a toolbox of resources and knowledge relevant to an outcomes-focused approach. The toolbox brings together some useful insights to consider, information about different outcome focused tools as well as case studies from people using an outcomes focused approach. You can sign up to the IRISS newsletter by going to their website.
Outcomes focused guide to evaluation methods and approaches – A guide to demonstrating the impact of the third sector. It outlines some common outcomes-focused evaluation methods and approaches which may prove useful for third sector organisations working within the context of Health and Social Care Integration and wider health and social care landscape.
Social Impact Scotland is a web resource for third sector organisations, funders, and public sector commissioners. The link above will take you to the methods and tools page which includes a number of established, and some less well known, approaches to understanding and measuring social impact. The site also includes some case studies documenting how organisations have used social evaluation to measure their impact.
The Joint Improvement Team held three Living Well In Localities events in May 2015 across Clydebank, Dundee and Edinburgh, featuring activities, discussions and workshops.
- Flash report (June 2015) – Highlights key points of learning.
- Video playlist (May 2015) – Captures some of the on the day learning and messages.
This briefing outlines the approach to quality improvement implemented by NHS Scotland that has extended to the public service reform agenda more widely e.g. the Early Years Collaborative approach. Quality Improvement is a systematic approach to making change using tried and tested tools and techniques. Published July 2013.
Meaningful and Measurable
Meaningful and Measurable was a Collaborative Action Research project funded by the Economic and Social Research Council (ESRC).
The project brought together 3 academic organisations, 8 practice partners and 4 national stakeholder organisations with a shared interest: adopting a focus on personal outcomes in health, social care and other human services .
Over the past decade, developing approaches to the recording, analysis and use of personal outcomes data has been found to pose challenges at all levels of organisations. Within this project, we explored the tension between:
- Meaning: the need for detailed, contextualised information on individual experience to inform individual planning and service improvement.
- Measurement: the need to aggregate information on personal outcomes to inform decision making at organisational and national levels.
Learning from the Meaningful and Measurable project:
|Strengthening links between identity, action and decision-making (September 2016):|
- Recording Outcomes in Support Planning and Review: Practical Examples (March 2015) – This guide produced by a number of partners in association with the Joint Improvement Team aims to further embed personal outcomes in practice by offering practical examples from a number of service settings.
- Measuring Personal Outcomes in Service Settings (March 2015) – These briefings, collated as a function of the collaborative Meaningful and Measurable project, examine the issues with associated with utilizing a more quantitative approach.
- Supporting Use of Qualitative Data in Service Settings: Collected Briefings from the Meaningful and Measurable Project (May 2015) – This report contains a collection of briefings gathered through the collaborative Meaningful and Measurable project, which addresses a number of commonly encountered misunderstandings concerning the possibilities and implications of working with qualitative data.