The Alliance for Useful evidence devoted its Christmas party to a discussion on what constitutes good evidence for policy-makers – and to a discussion of some examples of what doesn’t. Carey Oppenheim, Chief Executive of the Early Intervention Foundation, reflects on the event and on her own organisation’s evidence-based work.
Despite winds ripping through much of Scotland and England, a sizeable gathering sheltered in NESTA’s decidedly funky offices to celebrate Christmas with the Alliance for Useful Evidence. Party games take on a rather different form in this setting – rather than charades – members of the audience submitted their own ideas for good and bad evidenced based policy making – with early years education cited as What Works – and some of the welfare changes cited as What doesn’t Work.
The former Cabinet Secretary Sir Gus O’Donnell introduced proceedings with some reflections on what constituted robust evidence in his time in Government – the five tests to join the Euro (accompanied by mountains of documents) definitely came out on the positive side as did automatic enrolment into a work based pension – the wonders of human inertia making a vital contribution to building future savings.In contrast, energy and migration policy had remained relatively resistant to the harsh light of evidence. Sir Gus finished with a flourish – the pride in getting Randomised Controlled Trials – into the heart of government. And a real success it is too, not because RCTs are the only way to assess what works – we might not want to do an RCT for nuclear weapons, say – but it creates a climate and expectation that proposals in government should be properly tested
Professor Henry Overman, Director of the Local Economic Growth Centre, began with a question about the evidence base for Santa Claus, but felt that we shouldn’t pursue it in case there were any children present. More seriously his centre is working its way forensically through the government evaluations of local growth strategies. In conversation later, Henry recalled the experience of giving evidence to the Treasury Select Committee on HS2’s stated £15bn boost to the economy – when asked what’ no firm statistical foundation’ meant for this figure he replied ‘it’s made up, to put it bluntly’. As a result he found himself featured in almost every newspaper. It takes guts to be a What Works Centre.
And so to EIF – also new on the block – we are focused on getting to the root causes of social problems though Early Intervention for children from conception to 19. So early in a child’s life as well as early before problems become entrenched.
At the moment we are creating a continuum of evidence which captures what has worked, what hasn’t and what is promising in Early Intervention. We are also identifying the gaps in evidence. And while Randomly Controlled Trials are important in the assessment of programmes in particular, they are not the whole story. Qualitative evidence can tell us important things about the users of a service or the practitioners providing the service. An interesting innovation with a sound logic model might be worth testing. As a foundation we are interested in supporting organisations to build a stronger evidence base for their work. However, it is important to be clear both that we recognise a continuum of evidence and nonetheless recognise standards of evidence. Valuing methodological diversity doesn’t mean anything goes.
And to doubly ensure our work is relevant on the ground we are working together with 20 Pioneering Early Intervention places across the country. As well as quality evidence we are interested in learning how to implement Early Intervention. The core qualities of what makes an excellent family support worker or health visitor may be as important as the programmes they deliver.
It was indeed local areas, like those we are working alongside, who provided the most enjoyable part of the evening with their sheer enthusiasm. Many are trying out innovative practice and wanting to provide an evidence base for it – from Youth Zone in Blackburn – a collaborative project to raise young people’s self-esteem, to finding a control group for a strategy to reduce exclusions from school, to learning what makes a successful evidence based centre and so on. Perhaps we are, as Geoff Mulgan, chief executive of NESTA, has suggested, in a golden age of evidence.
A version of this article first appeared on the Alliance for Useful Evidence blog: http://www.alliance4usefulevidence.org/early_intervention_foundation/