Work experience: we still need better evidence

Last week I wrote here that the debate on work experience seemed almost entirely divorced from the evidence of whether work experience actually improved the employment opportunities of jobless young people.  I pointed out that, when Iain Duncan Smith and Chris Grayling argue that around half of those on the scheme leave benefits within 13 weeks, this in itself tells us nothing about the success of the programme, since many would have left benefits without the scheme.  It is worth reproducing again Inclusion's chart here, since it remains the best data we have: 

What has happened since then, and have we learned more?  Unfortunately, exaggerated claims appear to continue.  The Prime Minister, at last week's PMQs, stated

"around half of them [young people on the scheme] are actually getting work at the end of these schemes." 

On the basis of the data and information we have at the moment, this is clearly false, as fullfact.org pointed out here. While the Prime Minister was presumably badly briefed, there was much less excuse for Sarah Teather two days later, who repeated again (on Any Questions):

"50% of people who start these schemes have got a job"

Now, the fact that Ministers have made exaggerated (and sometimes false) claims for the success of the scheme definitely does not imply it doesn't work.   For that reason, I argued 

"DWP should attempt to produce a proper counterfactual analysis that would allow us to come to a considered judgement on the programme's success."  

This would involve  the construction of a proper "control group" of non-participants, similar to participants on the scheme, so we could see whether outcomes were better for the latter.  And interestingly, Chris Grayling, on the Today programme, hinted that DWP were doing just that:  

"Now we’re now crunching further numbers and all of the evidence we can see is that this does better than simply leaving people on JSA. It actually helps more young people get into work."

This is very encouraging.  DWP should never have published the misleading analysis here, which does not even mention the need for a counterfactual analysis to assess the success of the programme. 


But - and here I would like to be resolutely optimistic, both about evaluation and analysis, and the programme itself - there is an opportunity here to improve the quality of the current debate.   DWP should publish the evidence referred to by Chris Grayling as soon as possible, and open up the data and analysis to outside scrutiny. This would help explain both what welfare-to-work programme can and cannot do, and how they should be evaluated.   My view - shared by most of those who study and evaluate such programmes - is that work experience is likely to improve outcomes for young people, albeit not as much as implied by Ministers' claims.  I hope I am right, and look forward to seeing the evidence. 


Furthermore, the current intense scrutiny of the programme should be taken as an opportunity, not to rubbish it, but to improve it.  The ACEVO Commission on Youth Unemployment (of which I was a member) found:

"currently work experience placements are too often short, of poor quality, with young people given little to do and the placement poorly linked to their wider education or the advice and guidance they receive"

This is to a large extent the responsibility of employers at least as much as government. Channel 4's Factcheck found that only one in five of those doing work experience at Tesco was offered a permanent job - and unfortunately this is entirely consistent with what the ACEVO Commission was told.  


While this does not contradict the analysis above (they might have got jobs somewhere else) this suggests to me that Tesco are fully abiding by the spirit of the scheme, which is supposed to offer the chance of a permanent job to those who successfully complete a placement.  Research by Alex Bryson, now at NIESR, and others found back in the 1990s that this was the key to success for such schemes.  Do we really think that only one in five of the young unemployed people who go on the current scheme proves to be sufficiently competent and motivated to fill a permanent position?  To make the scheme work for everyone - employers, government, and most of all young unemployed people - the most important point is that it needs to offer them a fair chance of a proper job at the end.