Tuesday, 21 February 2012

Work experience: does it work? (updated)

[Updated 22/2 at 8pm with new analysis and chart from Inclusion].


The amount of political "heat" surrounding the government's Work Experience programme seems to be in inverse proportion to the amount of policy "light". Critics describe it as akin to "slavery", while the Secretary of State retorts by describing them as "modern day Luddites".  Not only is this exchange of insults neither sensible nor constructive, it obscures the more interesting and important issues.

Foremost among these is surely the question of whether work experience actually works; that is, does it actually improve the job prospects and opportunities of the young people it is supposed to help.  Iain Duncan Smith is clear on this. He argues
The fact is that 13 weeks after starting their placements, around 50 per cent of those taking part have either taken up permanent posts or have stopped claiming benefits.
This is indeed a fact, backed up by DWP analysis here.  But it's not a very meaningful one, because in itself it proves nothing; we don't know what would have happened if they hadn't been put on the programme in the first place.  In fact, I was surprised that the number was so low, for the following reasons:
  • it is well known that most people claiming Jobseekers' Allowance (JSA) leave quite quickly.  The UK has a very dynamic and flexible labour market; so although the most recent labour market statistics showed a small rise in the claimant count of about 7,000 (to just over 1.6 million) this reflected more than 300,000 people signing on, and somewhat fewer ceasing their claim
  • remember that - as Iain Duncan Smith is careful to make clear, albeit only implicitly,in the quote above - that leaving JSA doesn't actually mean getting a job.  Especially for young people, many people who stop claiming benefits don't get a job; they may return to education or training (generally a good thing) or drop out entirely (not so good).  But it is reasonable to conclude that far fewer than 50% of this cohort actually got a real job.
  • on the whole, I would expect the young people moving on to work experience placements to be relatively "employable"; Jobcentre Plus advisers wouldn't normally send young people on a work experience placement unless there was a reasonable prospect of a successful outcome. If the young person had real basic skills or attitude problems, one would expect them to try some other intervention first before sending them on a placement doomed to failure.
So how should we evaluate whether this 50 percent number represents success? The obvious comparator is JSA off-flow rates in the absence of the programme.  And here the numbers do not look good. As the Secretary of State himself pointed out in another context:
the vast majority of jobseekers spend only a very short time in that situation; over half are back in work within 3 months.
This is probably untrue as stated, for the reason given above (most jobseekers leave JSA in three months, but that doesn't mean they are in work) but the basic point that off-flows from JSA are high is correct, as pointed out by DWP  here  
Off-flows from JSA remain high - almost  60% of claimants leave within three months and almost 80% leave within six months of making their claim.  
Moreover, the same document also points out that young people leave JSA even faster
Jobseekers aged 25 and over are significantly more likely to have claims lasting more than one year than jobseekers aged 18 to 24,
So well over 60% of young jobseekers leave JSA within three months, suggesting that the record of the Work Experience programme - 50% off benefit in three months - is pretty unimpressive at best, comparing poorly with what happens to young people on JSA in any event.  It's certainly not in itself a statistic that the Secretary of State should be trumpeting as a success. 


Now this is not definitive - without a proper control group and a counterfactual, we do not know what would have happened to the participants without the programme.  Maybe I am wrong, and in fact those who go on the programme have very poor characteristics (they might have already been unemployed for a while, for example), and would have done even worse without it.  Without proper evaluation, we just don't know. But certainly the evidence and analysis so far published by DWP does not make a good case.


So what should we conclude?  Well, as usual, "more research is needed"; DWP should attempt to produce a proper counterfactual analysis that would allow us to come to a considered judgement on the programme's success.   


But more broadly, even if current outcomes are not particularly encouraging, we should be looking to improve the programme rather than discard it. Properly structured work experience can make a significant contribution to the employment prospects of young people.  And - if the work experience offered does genuinely do that - I think the element of compulsion is justified.  But, as the ACEVO Commission on Youth Unemployment (of which I was a member) found:
currently work experience placements are too often short, of poor quality, with young people given little to do and the placement poorly linked to their wider education or the advice and guidance they receive
A finding which could well explain the rather disappointing outcomes emerging from DWP's statistics. I would therefore argue that policy makers would be better occupied dealing with these issues and ensuring that work experience is genuinely worthwhile - for the participants, not for the employers- with the real, not theoretical, prospect of a job at the end of it.  That would be more constructive than hurling accusations or making assertions on the basis of a very selective reading of the evidence. 


UPDATE:


The excellent Inclusion have been in touch to point out that they have, in fact, already done a simple version of the counterfactual analysis I suggest.  Here's their chart, taken from their report "Youth Unemployment: A million reasons to act": 




Their measured conclusion is the following:

 "This appears to show that the youth work experience scheme has had no additional impact on the speed at which young people leave benefit, and may have actually led to them spending longer on benefit than they would have done.  However, these figures require some caution  – the stated intent of the Department has been to target work experience at those with the biggest barriers to work, who would likely have had offflow rates below the average for all claimants."
So not definitive, pending proper analysis from DWP, but as yet it seems clear that there is no substantive evidence suggesting work experience has a positive impact on the employment prospects of young people. 


4 comments:

  1. Thank you for this - some actual light to go with all the heat...

    ReplyDelete
  2. Even if Work Experience does not hasten finding a permanent job for those concerned, there are still two other possible beneficial effects which I think you missed out. First, in that an element of compulsion is involved, the process may induce JSA claimants who have no intention of doing WE to find work more quickly.

    In fact I’ve come across more than one study over the years showing that a threat to withdraw unemployment benefit results in a substantial proportion of the unemployed finding work in short order.

    Second, where someone does two or three months of WE, that is two or three months of work or extra GDP which we arguably would not have had but for WE. Of course it would be ludicrously optimistic to claim that every person hour of work under WE is an addition to GDP (i.e. that WE does not to some extent displace regular employees). But there is a theoretical reason for thinking that WE type schemes bring a finite increase in GDP. See:

    http://ralphanomics.blogspot.com/2011/12/unproductive-employees.html

    Re empirical evidence as to the effect of WE type schemes, there is loads of evidence from around Europe going back over the last twenty years or so. See:

    http://ralphanomics.blogspot.com/2012/01/effect-of-temporary-subsidised.html

    ReplyDelete
  3. The graph is very misleading because it implicitly assumes that the start point of both cohorts (those on JSA and those on the work experience programme) are the same. In fact, according to the DWP (see this document http://www.dwp.gov.uk/docs/eia-work-experience.pdf) the work experience programme is aimed at those who have been on JSA for “3-12 months” (see para 35). So the graph should show the work experience line flat at 100% for 13 weeks before beginning to decline.

    It is interesting that the rationale set out in the document is sensible and apparently well thought out, and also clearly acknowledges that most people on JSA leave it relatively quickly – “80 per cent leave within six months” (para 21). The whole (stated) point of the scheme is to target those who do not leave JSA rapidly. It may be that, in practice, that is not happening – but at the very least it would seem sensible to comment on the programme on the basis that it is designed to deal with the “difficult” JSA cases. It is disappointing to see that in fact most (all?) “fact check” and policy commentators have started from the opposite perspective, i.e. that there is a direct comparison that can be made to “normal” JSA churn rates, albeit with half-hearted caveats that in fact the cohort on the work experience may be different.

    Finally, having now read the DWP document cited above one can’t help feeling that the programme was established with all the right intentions – as a serious attempt to really help people who otherwise would otherwise struggle to make their way in the world of work. To see it so successfully traduced, and the companies that participated accused of exploiting “slave labour”, is a depressing.

    ReplyDelete
  4. Naukri24x7.com would like to inform to all the Job seekers registered on our site that we do not make any extensive verification of the details submitted by employers / Placement consultants registered on our site and also of the contents / Jobs posted by them.
    Colleges in India

    ReplyDelete