Monday, 3 April 2017

Spreadsheets are people too: statistics and reality

One of the occupational hazards of taking data and statistics seriously – and using social media to do so – is frequent accusations that my focus on hard numbers means that I have my head in the clouds, or buried in a computer, and am therefore somehow detached from “reality”.  I’ve lost count of the number of times a Twitter link to a chart, research paper or ONS data release, making a point about what the evidence shows, is greeted by some version of “why don’t you stop looking at spreadsheets and get out into the real world?

This is particularly the case for immigration.  The response to the overwhelming evidence that, in the UK, there is no measurable impact of immigration on employment, and only a very modest impact on the wages of the low-paid, is often an anecdote, a reference to (perceived) personal experience, or just an injunction that I should spend more time in the pub.  Here’s a recent typicalexample (amazingly and embarrassingly, from a lecturer at King’s, my own university, in a scientific discipline):
Jonathan Portes gave an accomplished performance, asserting that immigration has boosted society, and that it hasn’t depressed wages. Such thinking may be lapped up by a forum of intelligentsia, but try telling that to my landscape gardener friend, who has been severely undercut by waves of east Europeans. 
I was reminded of this by a tweet, attached to a passage in David Goodhart’s new book.  This, Daniel Bentley at Civitas argued, is a “really underappreciated point”

In recent years the falling relative pay for basic jobs, the overwhelming stress on mobility and educational stress, the hourglass labour market and the apartheid system created by a mass higher education system have all made it harder for the mainly Somewhere people doing routine jobs to feel valued and dignified in the modern economy.

Underappreciated, perhaps, because it’s wrong.  Leaving aside the truly bizarre apartheid analogy (Mr Goodhart has a habit of this sort of thing, as I've noted before) the claim that “in recent years” relative pay has fallen for basic jobs is simply false. It’s true enough that earnings inequality did widen sharply in the 1980s and early 90s.  But this isn’t exactly “recently”, and has absolutely nothing to do with the post-1997 immigration Mr Goodhart blames for much of the UK’s contemporary problems.  Indeed, over the last decade, if anything, boosted by increases in the National Minimum Wage, it has improved somewhat, as this chart shows:



Now I would argue that this point – that workers at the lower end of the earnings distribution have done particularly badly - is pretty fundamental to the entire argument Mr Goodhart is making (indeed, the importance of this paragraph to Mr Goodhart’s thesis is presumably why Mr Bentley highlighted it). So you might think that getting it wrong matters rather a lot.  But I have no doubt what Mr Goodhart’s response will be.  As with my review  of his earlier book, “The British Dream”, which listed (very much non-exhaustively) a number of his more glaring factual errors, he wouldsay that I am “sniping in the footnotes” and that I spend too much time with databases and not enough in the “real world”.

But there is a fundamental problem with the argument by those like Dr McCrae or Mr Goodhart that “spreadsheets” or “databases” are somehow divorced from reality, while the experiences of  (selected) individuals represents it.  In fact, spreadsheets – or at least the ones used by labour market economists and, indeed, quantitative social scientists more broadly, are far more closely connected to the “real world” than any individuals’ experience can hope to be.

Consider the Labour Force Survey (LFS), the primary data source for economists analysing the UK labour market. Each quarter the LFS samples 40,000 households, covering 100,000 individuals, a representative sample of (broadly) the UK resident population; lengthy interviews are conducted in person (and subsequently by phone) and cover a wide range of topics in considerable detail, from education, earnings and employment to age, marital and family status, country of birth, and disability.

So when I say that the evidence is clear that immigration doesn’t impact on the employment of UK-born residents, this analysis is formulated in terms of numbers on a spreadsheet or data points in a regression. But behind those numbers are what tens of thousands of real people have told professional interviewers, and in a way which means that the results are in turn representative of lived experience of the UK population as a whole.  

So the statement that, say, the bottom decile of full-time workers have recently seen their pay rise faster than average is not (just) a statement about numbers, or a claim that Mr Goodhart has failed to read the right ONS spreadsheet.  It is a statement about what has – contrary to Mr Goodhart’s claim - actually happened to the pay packets of several million people.  It is the spreadsheet, not what my nephew looking for a job, your cab driver, or Dr McCrae’s landscape gardener friend say that best reflect the real world.

A similar, but even more toxic, disjunction from reality is seen in those who claim that poverty is not about money. For example, Fraser Nelson frequently claims that the Labour government saw poverty solely through the lense of numbers, and that the Brown strategy of attacking child poverty by redistributing money to the poor via tax credits was simply manipulating numbers on aspreadsheet:
Someone who is nudged just above this threshold, with an extra £10 a week, is deemed to be “lifted out of poverty”, although the people concerned would be astounded to hear themselves so described. If they had a family, then their children would be described as being “lifted out of poverty”. So, by precision-bombing the right people with tax credits, you could claim to have lifted hundreds of thousands of children out of poverty.,, instead of fighting poverty, the Labour government spent billions manipulating a spreadsheet – to catastrophic effect.
[UPDATE: Fraser Nelson doesn't like this description of his views. I discuss his contradictions here]

And indeed it is true that this approach was shaped and driven by numbers on spreadsheets – numbers which represented hundreds of thousands of low-income families with more money in their bank accounts to spend on food, shoes or the occasional holiday. There’s plenty of evidence that’s exactly what happened, with commensurate improvements in child welfare.  A comprehensive review by the Joseph Rowntree Foundation concluded – unsurprisingly to anyone who knows anything at all about the topic:
Children in low-income households do less well than their better-off peers on many outcomes in life, such as education or health, simply because they are poorer 
In other words, despite Mr Nelson’s rather convenient assumption, the spreadsheet-driven policy – by ensuring that low-income parents had enough money to look after their children - worked in the real world. 

None of this means that talking to people – or, more relevantly for these topics, rigorous qualitative research, which is generally not what the critics mean – is not useful and sometimes necessary to get a full picture.  But suggesting that data doesn’t represent reality as well as personal experience is simply the opposite of the truth.   If you want to know what’s actually going on in the real world, look at the data.

11 comments:

  1. Fraser Nelson's argument reminds me of the "poverty plus a pound" trope - I hesitate to call it an argument - which was shamefully repeated by Nick Clegg during the Coalition. Anyone who's ever been really short of money knows that Mr Micawber was right - there's a level at which every pound genuinely counts, and one extra pound can make all the difference between happiness and misery. But the difference between those who recognise that and those who don't (or affect not to) has nothing to do with spreadsheets or aggregate data.

    ReplyDelete
  2. Yes, the data is needed to guard us against the inbuilt heuristic that assesses importance by reference to vividness.

    But "lived experience of the UK population as a whole" seems to me a difficult concept. Surely only individuals can actually experience life?

    Some of the pub stories are simply untrue, some are irrelevant (yes, the person you lost out to happened to be Bulgarian, but the data shows that others lost out to British people and nationality is not significant) and some are true but incomplete (yes, you list out to a Russian, but A Pole gave my mate Jim a job). Can we identify vivid counter stories and use the data to put them in context?

    ReplyDelete
  3. As the saying goes, the plural of anecdote is statistics.

    ReplyDelete
  4. What I'm not clear about is the extent to which the data suggests the benefit has been positive across all geographies? Clear about net benefit at a population level, but has there been a negative impact in some areas or consistent across all parts of UK?

    ReplyDelete
  5. Immigration does not need to have a "positive" impact", certainly not on the short and median term, to be acceptable. People have always migrated (which is why millions of English-speakers now think of themselves as Anglo-Saxons)and migration has huge benefits on the long run. Migrants tend to be resourceful, youngish people bringing energy, new ideas and, frequently huge new sources of employment. Who do you think created Britain's modern retail sector, independent television or the top US IT companies? There can be an uncomfortable transition period, but good governments will endeavour to shorten this, rather than base their policies on an ill-informed xenophobia.

    ReplyDelete
    Replies
    1. "Migrants tend to be resourceful, youngish people bringing energy, new ideas and, frequently huge new sources of employment" ... especially if you misjudge the situation and leave your borders wide open while everyone else enforces "transitional arrangements". I've heard bitter complaints from Germany that we bagged all the entrepreneurial risk takers and they were left with the risk averse who wouldn't move until they had seen others prosper.

      Delete
  6. I don't have the expertise to debate the statistics, and I certainly don't think that immigration is the cause of all our woes. However, if people feel worse off then there is no comfort in telling them that the statistics demonstrate that actually your better off than you used to be. It's what people feel that really matters, because that is what drives their actions.

    ReplyDelete
    Replies
    1. But that's not the argument being made. People may feel worse off because they are indeed worse off. However, the data shows that immigration is not the reason.

      If in 2007 and 2008 people had been told that it was the bankers fault that the economy suffered, and that it was the politicians fault for then assuming all the risk in bailing out the banks while not getting a share in the profits, people may have rightly blamed bankers and politicians. That's what happened in Iceland where they lost their jobs and some even went to prison.

      Instead, the financial crisis led to countless articles concentrating on benefits scroungers, the work-shy poor who'd rather have a lie-in than get up to earn their own way. And when the majority of people finally understood how many recipients were in work, a new population group was found to pin the blame on - immigrants.

      This just keeps going on and on - now the new benefits scroungers are those who have more than two children "on the state" as those papers have put it, and all of England sits quietly by as the UK introduces a limit to benefits that is unique in the Western world. The data, of course, once again shows that benefits do not incentivize the poor to have more children, but why let data get in the way of a good story...

      Delete
  7. David Goodharts book does try to describe a reality of a certain section of the population but he then does 2+2=6 by saying there has been a lot of immigration, I can see changes in the labour market, therefore one must have caused the other.

    His terrible terms of apartheid in education standards is presumably trying to look to a period in the 60's and 70's when very few people had formal higher education, in contrast now close to 50% of the present cohort will go on to some form of post school education.

    The people who were employed in a culture which valued apprenticeships and moved into well paid jobs in the 3rd 4th 5th decile of earnings or higher were never competing with the now higher minimum wage. However those jobs were wiped away in the 80's-90's by a combination of destruction of strong trade unions, automation both IT and industrial robots and presumably some impact of opening up the opportunities to the female half of the population.

    Post 1997 immigration has nothing to do with it, but there is a section of the population that is rightly angry that their father could hold down a job and affoard a hose and a car, and they can not, and they want to blame someone.

    ReplyDelete
  8. Excellent! Thanks. I have few people to share this with. It's long overdue for a public acknowledgement of the influence of conscious and implicit bias in the public discourse. Your post is helpful in exposing this.

    ReplyDelete