Economists Slow to Detect Dodgy Data
By Gareth Hutchens
The Age (Melbourne), April 21, 2013. All Rights Reserved.
If we needed more evidence that economics is not a science, we have it now.
A shock wave hit the economics world this week when two of its most famous practitioners—Kenneth Rogoff and Carmen Reinhart—were found to have produced some very dodgy data to support their claims about the consequences of high government debt.
It comes back to a research paper of theirs, Growth in a Time of Debt widely quoted since it was published in 2010. The paper shows that if government debt becomes too high—say, around 90 per cent of gross domestic product—then economic growth will almost always suffer. Global policymakers have taken it to mean that if countries with too-high debt levels want to kick-start flagging economies then they ought to begin the resuscitation process by reducing debt levels first.
It has been repackaged into a simple message: Reduce your debt and economic growth will begin to pick up. But the corollary is that highly indebted governments should not try to spend their way out of economic stagnation because spending more will only make things worse. It has helped to provide the intellectual justification that the proponents of austerity wanted; thus the wave of austerity policies washing around the world since 2010. Millions of people have suffered because of it.
But the intellectual edifice for the global austerity movement was severely weakened this week after it emerged that professors Reinhart and Rogoff had made some basic errors in their interpretation of data that supported their research.
The paper shows Reinhart and Rogoff had omitted data, made a mistake in their Excel spreadsheet, and used a bizarre statistical methodology, all of which skewed results. It set the academic world ablaze.
As Nobel laureate Paul Krugman wrote: "In this age of information, maths errors can lead to disaster. NASA’s Mars Orbiter crashed because engineers forgot to convert to metric measurements; JPMorgan Chase’s "London Whale" venture went bad because modellers divided a sum instead of an average. So, did an Excel coding error destroy the economies of the Western world?"
Reinhart and Rogoff have acknowledged they made a spreadsheet error, but they also say it didn’t affect their result much.
"It is sobering that such an error slipped into one of our papers despite our best efforts to be consistently careful," they said. "We do not, however, believe this regrettable slip affects in any significant way the central message of the paper or that in our subsequent work."
But in the brouhaha that followed, a few people have been asking why it took so long for Reinhart and Rogoff’s research to be tested.
Imagine you’ve handed your assignment in at school. You make some wonderful claims in it about the way the world works. Your research—based on an analysis of data of 44 countries spanning 200 years—has led you to discover that high government debt to GDP ratios above a "90 per cent threshold" almost always lead to a slowdown in economic growth. It’s a law that seems to hold no matter what you throw at it. You can compare different countries in disparate regions, and once you try to take account of the fact that a country’s political and financial systems evolve over time you can mix and match these things across centuries of data and the law stays the same.
It’s a striking thesis. And luckily for you, you’re not expected to hand your data in with your assignment so your work can be checked. Your teacher takes your word for it. That’s not how the scientific method is supposed to work.
It is sobering to be reminded that economic analyses, produced in this way, can have such influence in the real world. It’s worth remembering next time we hear some politician referring to "economic modelling" that supports his or her claim.