The buck does not stop with Reinhart and Rogoff

May 6th, 2013

May 6, 2013

The economics commentariat – and no small part of the political debate – has been consumed in the past few weeks with controversy surrounding a piece of research by my Harvard colleagues (and friends) Carmen Reinhart and Kenneth Rogoff. The article, published in 2010, had been widely interpreted as showing that economic growth is likely to stagnate in a given country once the ratio of its government debt to gross domestic product exceeded a threshold of 90 per cent. But scholars at the University of Massachusetts have demonstrated – and the duo have acknowledged – that the two professors accidentally omitted some relevant data in forming their results, thanks to a coding error. Questions have also raised with respect to how they weighted observations and which data they used.

Many have asserted that the debate undermines the claims of austerity advocates around the world that deficits should be reduced quickly. Some have gone so far as to blame Profs Reinhart and Rogoff for the unemployment of millions, asserting that they were crucial intellectual ammunition for austerity policies. Others believe that, even after review, the data support the view that deficit and debt burden reduction is important in most of the industrialised world. Still others say the controversy has called into question the usefulness of statistical research on economic policy questions.

Where should these debates settle? First, the whole experience should change the way we approach economic and statistical research. Profs Rogoff and Reinhart are rightly regarded as careful, honest scholars. Anyone close to the process of economic research or financial markets will recognise that data errors such as the ones they made are distressingly common.

Indeed, an internal investigation by JPMorgan into the $6bn loss it made last year on the “London whale” trade found mistakes not unlike those made by Profs Reinhart and Rogoff. Simple errors in a model meant that the bank dramatically underestimated the risks that it was running. In future, authors, academic journals and commentators need to devote more effort to replicating significant research results before broadcasting them widely.

More generally, no important policy conclusion should ever be based solely on a single statistical result. Policy judgments should be based on the accumulation of evidence from multiple studies done with differing approaches. Even then, there should be a reluctance to accept conclusions from “models” without an intuitive understanding of what is driving them. It is right and understandable that scholars want their findings to inform the policy debate. But they have an obligation to discourage and, on occasion, contradict those who would oversimplify and exaggerate their conclusions.

Second, all participants in policy debates should retain a healthy scepticism about retrospective statistical analysis. Trillions of dollars have been lost and millions have been unemployed because the lesson was learnt from 60 years of experience between 1945 and 2005 that “American house prices in aggregate always go up”. This was not a data problem or misanalysis. It was a data regularity – right up until it wasn’t.

The extrapolation from past experience to future outlook is always deeply problematic and needs to be done with great care. In retrospect, it was folly to believe that with data on about 30 countries it was possible to estimate a threshold beyond which debt became dangerous.

Even if such a threshold existed, why should it be the same in countries with and without their own currency, with very different financial systems, cultures, degrees of openness and growth experiences? And there is the old chestnut that correlation does not establish causation. Any tendency for high debt and low growth to go together might reflect the way that debts can rapidly accumulate as a consequence of slow growth.

Third, while Reinhart and Rogoff’s work, even before the recent replication efforts, did not support the claims made by prominent figures on the right in the US and UK regarding the urgency of deficit reduction efforts, the joy taken by some on the left from their embarrassment is inappropriate.

It is absurd to blame Reinhart and Rogoff for austerity policies. The political leaders advancing austerity measures made their choice of policy first, and then cast about for intellectual buttresses. While there may be no threshold beyond which debt becomes catastrophic, and while the British and US experiences both suggest that fiscal contraction in a slack economy where interest rates are near zero is inimical to growth, it is a grave mistake to suppose that debt can or should be accumulated with abandon.

On all but the most optimistic forecasts, further actions will be necessary almost everywhere in the industrial world to assure that debt levels are sustainable after economies recover.

This is not the time for austerity, but we forget at our peril that debt- financed spending is not an alternative to cutting other spending or raising taxes. It is only a way of deferring those painful acts.

The writer is Charles W. Eliot university professor at Harvard and a former US Treasury secretary