article

The fortunes of forecasters are telling, so why do we still listen?

“It’s frightening to think that you might not know something, but more frightening to think that, by and large, the world is run by people who have faith that they know exactly what’s going on” Amos Tversky 

Human beings are constantly surprised at the way life turns out. People living on flood plains in the UK in early 2014 were surprised to be pumping out flood water from their houses. Lottery ticket buyers are mostly disappointed when they don’t win. And economists regularly have to change their forecasts in the light of unexpected news. None of these people were expecting to be surprised, no-one ever is, of course.

Economists have a particularly tricky time of it. Rarely have they collectively forecasted anything correctly, consistently over a multi-year phase. Tracking the aggregate consensus expectation for GDP, inflation and profits illustrates the problem. Without exception, they show changing beliefs over time, with the largest forecast errors tending to occur around turning points, or recessions.  The chart below shows the changing expectations over time for Brazilian GDP forecasts.

Forecasting charts Mar 14 no1

Sometimes there are patterns to the changes in beliefs. When structural changes occur, for example the fall in core inflation during the last decade or so, it takes many years for the consensus of economists to finally embed that change in their forecasts. At other times an exogenous event causes a significant change in the economic environment, causing everyone to revise their forecasts. An example of this is the credit crisis and bankruptcy of Lehman Brothers. The liquidity crisis that took place in Q4 2008 drastically changed the outlook for 2009.

The fact that economic forecasting is hard to do is nothing new. That idea has been part of the basic economics syllabus at school for a long time and for three good reasons. Firstly, economies are complex systems. The relationships between the various parts of the system are not static and structural changes do occur. Secondly, there are long and variable lags between policy changes and their impact on the real economy. This is true particularly for monetary policy. And thirdly, economic data are prone to revisions, sometimes a long time after the event, which means that policy changes are made with incomplete or inaccurate data.

More recently it has become clear that economic forecasting is likely to be an even greater challenge. The combination of globalisation, technological change and the aftermath of a credit boom and bust will change many of the existing relationships forever. Globalisation has been a factor for a while now. A key factor in this is the entry to the global labour market of millions of workers in emerging markets. Technological change and its impact on economies was written about a lot in the 1990s but it appears that we are only now beginning to see the full implications in terms of an environment of winners and losers. Finally, the ongoing implication of financial deregulation and the struggle to deal with the aftermath of a credit boom and bust is a completely new phenomenon and one which will take time to fully understand.

Even Central Banks, who have hundreds of highly qualified economists and expensive computer models do not manage to generate a superior forecasting track record compared to anyone else. The experience of the ECB from 2011 onwards illustrates the problem.

In 2011 the European Central Bank raised interest rates twice to deal with what they saw as a risk to the inflation outlook. At the time that they hiked rates, inflation was 2.7%, and the ECB stated that risks to the medium-term outlook for price inflation remained on the upside. From that point in time, measured inflation began to fall in the Euro area and is now expected to be 1.1% for 2014. Did they get their forecast wrong? Perhaps the tighter monetary was required to ensure that inflation did not become a problem. But would the ECB really have put rates up in July 2011 if they knew that from that point real growth would decelerate, with the Euro area as a whole back in recession in 2012? Trichet said that the underlying momentum of economic growth in July 2011 continued to be positive. By September he acknowledged that there were downside risks to growth. When asked at the September 2011 press conference whether the deceleration in growth implied that earlier interest rate hikes needed to be reversed, he answered that inflation expectations need to be firmly anchored and he did not see any threat of deflation.  By early November, and with new President Mario Draghi at the helm, interest rates were reduced once again by 25 basis points, followed by a further four cuts in rates down to a Euro-area low of 0.25% in late 2013.

The Bank of England has not had an easy time of it either. In 2012 David Stockton was asked to review the Monetary Policy Committees forecasting capability – his paper can be read here. He concludes that “The MPC’s recent forecast performance has been noticeably worse than prior to the crisis, and marginally worse than that of outside forecasters”.

Subsequent to Stockton’s report, the Bank of England’s forecasting capability was further discussed in 2013 by Ben Broadbent in a speech in May in which he described three categories of errors – known unknowns (for example change in oil supply and political surprises which account for the existence of the fan charts), forecasting mistakes (would explain why some forecasts are better than others) and unknown unknowns – a whole host of things that weren’t even countenanced beforehand, including structural breaks.

Then later in the year, Charlie Bean gave a speech containing outcomes relative to fan charts, and included a discussion of the introduction of forward guidance. Significantly, he concludes that “We have discovered that we knew even less about the workings of the economy than we thought we did.” Figure 2 below illustrates the GDP outturn versus the prior expectation as presented by the Bank in fan chart form.

Forecasting charts Mar 14 no2

Having established that forecasting is hard to do and that very few people succeed in making consistently accurate economic forecasts, what is more perplexing is that human beings don’t seem to learn from their mistakes. Perhaps many don’t look back to see whether life turned out as they had previously expected – this could be because it would be too painful to admit to the errors, or perhaps there is nothing to learn from doing so. It would also harm their ongoing credibility to have to admit to a series of inaccurate forecasts.

And the poor forecasting track record of most economists does also raise another question – why do people persist with making these forecasts? There are several reasons for making forecasts. It is both enjoyable to do and conveys a sense of power, since the person making the forecast appears to know more about the future than do their audience. Economists also make forecasts because they are asked to do so, by people who think that the forecasts are a useful input to other activities, such as economic policy making and making investment decisions.

So if forecasting is so hard to do on a consistent basis, what is the alternative? One option would be to not make forecasts. When asked, perhaps by a journalist, what might happen to the economy next year, the correct answer should really be “I don’t know” or “there is a wide range of possible outcomes”. Anyone who answered in this way would probably not be asked again, instead someone who is prepared to make forecasts would be in the seat. A more practical approach is to think about the economic regime and what that implies for the key variables – output, inflation and profits. Having a bias to think that inflation is not a problem (due to globalisation and technological change), means that at times when markets become worried about inflation it is easier to work out whether bonds are an attractive investment. But that is very different to joining in the game of making calendar year CPI forecasts and tracking where they are compared to the consensus. Observing that capital tends to be winning over labour, again due to those same structural factors, implies that a rising profit share of GDP is completely understandable and does not suggest that margins are more likely to fall than rise in the period ahead. By focusing on a structural view and understanding why short term cyclical forecasting is fraught with dangers, there is a chance of being able to ignore some of the noise and perhaps of being less surprised at how the world turns out compared to prior expectations.

When the Queen visited the London School of Economics in November 2008, she described the financial crisis as “awful” and asked why no-one had seen it coming.  The director of research politely answered that “At every stage, someone was relying on somebody else and everyone thought they were doing the right thing”, but perhaps he could have also explained that while the recent events were quite extreme, the lack of foresight was nothing new. He could even have quoted John Kenneth Galbraith who said “The only function of economic forecasting is to make astrology look respectable”.