Tracking the Bank of Canada's forecasts during the recession

Globe and Mail journalist Jeremy Torobin called me a while back to talk about Bank of Canada forecasts for this article, and that call prompted me to go back and look at how the Bank's forecasts evolved during the recession.

The Bank's Monetary Policy Reports publish projections for GDP growth rates, and it is these numbers that are discussed in the media. But what really matters for monetary policy isn't the growth rate of GDP, but its level - in particular, the difference between actual GDP and potential output. So in what follows, the various forecasts will be presented as projections for the path of future GDP.

Another thing to remember is that the Bank has to work with the data that are available when the forecast is being put together. As we shall see, these data are repeatedly revised over time.

We'll start in 2008. It's important to understand the timing here: even though the January 2008 forecast was published in the first quarter of 2008, GDP for that quarter wasn't available. Nor were the data for the preceding period; the 2007Q4 numbers weren't announced until the end of February 2008. So each projected path has a diamond indicating the last available data point at the time.

In January 2008, we were still 8 months away from the collapse of Lehman Brothers, but the outlook for the US had already started to deteriorate. The Bank's forecast for Canada was for weaker-than-previously-predicted growth in the first half of 2008.

By April, it had become apparent that the US slowdown was more severe than anticipated, and this produced a significant downward revision in the projected path, and this path wasn't significantly revised in July.

The October forecast was based on GDP data up to the second quarter, but the financial crisis was enough to force a substantial downward revision in the projected path of output. Even though GDP was expected to grow after a small decline in 2008Q4, the return to potential was going to take a long time.

The black line in that graph is actual GDP growth as of the November 2010 release up to 2010Q3 - the most recent data. Its path lies below the 2008 projections, meaning that estimates for GDP have been revised downward since 2008. If those forecasts look too optimistic in retrospect, then data revisions are part of the story.

On to 2009:

In January 2009, the most recent data were from 2008Q3, which was up over the previous quarter. But the signals from the other indicators from the fourth quarter were enough to increase the duration of the projected downturn from one quarter to three; the trough was expected to occur in the second quarter of 2009.

The forecast of April 2009 was made while the economy seemed to be in free fall, although there weren't much in the way of hard data available at the time. The Bank's projection was revised downward yet again, and the projected duration of the recession went from three quarters to four.

July 2009's forecast was based on the GDP data from the first quarter, which was the worst of the recession. But since other, more recent data such as employment suggested that the downturn was slowing (July would mark the trough of the employment recession), the Bank moved its estimate for the trough back to 2009Q2. The October 2009 forecast wasn't revised much from July.

Once again, look at how the lines shift steadily down with each forecast. Not only was GDP falling, the numbers from the previous periods were being continually revised downward throughout 2009.

And here's the graph for 2010 along with the January 2011 projection:

If the story of 2009 was trying to predict when and where the economy would bottom out, that of 2010 was one of trying to forecast the speed of the recovery. The January 2010 forecast projected a fairly steady return to potential, but growth turned out to be stronger than predicted; the April and July forecasts were revised correspondingly upwards. The unexpected slowdown in 2010Q2 and 2010Q3 doesn't seem to affect the Bank's medium-term outlook; the last four forecasts all converge at the same spot, albeit with different starting points.

Looking back over the past three years, it's clear that the Bank - like pretty much everyone else - didn't predict the financial meltdown and underestimated its effects when it did hit. But it's not fair to say that it was taken completely off-guard: the Bank had been steadily revising downward its forecasts in the months before the crisis, and it cut interest rates by 125 basis points in the first half of 2008.

I think this sort of exercise is important for people who pay attention to these sorts of forecasts. Projections are continually revised as new data come in, and these data are themselves subject to revision. Forecast errors cannot be avoided; the best we can hope for is to learn from them.

 

Comments

[Simon van Norden]

I agree that this kind of exercise is very important (and three cheers to Stephen for taking into account data revisions!) One of the reasons I think it is important is because it highlights the problems of using monetary policy to prevent recessions.

Look at the July 2008 forecast, for example. You are sitting at 2008:3, but have only data up to 2008:1. Assuming that monetary policy has a 2-3Q lag, your policy settings are largely based on your forecasts for 2009:1 and beyond. However, your forecast for 2009:1 is disasterously wrong. In retrospect, you will find that the economy has effectively had zero growth for three quarters and is about to contract sharply. However, you believe that growth is resuming after a brief pause.

In fact, it will take you another two quarters (2009:1) before you forecast any contraction in output, and one further quarter before the depth of the recession becomes clear. By that time, policy reactions will not their main effect until the start of 2010.

Let me be clear that I'm not saying this as criticism of the Bank of Canada; I think this is probably as good as we can expect from our macroeconomic forecasters. However, that has implications for the degree to which monetary policy can hope to stabilize the economy.

[Greg Tkacz]

I think a lot of people will agree that 2008-09 was an "unusual" period for macroeconomic forecasting, in the sense we were venturing into unexplored territory. For instance: U.S. and Canadian GDP growth posted their worst quarter ever in 2009Q1 (or, at least, it was the worst quarter in the last fifty years); interest rate were driven down to zero, which had never happened here before; and the full implications of the financial crisis were difficult to quantify in a macroeconomic perspective, given the rapid aversion to risky assets that developed in the second half of 2008 which drove some credit spreads to unseen levels.

Despite all of the above, the forecasts turned-out to be pretty sensible. After the GDP revisions were eventually released, I think the Bank's forecast of GDP growth in 2009Q1 of -7.0% was pretty close, and its forecast of 2009Q3 for the end of the recession was also on the mark as early as January 2009.

[Stephen Gordon]

I agree - and as far as giving guidance for monetary goes, the forecasts weren't so far off as to induce any egregious mistakes in monetary policy, which is what matters. In retrospect, the Bank might have wanted to go to zero faster than it did, but it's not clear how big a mistake that really was.

[Angelo Melino]

It would be interesting to see how the evolution of the Bank of Canada's forecasts compared to those of the big banks.

[Greg Tkacz]

Sounds like Campbell and Murphy would have to update their work again to answer the above, Angelo. I do recall the Governor getting a lot of flack in Jan 2009 when he released a 2010 forecast for GDP growth that was quite a bit higher than those of the private sector (but, in retrospect, I think the Bank's 2010 forecast was pretty accurate). I think his response to a query about why the Bank's forecasts were higher than the private sector's was "We don't deal in optimism or pessimism -- we deal in realism."

[brendon]

at the beginning of 2010 - the 5 banks forecast for GDP growth were:

RBC: 3.1
BMO: 2.5
SC: 2.6
TD: 2.4
CIBC: 2.0

Bank of Canada: 2.9

Actual growth in 2010 is likely going to be right on the Bank's original forecast, though the quarterly distribution is way off what was originally assumed.

[brendon]

I recall reading something recently that suggested that professional forecasts are not out to minimize mean-squared error anyway - their incentives are skewed towards getting press. Hence Jeff Rubin's long-shot calls, RBC's enduring optimism and CIBC's consistent bearishness.

[Simon van Norden]

I'm think some of the good points raised by Greg Tkacz underline how limited monetary policy can be. (Hi Greg! You're still at the Bank of Canada aren't you?)

Greg's right that 2008-09 is unusual because the collapse in economic activity was so large. Those really big collapses are of course the ones that we'd most like to avoid. The success or failure of monetary policy matters most at times like those.

Greg's also right that even though no decline in activity is forecast as late as 2008:Q4, the forecasts were "sensible." And that's at the heart of the problem; with the biggest economic collapse in two generations getting underway, even sensible forecasts are unable to give useful warnings.

However, I don't understand Stephen's claim that "the forecasts weren't so far off as to induce any egregious mistakes in monetary policy, which is what matters." I thought that first graph showed that the forecasts throughout 2008 missed the collapse in 2009. Given the lags, that's when policy would have had to react to prevent the biggest economic declines in a generation. Stephen, what am I misunderstanding here?

Let me say again, I'm not faulting the BoC. I think they did as well as can be expected. My point is that given the nature of monetary policy and macroeconomic forecasting, there's only so much that can be done. 2008-09 is proof.

[Simon van Norden]

Another way to put the BoC's performance into perspective is to compare the reaction of monetary policy to that of fiscal policy. We had elections in late 2008; in the English leaders' debate in late October (after the collapse of Lehman Bros) all *five* party leaders solemnly stated that, if elected, they would only run surpluses.

[Stephen Gordon]

That's more or less what I was thinking. I wasn't comparing the BoC's actions to the perfect foresight case; I was judging them by what could reasonably be expected given the information available at the time. The alternative scenario I had in mind was going to zero a month or two earlier than they did.

[Nick Rowe]

Simon: click on Greg's name. He's at StFX.

I remember a meeting of 20 economists with the PM in Toronto. Late 2008/early 2009. We advised him it was OK to run a deficit. He came out of the meeting and said he was going to run a deficit. I think he had already decided, and just wanted cover. He's not stupid.

[Nick Rowe]

I went to 0.25% on 15th January 2009 http://www.cdhowe.org/english/monetary_policy_council/mpc_pressrelease_jan_15_2009.html I got that one righter than most. David Laidler was close at 0.50%. The median CD Howe was 1%, and the Bank did 1% on January 20th.

The Bank finally cut to 0.25% on 21 April 2009. http://www.bankofcanada.ca/en/fixed-dates/2009/rate_210409.html

From my perspective, the Bank was 3 months behind the curve. But I was lucky that time.

[Stephen Gordon]

Wait - you were at a meeting with the PM and you didn't blog about it? For shame!

[Nick Rowe]

I was far too new and nervous, both to blogging and to PMs, back then. There were loads of us. CD Howe MPC plus some others. We each got 2 minutes to say our pitch. I think I said "It's OK to run a deficit at times like this; and keep an eye on CMHC". But I can't really remember much. Except being nervous, and not wanting to make a complete fool of myself.

[Simon van Norden]

Again, remember the policy lags.....deciding in Dec. 2008 that you're going to run a decifit in the future is a good decision.....but much too late to avoid a large and painful recession.

Of course, a PM who had campaigned on running a deficit wouldn't have needed cover...but that would not have made much difference in timing; the PM reversed his campaign promises *fast*.

But where were we in Oct 2008? Who tried to make an issue out of the unconditional pledge to only run surpluses when it was clear (this was after Lehman Bros, remember) that some kind of recession was coming. I didn't. You?

[Nick Rowe]

Yep. Then Ted Carmichael and Andrew Spence joined me at 0.25% February 26. The CD Howe median was 0.50%, and the Bank did the same 0.50% March 3.

http://www.cdhowe.org/english/monetary_policy_council/mpc_pressrelease_feb_26_2009.html

[Nick Rowe]

Simon: "But where were we in Oct 2008? Who tried to make an issue out of the unconditional pledge to only run surpluses when it was clear (this was after Lehman Bros, remember) that some kind of recession was coming. I didn't. You?"

October 19 2008 Stephen was saying we shouldn't panic about a projected $10b deficit.

http://worthwhile.typepad.com/worthwhile_canadian_initi/2008/10/a-10b-deficit-isnt-what-it-used-to-be.html

October 22 2008 Stephen was saying it's OK to run a cyclical deficit but was not yet ready to recommend active fiscal policy.

"If the federal and provincial governments do nothing, there's a good chance that they'll run a deficit, either this year, next year or both. And that's okay: we've spent the better part of a generation getting our fiscal house in order - we can handle it.

If Canada does go into recession (let's all remember that available data indicates that we were not in recession as of September), then we should not exclude the possibility of fiscal measures beyond the automatic stabilisers. But we're not there yet."

http://worthwhile.typepad.com/worthwhile_canadian_initi/2008/10/what-should-canadian-governments-be-doing-right-now.html

This is what I said November 29 2008:

"My own view, for what it's worth, is that Canada does not need a fiscal stimulus right now, but there's a possibility it might need one soon, so the government should get everything ready to introduce one as quickly as possible, but not pull the trigger quite yet."

http://worthwhile.typepad.com/worthwhile_canadian_initi/2008/11/does-canada-need-a-fiscal-stimulus-discuss.html

With hindsight, we were too optimistic. But we were saying it's OK to run a deficit if needed. We don't look too bad, in retrospect.

(The danger of blogging is that everything you think is recoded to haunt you later.)

[Stephen Gordon]

Here's a post from September 2008:

As unreasoning aversions go, the fear of a budgetary deficit is probably not the worst for voters to have. And there once was a time - back when deficits were large, persistent and unsustainable - when the fear of a deficit was in fact a rational response to a truly scary situation.

But that time is past. The deficit-phobia we see now is a good example of hysteresis: the lagging of an effect behind its cause. Public debt is under control, and if the economy goes pear-shaped for a few quarters, we needn't panic if we run a deficit for a year, or maybe even two.

[Simon van Norden]

Take a bow; you both deserve it!

[Greg Tkacz]

(Quick disclaimer: I've been away from the Bank for six months, so my "cooling-off" period is long over, allowing me to play in a sandbox such as this one.)

Anyway, from my recollection in the trenches:

By October 2008 we were definitely getting some signals of the impending doom for 2009Q1 -- numbers such as -10% for the U.S. GDP growth were initially being tossed-around, but being revised on a weekly basis as new information was coming-in. In early October of that year there was even a coordinated interest rate reduction by major central banks -- I believe the BoC dropped the overnight rate by 50 b.p. outside of the regular rate-setting periods. I don't think there was much talk about dropping to zero right away, but certainly we were working on plans on what had to be done if we ever did reach the ZLB.

[Nick Rowe]

I had forgotten that the BoC dropped to .50% outside of an FAD. What I said above about the BoC timing is wrong. I tried to get the CD Howe to drop outside of a regular FAD, but couldn't persuade them. Then the BoC did it anyway.

It's good you are teaching Greg. "The one that got away".

[Dirk]

Nick, I asked a similar question on Scott Sumner's blog recently: is there any evidence that forecasters at central banks have any ability to forecast better than someone rolling some dice and adding or subtracting the result from normal trend growth? (I might try this experiment and see how my dice do compared to official forecasts.)

[Dirk]

I think I answered my own question. According to the Federal Reserve Bank of Philadelphia’s own error statistics:

http://www.philadelphiafed.org/research-and-data/real-time-center/survey-of-professional-forecasters/data-files/NGDP/SPF_Error_Statistics_NGDP_3_AIC.pdf

The survey of forecasters performed better than a simple autoregression over the period 1985 - 1996, however the AR beat the forecasters slightly on forecasts 3 - 5 quarters out between 1997 - 2008. Over the latter period, the forecasters beat the AR slightly (but insignificantly) 2 quarters out, and beat it moderately 1 quarter out (although it isn’t clear that the forecasters had no data from that quarter before making their forecast, as the deadline for the forecasts is the middle of the quarter they are forecasting).

So it isn’t clear that forecasters are making better predictions than an AR.

That said, the error report itself concludes that the forecasters did forecast more accurately than the AR since for the whole 1985 - 2008 period the forecasters won the contest, but 1997 - 2008 looks to me like a reversion to the mean. Not beating the benchmark for the past 11 years doesn’t seem too robust. I suggest we stop paying any attention to forecasters.