James Hamilton on Christopher Sims and identifying monetary policy "shocks"

I am not an econometrician. I'm not very good at econometrics. Read this post in that light.

I think that what some econometricians are doing is horribly wrong. They are making a wildly implausible assumption.

I want to give my reactions to what James Hamilton said about what Christopher Sims said. James is nicely clear.

Here's James:

"We've observed that when monetary policy becomes looser, GDP will often temporarily grow a little faster. But these responses don't happen instantaneously, and they don't last forever. A key question economists would like to ask is how big these effects are and how they unfold over time.

Prior to Sargent and Sims, there was a great deal that was ad hoc about the way economists would approach these questions. What I would characterize as Sims' greatest contribution is noting that it is possible to summarize the key facts about these relations without having to believe in what he referred to in his 1980 paper Macroeconomics and Reality as "the discouraging baggage of standard, but incredible, assumptions macroeconometricians" had been used to making. Sims suggested that we should always begin with straightforward and completely objective forecasting questions, such as, if I learned that the Fed is setting a lower interest rate this month than I had been expecting, how should that cause me to revise my forecast of what GDP growth is going to be 1, 2, and 3 quarters from now? Sims developed the objective tools we can use for answering those questions that do not depend on my ideological biases or preconceived theories. He suggested that the answers that such an objective analysis produces should be recognized as the basic facts that any model needs to be able to explain. This was a radical step in the direction of transforming macroeconomics into a much more objective, scientific inquiry, and is part of the bedrock of the discipline these days."

Suppose I see the Bank of Canada cut the overnight rate target. And suppose this comes as a surprise to me. My immediate reaction is: "That means the Bank of Canada thinks that demand is weaker than I thought the Bank of Canada thought it was." In other words, I revise my beliefs about the Bank of Canada's beliefs.

What happens next depends on whether I think I know anything about demand that the the Bank of Canada doesn't know.

Suppose I think that I know more than the Bank, and that the Bank just made a mistake. In that case I revise upwards my expectation for GDP growth over the next few months. An exogenous loosening of monetary policy will cause higher GDP growth.

Suppose instead I think there is nothing that I know that the Bank of Canada doesn't know; and that the Bank of Canada knows some things I don't know. My knowledge is a proper subset of the Bank's knowledge. That's a plausible assumption for an econometrician to make. The Bank has all the data the econometrician has, and probably a bit more, because there is almost always some omitted variable in the econometrician's equation. Hell, if the econometrician had all the data the Bank had, the Bank's move shouldn't be a surprise to the econometrician in any case.

So I revise my belief about the state of demand in accordance with my new belief about the Bank's belief. "If the Bank cut the overnight rate from 5% to 4%, that must mean the economy is weaker than I thought it was, and that the Bank needed to cut the overnight rate from 5% to 4%."

How do I revise my belief about GDP growth for the next couple of quarters?

I know that the Bank gets most of its data with a lag. I know the Bank reacts to that data with a lag. I also think that it takes time for the Bank's actions to affect GDP growth. I therefore know that the Bank is always "late" in reacting to changes in demand. Being behind the curve is an occupational hazard of central banking. So if the Bank cuts the overnight rate from 5% to 4%, that means it should have cut the overnight rate a few months back, to offset the fall in demand.

I therefore revise downward my expectation of GDP growth for the next couple of quarters.

If I were using Sims' method, I would conclude that a surprise cut in the overnight rate leads to a reduction in GDP growth.

Econometricians believe they can identify monetary policy "shocks" by using Sims' method. How is that even possible? Does it mean that the Bank rolls a dice to decide on monetary policy, so that the Bank is using data (the number on the dice) that the econometrician does not observe but knows to be irrelevant? I find that assumption to be wildly implausible, and the econometrician who attributes all actions that he cannot explain to "mistakes" to be wildly arrogant. If the Bank does something that the econometrician didn't predict, isn't it more plausible that the Bank has useful data that the econometrician left out of his equation, and the Bank is responding to it? The econometrician can't include everything, without blowing all his degrees of freedom.

I can think of one context where the econometrician's assumption might make sense. Econometricians have final revised data, while the Bank has only initial estimates. In principle, an econometrician could forecast the Bank's actions using real time data, and then identify "mistakes" by using deviations between real time and final revised data.

I don't know if any econometricians actually do anything like this. But I do think that what they are doing now is just wrong. If someone who knows more than you does something you didn't expect him to do, that doesn't mean he made a mistake.

Alternatively, you can do something like what I and Gabriel Rodriguez did.

Comments

[Sandwichman]

Nice!

[Ritwik]

"If I were using Sims' method, I would conclude that a surprise cut in the overnight rate leads to a reduction in GDP growth."

Isn't this better framed as "...leads to a reduction in my beliefs about GDP growth." My beliefs in GDP growth over the next two quarters (or whatever the lag duration is) would surely be reduced, and my beliefs about GDP growth post that would be unchanged, precisely because of the central bank's actions. Such unanticipated monetary policy shocks wouldn't be 'mistakes' because

1) My beliefs about GDP growth are not central to GDP growth
2) The central bank's action has been the correct one because it ensured that GDP growth returned back on track, while otherwise it may not have.

Or perhaps I'm saying something rather stupid here.

By the way, one of the interesting things about Sims's work is that he doesn't require that standard monetary policy (open market operations or targeting announcements) have the result that we just assume they do. In fact, his (and Sargent's) work shows that very often they might fail to have such effect if not 'fiscally accommodated'. Again, I hope I have not been stupid with my interpretation here.

But the most interesting thing about Sims for whatever I have read over the last two days is that he is much, much more than an econometrician. He is an innovative monetary theorist and model builder to boot. This point seems to be missing from almost every discussion on him, so I blogged about it. :)

[Ritwik]

Though, his willingness to use the fiscal theory of the price level to arrive at his results in his models means that you (and other market monetarists/ monetary disequilibrium theorists) might end up thinking of him as a crank. He is quite convinced that one can 'exchange bonds for goods' simply enough and so 'money' is not really that special. :)

[Min]

"Econometricians believe they can identify monetary policy "shocks" by using Sims' method. How is that even possible?"

I once had a pamphlet by Savage on "the best statistical test in the world, the Interocular Traumatic Method: It hits you right between the eyes." (The quote may not be exact. :)) If econometricians can identify shocks, then shouldn't they be able to show them so that they hit you right between the eyes? (I would think that shocks, in particular, would be amenable to the Interocular Traumatic Method. :))

[Min]

Hmmm. Or perhaps the question is one of interpretation. Perhaps by "shocks" you mean something like, as they say dans La Belle France, "Les Fuckups". ;)

[K]

I think the question of whether the BoC knows more or less that some econometrician is less interesting than the question of whether they know more or less than the market. Also, the difference in opinion between the BoC and the market is readily observable from the front end of the curve (OIS or BAX futures in Canada). It may be wrong, but it's not *crazy* to assume that BoC deviations from the market expectation is an "error". You just need to assume 1) that the market is rational and 2) the market expects the BoC to be rational too.

[Gregor Bush]

But Nic, if the IRFs extracted from, say, a three variable VAR (GDP, CPI and policy rates) show a negative response of output to policy rate shocks, doesn’t that tell you that those shocks are, in fact, something that the central bank has missed? Or at the very least, that the deviation in the policy rate from the path implied by past movements in output and inflation tells us something about the future paths of output and inflation? (maybe the central bank didn't mind temporarily missing its target by a modest amount because it’s risk averse and saw an elevated risk of a financial crisis that would have led to a much larger miss had it occurred and therefore lowered rates more than would be expected given the current conditions)

It seems to me that if you're right that it is indeed impossible for econometricians to identify monetary policy shocks, the IRFs should be close to zero (or incorrectly signed).

What am I missing?

[Nick Rowe]

Sandwichman: Thanks! First time you've agreed with one of my posts?

Ritwik: "Isn't this better framed as "...leads to a reduction in my beliefs about GDP growth.""

Yes, but econometricians want to say something about whether exogenous monetary shocks *cause* changes in GDP growth. See my response to Gregor below.

Interesting post on Sims, by the way.

Min: I didn't know that the interocular method originated with Savage! Historians can sometimes identity exogenous shocks, for example if there's a new Governor, with a very different target, or economic model, that would plausibly be an exogenous shock. But see my response to Gregor below.

K: "I think the question of whether the BoC knows more or less that some econometrician is less interesting than the question of whether they know more or less than the market."

I tend to agree, but I carefully dodged that question in my post. It's too hard to bring that in too.

[Nick Rowe]

Gregor: Good question. I think I can give a good answer.

Let me first simplify.

Assume we are talking about inflation rather than GDP, and assume the Bank is targeting 2% inflation at a 2 year horizon. And ignore any funny stuff about real time data and data lags.

I'm an econometrician using an information set I to estimate the Bank's reaction function. Suppose I identify what seem to be random shocks to the Bank's overnight rate that are not explained by my equation. Further suppose that these random shocks are correlated with inflation 2 years ahead. How do I interpret this correlation?

My answer:

1. I conclude that the Bank is looking at and responding to additional information J that is not included in my information set I. And that J is uncorrelated with I.

2. If those "shocks" are positively/negatively correlated with 2 year ahead inflation, I infer that the Bank was reacting too weakly/too strongly to J as an indicator of inflationary pressure. I do not infer that the estimated beta of the monetary policy "shock" on 2 year ahead inflation is a measure of the effect of a truly random monetary policy shock on 2 year ahead inflation. Rather, I infer that the estimated beta is a measure of the *difference* between the effect of a truly random shock and the inflationary pressure created by an increase in J to which the Bank did not respond.

Only if I knew a priori that J was the roll of a dice, that gave no true information on inflationary pressure, would I be correct in making the inference that beta correctly measures the effect of an exogenous monetary policy shock on 2 year ahead inflation.

[Phil Koop]

I did not in fact find Hamilton very informative and much preferred Mark Thoma here). (See also his previous post).)

Given Thoma's more detailed explanation, I do not follow how your complaint really engages with Sims' work. Can you and Sims (& Gregor) really be speaking the same language here?

[Gregor Bush]

Nic,
That IS a good answer.

So my VAR will think the "errors" are purely exogenous innovations when in reality it's the Bank's overreaction to a variable that I'm missing that is giving me forecasting power. So my coefficients won't give me the impact of a "true" innovation. If somebody asked me "Gregor, if Carney went completely mad and raised the overnight rate by 300bps tomorrow, what would happen to inflation?" and I used the IRFs from my VAR I would get the answer wrong.

But if the Bank always overreacts to an increase in J, and those overreactions show up as errors in the VAR, and those "errors" help predict inflation, is the VAR not still giving me useful information? If (hypothetically) all I cared about was placing bets on whether inflation would be above or below the Bank's forecast 2 years out, would I necessarily want to ignore the forecast from the VAR, even if I know that the "innovations" are not purely exogenous monetary shocks?

[Mik]

Nick, aren't you essentially just saying that you can't infer structural shocks from the VAR innovations if there is omitted variable bias? If so, that's a fair and pretty common criticism of VARs. One way to get around this is to use a factor-augmented VAR, thereby incorporating as many variables as you want without losing degrees of freedom (see Stock and Watson (2005)).

[Nick Rowe]

Phil: I only saw one of Mark Thoma's posts, and skimmed it, after writing this. I didn't see anything I wanted to change. But let me have a closer look.

Gregor:

1st paragraph. Yep, that's exactly what I'm saying. (OK, one very minor chnage: change it into "overreaction or underreaction".)

2nd paragraph. Agreed. The VAR is useful for atheoretical forecasting. But it tells you nothing about the effects of monetary policy. And it won't even be any good for forecasting if Mark Carney flips.

Mik: "Nick, aren't you essentially just saying that you can't infer structural shocks from the VAR innovations if there is omitted variable bias?"

I'm not sure about the distinction between "structural" shocks and shocks. I'm saying that if you find a monetary policy shock there *must* be an omitted variable. And only an econometrician who knew a lot more than the Bank of Canada could safely assume that the omitted variable was the roll of a die, which is what they implicitly assume.

Factor analysis doesn't seem to help either. If you identify a shock, it just means you don't have enough dimensions in your factor analysis.

[Nick Rowe]

God does not play dice with the universe. And the Bank of Canada does not play dice with monetary policy. That is my maintained hypothesis. An econometrician who thinks he can identify exogenous innovations in monetary policy and tell us the effects of those exogenous innovations adopts the opposite hypothesis.

I used to tease guys at the Bank of Canada: "If you really believed your econometrics, you would be demanding the Governor's resignation!"

[Nick Rowe]

Phil: From Mark Thoma's second post: "Other pitfalls can occur, for example, when there is optimal control or when expectations of future variables are in the model."

I expect I am talking about optimal control.

[Phil Koop]

OK, I'm with you. Thanks, Nick.

[Phil Koop]

"I'm not sure about the distinction between "structural" shocks and shocks."

I think that Gregor just meant "structural" as opposed to reduced-form: identifying a shock in the reduced-form model is a mechanical exercise. I agree with you about not being able to recover a structural interpretation via factor analysis though.

[Nick Rowe]

Thanks Phil.

I'm a bit puzzled by the (non-)reaction to this post though. Given my wild(ish) words, I was curled up in the fetal position, under my desk, waiting for a swarm of time-series econometricians to tell me I didn't know what I was talking about. (Which would be at least partly true).

Maybe they all already know this? But then why do they keep on doing VARs claiming to identify exogenous monetary policy shocks? Or did they stop doing this years ago, and I never noticed?

[Scott Sumner]

No need to curl up for me, unless you are worried I'd give you a big hug for doing this post.

I vaguely recall that VAR modelers assumed that monetary shocks have no economic impact during the quarter in which they occur? Is that right? We know that QE2 raised US oil prices and hence the CPI in the quarter in which it occurred.

Once we create the proper monetary instrument (NGDP futures) we'll be able to do high quality VARs linking NGDP futures innovations and RGDP changes.

Until then, our best bet is to look at pre-WWII data, when there were huge monetary shocks that are easily identified.

BTW, I like your study of the impact of US monetary policy on HK. I've often argued that HK is the best country to use when estimating a Phillips Curve

[Patrick]

Somewhat OT: Oh my... http://www.bloomberg.com/news/2011-10-13/kocherlakota-says-fed-risks-its-credibility-with-easing-as-inflation-rises.html.

[Stephen]

I think a major problem is equating GDP with economic growth. If the government prints up a bunch of money and spends it, the GDP goes up. But has anything of value actually been produced? No, so why the fixation on GDP?

[Jacques René Giguère]

Stephen: because GDP as understood and used today in a monetary economy is essentially a measure of purchasing power being transferred in the form of money. That's why leasure , even if we value it enough to sacrifice other productions and the monetary income that come with it is not included. No purchasing power is transfered.
And therefore Americans think they are richer because they work to pay prison wardens and that the French are poor because they goof up six weeks in August.

[Nick Rowe]

Stephen: we still want to find out what the effects of monetary policy are. Does it affect real activity? Does it just affect the price level? Neither? Both? And how much? For the purposes of this question, GDP (either real or nominal) is just a convenient proxy for economic activity, either adjusted for inflation or not.