I am not an econometrician. I'm not very good at econometrics. Read this post in that light.
I think that what some econometricians are doing is horribly wrong. They are making a wildly implausible assumption.
I want to give my reactions to what James Hamilton said about what Christopher Sims said. James is nicely clear.
"We've observed that when monetary policy becomes looser, GDP will often temporarily grow a little faster. But these responses don't happen instantaneously, and they don't last forever. A key question economists would like to ask is how big these effects are and how they unfold over time.
Prior to Sargent and Sims, there was a great deal that was ad hoc about the way economists would approach these questions. What I would characterize as Sims' greatest contribution is noting that it is possible to summarize the key facts about these relations without having to believe in what he referred to in his 1980 paper Macroeconomics and Reality as "the discouraging baggage of standard, but incredible, assumptions macroeconometricians" had been used to making. Sims suggested that we should always begin with straightforward and completely objective forecasting questions, such as, if I learned that the Fed is setting a lower interest rate this month than I had been expecting, how should that cause me to revise my forecast of what GDP growth is going to be 1, 2, and 3 quarters from now? Sims developed the objective tools we can use for answering those questions that do not depend on my ideological biases or preconceived theories. He suggested that the answers that such an objective analysis produces should be recognized as the basic facts that any model needs to be able to explain. This was a radical step in the direction of transforming macroeconomics into a much more objective, scientific inquiry, and is part of the bedrock of the discipline these days."
Suppose I see the Bank of Canada cut the overnight rate target. And suppose this comes as a surprise to me. My immediate reaction is: "That means the Bank of Canada thinks that demand is weaker than I thought the Bank of Canada thought it was." In other words, I revise my beliefs about the Bank of Canada's beliefs.
What happens next depends on whether I think I know anything about demand that the the Bank of Canada doesn't know.
Suppose I think that I know more than the Bank, and that the Bank just made a mistake. In that case I revise upwards my expectation for GDP growth over the next few months. An exogenous loosening of monetary policy will cause higher GDP growth.
Suppose instead I think there is nothing that I know that the Bank of Canada doesn't know; and that the Bank of Canada knows some things I don't know. My knowledge is a proper subset of the Bank's knowledge. That's a plausible assumption for an econometrician to make. The Bank has all the data the econometrician has, and probably a bit more, because there is almost always some omitted variable in the econometrician's equation. Hell, if the econometrician had all the data the Bank had, the Bank's move shouldn't be a surprise to the econometrician in any case.
So I revise my belief about the state of demand in accordance with my new belief about the Bank's belief. "If the Bank cut the overnight rate from 5% to 4%, that must mean the economy is weaker than I thought it was, and that the Bank needed to cut the overnight rate from 5% to 4%."
How do I revise my belief about GDP growth for the next couple of quarters?
I know that the Bank gets most of its data with a lag. I know the Bank reacts to that data with a lag. I also think that it takes time for the Bank's actions to affect GDP growth. I therefore know that the Bank is always "late" in reacting to changes in demand. Being behind the curve is an occupational hazard of central banking. So if the Bank cuts the overnight rate from 5% to 4%, that means it should have cut the overnight rate a few months back, to offset the fall in demand.
I therefore revise downward my expectation of GDP growth for the next couple of quarters.
If I were using Sims' method, I would conclude that a surprise cut in the overnight rate leads to a reduction in GDP growth.
Econometricians believe they can identify monetary policy "shocks" by using Sims' method. How is that even possible? Does it mean that the Bank rolls a dice to decide on monetary policy, so that the Bank is using data (the number on the dice) that the econometrician does not observe but knows to be irrelevant? I find that assumption to be wildly implausible, and the econometrician who attributes all actions that he cannot explain to "mistakes" to be wildly arrogant. If the Bank does something that the econometrician didn't predict, isn't it more plausible that the Bank has useful data that the econometrician left out of his equation, and the Bank is responding to it? The econometrician can't include everything, without blowing all his degrees of freedom.
I can think of one context where the econometrician's assumption might make sense. Econometricians have final revised data, while the Bank has only initial estimates. In principle, an econometrician could forecast the Bank's actions using real time data, and then identify "mistakes" by using deviations between real time and final revised data.
I don't know if any econometricians actually do anything like this. But I do think that what they are doing now is just wrong. If someone who knows more than you does something you didn't expect him to do, that doesn't mean he made a mistake.
Alternatively, you can do something like what I and Gabriel Rodriguez did.