Why there's so little good evidence that fiscal (or monetary) policy works

Will fiscal (or monetary) policy work to prevent a recession? This is perhaps the central question of macroeconomics. We ought to know the answer, and we ought to have overwhelmingly good evidence to support our answer. But we don't. It's still being debated.

There's a reason we don't have good evidence.

There are two main macroeconomic policies: fiscal policy (F); and monetary policy (M). We can argue about how to measure fiscal or monetary policy, but let's just ignore that difficulty. Macroeconomists believe that F and M affect output (Y) and/or prices (P). Again we can argue about how to measure output and prices, and whether it's output or prices or both, but let's just ignore that too.

Here's the reason for our ignorance:

Suppose a fiscal authority is using F to try to keep Y at some target level Y* (you can think of Y* as full employment, or the natural rate, or just Y*, as long as Y* is independent of F and M). You can replace fiscal with monetary authority, F with M, and Y with P, and it won't make any difference to my argument.

So the fiscal authority sets F, conditional on M and all other information (I), so that its expectation of future Y is equal to Y*. In maths:

E(Y/{F,M,I}) = Y*

Unless the fiscal authority has perfect foresight, and knows perfectly how fiscal policy works, it won't hit the target exactly. There will be some forecast error Y', where:

Y' = Y-E(Y/{F,M,I}) = Y-Y*

Here's the punchline: if the fiscal authority has rational expectations, the forecast error Y' by definition will be uncorrelated with each and every element in the information set {F,M,I}, so if Y* is a constant (or growing at some constant rate) there will be zero correlation between Y and F (and also zero correlation between Y and M, and between Y and anything else in I). There will be no evidence that fiscal policy affects output. There will be no evidence that monetary policy affects output either. Same with prices (or inflation) if they are targeting prices.

Engineers know a lot more about engineering than macroeconomists know about macroeconomics (I know this from chairing several PhD defences in engineering). It's not (or not just) that macroeconomists are less intelligent than engineers. Our job really is harder.

Think of a good experimental design: randomised control variables, holding everything else constant, etc. Now think of the worst possible experimental design. Imagine something that engineers or psychologists might dream up over beers for a laugh, or to illustrate what not to do. That's what economists face. It's as if our lab assistants (the fiscal and monetary authorities) were deliberately trying to make our (economists') lives as hard as possible. They do this, of course, not to spite us, but to try to make everyone else's lives as easy as possible. To get a good experimental design for economists, both the fiscal and monetary authorities would need to be malevolent.

Even a moderate degree of incompetence by the fiscal and monetary authorities doesn't help us. If their expectations are far from rational, we will see correlations between Y (or P) and F, M, and I. But these correlations will not tell us what we want to know. All they tell us is whether the fiscal authorities are systematically under- or over-reacting to changes in M and I (a positive correlation means one, and a negative correlation means the other). As long as fiscal authorities react only to M and I, even if they react by the wrong amount, there will be perfect colinearity between F, M, and I, so the econometrician is helpless.

Even total incompetence by just one authority won't help us. If the fiscal authority is changing F at random, but the monetary authority is responding (even if incorrectly) only to F and I, then we still get perfect colinearity, and the econometrician is still helpless.

We could get good evidence if both the fiscal and monetary authorities were totally incompetent or malevolant. Now, do you really want better evidence that fiscal policy works?

(I wrote a working paper on this once.)

Comments

[PrestoPundit]

For one thing, engineers don't begin using pseudo-scientific "models" of phenomena utterly ungrounded in and essentially contrary to all of the foundational theories of the relevant sciences.

You wrote:

"Engineers know a lot more about engineering than macroeconomists know about macroeconomics (I know this from chairing several PhD defences in engineering). It's not (or not just) that macroeconomists are less intelligent than engineers. Our job really is harder"

[Adam]

Basically, you're saying that we don't have a very good "lab" in economics in which all other variables can be held constant, save the one under study. This is true across the social sciences, which is why we never get theories in, for example, political science that are as ironclad (or close to it) as, say, gravity.

What happens when people pretend that we know how to tweak the variables so as to obtain the political results we want? For starters, the Iraq war. Oops, turns out people don't behave in quite the way we thought they would (lots of political scientists said not to go to war, but my point is that you can't predict everything - even the ones who were against the war couldn't tell you much about exactly what would happen, just that it would probably be bad).

The question then is why economists insist on pretending that we know exactly how to tweak the variables so as to obtain the desired economic result. Decrease interest rates, purchase equity in failing institutions, loan money to depressed manufacturers, etc. It's all a pile of nonsense - policy-makers haven't the slightest clue what they're doing. Sure, they know more than I do if I were in their shoes. Much more. Infinitely more. But it's like comparing the size difference between a pebble and a mountain in the scale of the solar system. One's bigger, but they're both totally negligible in any meaningful way. So instead of pretending like we know how to fix the economy, why not just admit that we simply don't know enough to tinker with the machine and that anything we do is probably just going to make things worse?

[Nick Rowe]

Adam: replying to each of your 3 paragraphs in turn:
1. it's worse than not having a very good lab; we have a very bad lab, about the worst possible, for looking at the aggregate effects of monetary and fiscal policy (it's not so bad when economists are looking at other things). Our "lab assistants" (policymakers) are deliberately trying to minimise fluctuations in the variables we are interested in studying, just the opposite of what good lab assistants would do. And they don't play dice with monetary and fiscal policy (again, just the opposite of what good lab assistants would do, thank God).
2. Basically agree.
3. I think we do know something, but our knowledge is not learned in the normal way. For example, suppose I don't know which way a car's steering wheel works (If I turn it clockwise, does the car turn right or left?). I watch two drivers: one who believes the "clockwise-right" theory, and a second who believes the "clockwise-left" theory. The first driver stays on the road; and the second crashes. I then believe the "clockwise-right" theory is true. In monetary policy, we do have historical examples of "clockwise-left" central banks, like Germany in the hyperinflation, and perhaps Zimbabwe now, where they don't see the causal link between monetary policy and inflation, and print more money during a hyperinflation because they think people need more money when prices are rising. To continue the analogy, we also use theory, which is like trying to examine the linkage between the steering wheel and the front wheels, to see which way the car ought to turn if we turn the wheel clockwise.

And even total ignorance isn't always a recommendation to do nothing. If I was in an airplane which seemed to be heading for a cliff, I would sure pull or push some lever.

We have some idea how the linkages work. We have some idea about how economies have behaved historically when policymakers have different theories about how the linkages work. But we don't have the gold-standard, where we take a car out on a wide-open field and experiment by turning the wheel in different directions to see what happens.

By the way, my views about monetary and fiscal policy above are decidedly non-standard. I think most economists would disagree with what I wrote. And they could be right, and I could be wrong.

[Stephen Gordon]

Another thing to consider is sample size: recessions are infrequent.

[Richard H. Serlin]

so if Y* is a constant (or growing at some constant rate) there will be zero correlation between Y and F (and also zero correlation between Y and M, and between Y and anything else in I).

One reason why this problem is not that bad is that Y* is often not a constant (or growing at a constant rate). For example, in the late 70s and early 80s in the U.S., the Fed. specifically and clearly decided to go for a much lower Y* in order to break double digit inflation, and that episode showed a strong correlation between M and Y.

In addition, you can compare the variability and general behavior of Y as a function of how actively M and F are managed. For example, since M and F have been very actively managed we've had the great moderation. Take a look at graph 3 on page 7 of this paper by Carreras and Tafunell. It shows U.S. GDP from 1831-2000. Look at the massive swings in GDP before the introduction of a strong modern Keynesian policies towards the end of the depression.

Finally, a key problem in economics is a common snobbery about data and evidence that can lead to embarrassing conclusions. There can be extremely strong evidence and logic chains that rely on far more reasonable and realistic assumptions than the evidence from formal fancy regressions and models, but many economists snobbishly ignore this very strong and valuable evidence and logic leading to terrible conclusions and advice.

[DW]

Well, most engineers count on people trying to break their stuff. Most economics count on everything being perfect and people being rational. Therein lies your problem.

[Winslow R.]

Would it be that hard to set up a controlled micro/macro economic experiment?

My kids, along with many of their friends, played an online game called Runescape where they basically lost all the 'money' they had earned from turning trees into bows, through a purchase of santa hats that subsequently deflated in price after Christmas. The game has bank accounts but no loans and there is no central banker adjusting interest rates.

I've heard game designers hire economists to help set up the payment system/market place and design how these games work. There are thousands/millions of people playing these games and hundreds/thousands of these games.

Couldn't one of them be convinced to be aligned closer to the 'real' world?

[Nick Rowe]

Stephen: I agree, but I would argue that (one of) the reason(s) why we have such a small sample size of recessions is that our lab assistants (the Bank of Canada) are deliberately making the sample size small.

Richard: I basically agree with your paragraphs 1 and 2 (you lost me on 3). We can look at historical episodes when the target changed (your paragraph 1), and we can look at historical episodes when the drivers did and did not stay on the road, and compare the driving styles (your paragraph 2). But I am both less and more optimistic than you.

Less optimistic, because those historical episodes may have been in economies with very different structures from our economy today (different parameter values), and also when we change the policymaker's target the whole pattern of expectations-formation changes, and the economy might respond very differently than it does to the sort of policy actions that take places as instantiations of a given policy targeting regime (Lucas Critique).

More optimistic, because I think we can slowly improve policy by watching for correlations between Y (or P) and F (or M) and I, and learn from them in which direction we need to change the parameters in the feedback rule (I explore that idea in the working paper I linked to). But it's slow learning, gradient-climbing, learning-by-doing, and won't work well in novel or extreme cases.

Prestopundit and DW: I think this is a more general problem about how to estimate a system for control purposes, using data observed while the system is being controlled. It's independent of the nature of the structural theory of the system.

[Nick Rowe]

Winslow: what you describe (building a computer model that resembles the real world as close as we can, then letting a lot of economists or random policies loose on it and seeing what works) is precisely one of the ways we decide on monetary policy now. But the trouble is: can we know how closely a model economy resembles the real world? Will the real world respond to policy in the same way the computer model does? The sort of experiments which could tell us how well the model really works just don't get done (for good reasons). "The model tells us that if we do policy X, then unemployment will be 80%; let's implement policy X and see if the model's right".

[Winslow R.]

"Winslow: what you describe (building a computer model that resembles the real world as close as we can, then letting a lot of economists or random policies loose on it and seeing what works) is precisely one of the ways we decide on monetary policy now."

Close.

Given the reaction to the current economic crisis, I don't believe most economists have a clue regarding the basic structure of the real economy or how fiscal and monetary policy creates an impact..

I also don't believe current economic models have 'real world' participants creating transactions reacting to the random economic policies let loose on it.

Perhaps any game with extensive economist input would be sooooo boring that no one would participate. Just adding banking intermediators between a game's government and borrowers would be eye opening in regards to financial instability.

http://en.wikipedia.org/wiki/Second_Life

[Adam]

Nick, the idea behind your steering-wheel analogy is good but the problem is that you're comparing the economy to a device that only has two settings (left and right). OK, three if you count the middle (i.e. doing nothing). A better analogy might be trying to defuse a bomb, with a very complex and lengthy series of steps that need to be carried out just right in order to get the job done. You might have seen 10 people try it before (with bombs that are somewhat similar, but not perfectly identical to the one you have). They all failed. So you have a decent idea that what they did won't work. But what WILL work? Even looking at instances where the bomb was defused won't help that much because as I said, it's not quite the same device and even then, you're not 100% sure that their handiwork was what did the trick - maybe the bomb was defective, maybe the explosive was a dud, maybe the wiring was faulty, etc.

What I'm saying is that we know how to completely ruin an economy (Zimbabwe being a great example). I suppose that's valuable in that we know what NOT to do, which is a good thing to know. But as far as knowing what TO do... we don't know enough to start fiddling with things and hope it improves the situation. Of course if you're careening off a cliff and are going to die anyway, you may as well try - you couldn't possibly make things worse. But media hysteria notwithstanding, the economy is not nearly as bad as it could be, not even remotely so. But if we fiddle with enough, we could sure help it along in a hurry.

In any event, kudos to Prof. Gordon for admitting that the emperor, while not quite naked, certainly isn't wearing such luxurious finery as some would have us believe. Social science is like any tool: useful, but only if you're aware of its limitations.

[Stephen Gordon]

Actually, this is Nick's post.

[Nick Rowe]

Winslow: I only have a second-hand knowledge of the big computer macro-models, (I'm a small "let's try and get our heads around this" model-builder myself), but as far as I know:

"I also don't believe current economic models have 'real world' participants creating transactions reacting to the random economic policies let loose on it." No, that's what we do have in the models: agents react both to actual and to expected government policy.

"Just adding banking intermediators between a game's government and borrowers would be eye opening in regards to financial instability." Yes, that's the bit which is missing, or really inadequate from today's perspective, in the big macro-models.

By the way, I can't remember where I read it, on some economics blog a couple of months back, but some economists are playing around in Second Life, testing out their theories of money. I think they were from one of the US Feds.

[the last economist]

Actually, the problem is that fiscal and monetary policies affect different sectors of the economy differently, and the kind of allocation incentive being sent is never accounted for in macroeconomics. It's the whole problem of the microfoundations of macroeconomics, on one side, and of society's goals function on the other.

[Winslow R.]

" I only have a second-hand knowledge of the big computer macro-models"

Okay I admit my knowledge of macro-models is third hand though I've worked directly with other model types.

""I also don't believe current economic models have 'real world' participants creating transactions reacting to the random economic policies let loose on it."

No, that's what we do have in the models: agents react both to actual and to expected government policy."

Okay......

So before I become incredulous prematurely, are you talking about agents (aka real people) or models of agents?

No, I will forget caution and make an assumption. I find it incredible that you can say with a straight face (perhaps you are smiling?) modeled agents react to actual and to expected government policy. The reaction of these agents is based on the assumptions of the modeler as real agents have no direct input to these systems. Perhaps I'm mistaken and economists incorporate things people do like people being laid off from jobs and then incorporating the data of how these real people react into macro-models.

For the past couple of years I've considered hiring a local economic graduate student/computer programmer to work on an web based 'economic engine' that could be used as a user customizable gaming solution as well as a competitive platform for representatives from the various economic variants to publicly display/compare their macro abilities.

You are correct, it does look like there is some research going on though most doesn't look like it is economic. it also looks like they had deposit taking institutions in 2nd life.

http://secondliferesearch.blogspot.com/

"As of January 22, 2008, it will be prohibited to offer interest or any direct return on an investment (whether in L$ or other currency) from any object, such as an ATM, located in Second Life, without proof of an applicable government registration statement or financial institution charter. We’re implementing this policy after reviewing Resident complaints, banking activities, and the law, and we’re doing it to protect our Residents and the integrity of our economy."

http://blog.secondlife.com/2008/01/08/new-policy-regarding-in-world-banks/

"The recent collapse of Ginko Financial, a "virtual investment bank" in Second Life, has spurred calls for more oversight, transparency and accountability, especially when it comes to business practices in the metaverse.
Last week, Ginko Financial -- an unregulated bank that promised investors astronomical returns (in excess of 40 percent) and was run by a faceless owner whose identity is still a mystery -- announced it would no longer exist as a financial entity.
The declared insolvency meant the bank would be unable to repay approximately 200,000,000 Lindens (U.S. $750,000) to Second Life residents who had invested their money with the bank over the course of its three and a half years of existence."

http://www.wired.com/gaming/virtualworlds/news/2007/08/virtual_bank

[Nick Rowe]

the last economist: I don't think you're right on that. Even the very simplest macro model (e.g. ISLM) has monetary and fiscal policy affect C and I differently. Or perhaps I misunderstand you.

Winslow: models of agents. (Real ones are too big to fit inside the computer. Sorry.) And yes, the modeled agents react as the modeler assumes they do. Some modeled agents get laid off from jobs in recessions. In simpler models, agents just work fewer hours. Sounds like Second Life had its own Ponzi schemes! But unless my memory is wrong, it was also economists doing some sort of "participant"-research in Second Life.

[Winslow R.]

" Sounds like Second Life had its own Ponzi schemes! "

Right and no one had to program the agents to make it happen! Second Life already sounds like a better macro-model than the large Fed macro-models even though the didn't have a CB regulator or even a Fed chairman that I know of.

Kind of sad to think of all those high paid economists creating worthless macro-models that could instead be creating a really cool game (well at least from an economic perspective).

[Richard H. Serlin]

Nick,

With regard to the 3rd paragraph, I wrote it fast (short on time, but in a conversational forum it can be worthwhile to say something fast rather than to risk never having time to get to it at all). Basically, my experience with academics is that sometimes they will support conclusions based on formal models or formal statistical work based on strongly unrealistic assumptions, but will disregard less formal counter arguments that have completely solid logic chains anchored to more realistic assumptions.

Here's a little allegory for your entertainment:

A snobby empirical economist is hiking with a friend who eats a berry off a bush and immediately keels over and dies. The economist thinks to himself, well it's just a sample of one berry; you can't draw any conclusions from a sample of one. And being hungry himself, he eats a handful.

Moral of the story: It's very inefficient, and perhaps very dangerous, to ignore abundant a priori information at your disposal, even if that information is not formal, but still logical and based on relatively realistic assumptions, or you don't currently have a formal version of it.