The Canadian government decides to run an experiment to see if fiscal policy works. It throws 100 darts at a map of Canada. One dart lands on Wawa Ontario, so it spends an extra $1 million in Wawa. Local GDP in Wawa, and local GDP in all the other 99 places where the darts land, increases by $2 million relative to the control group of the Rest of Canada. The experiment clearly shows there's a government expenditure multiplier of 2.0.
Does that mean there's a government expenditure multiplier of 2.0 in the whole of Canada?
You should be able to see the underlying fallacy of composition. "Government spending has a multiplier of 2.0 in each part of Canada, therefore government spending has a multiplier of 2.0 in the whole of Canada."
- Taxes. Wawa will pay only a tiny fraction of the present and future taxes used to finance the extra $1 million expenditure in Wawa, but the higher taxes might reduce GDP in the Rest of Canada.
- Monetary Offset. Wawa does not have its own central bank, money, and exchange rate. But Canada does. Increasing government spending in Canada as a whole might lead to an appreciation of the exchange rate and a fall in net exports.
- The supply side. Resources like labour can more easily move within Canada in search of better jobs than resources can move to Canada from outside.
And I can immediately think of one reason why the whole might be more than the sum of the parts:
- Propensity to import. People living in Wawa probably spend a much smaller fraction of their income on goods produced in Wawa (unless they are addicted to the excellent Wawa summer sausage) than people living in Canada spend on goods produced in Canada. In a simple Old Keyensian model that would mean the Wawa multiplier is smaller than the Canadian multiplier.
And I can immediately think that the whole purpose of running an experiment would be to test whether there might be something that makes fiscal policy for the whole different from the sum of the parts that I can't immediately think of. Experiments are supposed to tell us whether there might be unknown unknowns. (Does anybody remember all the clever people laughing at Donald Rumsfeld when he first said "unknown unknowns"?)
Yesterday I read my colleague Vivek Dehejia on randomised control trials in economics, worrying about external validity of experiments. Today I read (H/T Mark Thoma) Price Fishback's survey paper on the microeconomic effects of Roosevelt's New Deal (as micro it looks legit to me, but we have to be very careful before drawing any macro conclusions from these micro results).
I don't have any good easy answers either. But this is why we need theory. Experiments alone aren't enough. If you do a drug trial in Wawa, the external validity of the experiment will depend on whether it's a communicable disease.
Update: Kevin Milligan tweets that I am saying that local macroeconomic multiplier experiments almost surely violate the Stable Unit Treatment Value Assumption. I hadn't heard of SUTVA before, but after reading the Wikipedia, that sounds right to me.
Update2: see Ryan Murphy's working paper on the sheer number of papers that ignore the basics like monetary offset by the central bank.