A "simple rule" is a formula that tells a central bank how to set the nominal interest rate as a fixed function of a small number of variables. The Taylor Rule, which sets the nominal rate as a function of the gap between actual and target inflation, and the gap between actual and potential output, is the classic example of a simple rule. Are simple rules better than discretion? Here's how we can do a test to find out, and the results of an old test where simple rules failed to do better than actual policy.
We need to distinguish between "instrument rules" and "target rules". The Bank of Canada, for example, sets a nominal interest rate instrument to target 2% inflation. The Bank of Canada follows a very simple target rule: "set (future) inflation at 2%". But it does not follow any instrument rule like the Taylor Rule. Instead it uses its discretion. It looks at everything it thinks might be relevant, and adjusts the nominal interest rate instrument accordingly to ensure that, in its own judgement, future inflation will converge to the 2% inflation target. "Simple rules" (in this context) means "instrument rules". If the Bank of Canada kept the 2% inflation target, but used a Taylor Rule instead of discretion to try to hit that same 2% inflation target, it would be following a simple rule.
Simon Wren-Lewis and Tony Yates have both recently posted about simple rules. They make good and sensible critiques. Tony tells us that John Taylor tells us that there is a bill before the US Congress that would require the Fed to follow a simple rule. So simple rules are topical.
Since simple rules are topical, I have an excuse to plug an old bit of empirical research I did with David Tulk. Plus, since that research is now a decade old, and could easily be updated (and could easily be replicated for other countries aside from Canada), I am hoping to entice someone to update it and replicate it. (I am too incompetent to do it by myself, and David has moved on to bigger and better things).
Our test was very simple. Take a simple rule off the shelf. The Taylor Rule for example. Then plug in the historical actual numbers for the inflation gap and the output gap into the formula (strictly, you need to have real-time estimates of the output gap to do this properly) to calculate the nominal interest rate specified by that simple rule. Then calculate the "interest rate gap" as the difference between the nominal interest rate specified by the simple rule and the actual interest rate set by the Bank of Canada. Then look at the correlation between that "interest rate gap" and the subsequent [update: we used a 2-year lag, because the Bank had a roughly 2-year targeting horizon] "inflation gap" (the gap between actual and target inflation). The sign of that correlation is telling you something important.
If you find a positive (and statistically significant) correlation between the interest rate gap and the future inflation gap, that is telling you the Bank of Canada should have put more weight on the "advice" given by the simple rule and less weight on its own judgement. (But it does not tell you it should have put 100% of the weight on the advice given by the simple rule and 0% weight on its own judgement.)
The intuition is also simple. If in one period the interest rate gap is positive, that means the simple rule is saying that the Bank of Canada was setting the nominal interest rate too low to keep future inflation at the 2% target, and so the simple rule is saying that future inflation will rise above the 2% target. And if in another period the interest rate gap is negative, that means the simple rule is saying that the Bank of Canada was setting the nominal interest rate too high to keep future inflation at the 2% target, and so the simple rule is saying that future inflation will fall below the 2% target. By looking at the sign of the correlation between the interest rate gap and the future inflation gap, we can test those implicit forecasts made by the simple rule.
To cut a long story short (read the paper for the details): simple rules did not perform well in our test.
The most plausible explanation is this: it is not that simple rules give bad advice; rather, the advice they give was already incorporated into the Bank of Canada's decisions, to which the Bank of Canada then added its own judgement from looking at other variables. The policy implication is not that the Bank of Canada should completely ignore the advice given by simple rules; the policy implication is that the Bank of Canada should put no more weight on that advice than it actually did. Simple rules leave stuff out, and what they leave out matters too.
This fits with what Simon and Tony are saying.
But maybe our test wasn't powerful enough, because we only had 10 years of data, because the Bank of Canada had only been targeting inflation for 10 years. 10 years later, with 20 years of data, the results might be different. Or might not. More research is needed (sorry).
(To keep it simple, I have written this post as if I were an orthodox New Keynesian/Neo-Wicksellian, which is what I was 10 years ago.)
[Update: please read Gregor Bush in comments below. Gregor has updated Rowe/Tulk for Canada. His main finding: "This probably isn't shocking but using overall inflation in the test equation instead of core results in a coefficent that is more negative and more significant on the interest rate gap. Taylor's rule, for example, wanted the overnight rate to be about 300bps higher than it actually was from late 2010 to early 2012 and of course inflation subsequently dropped well below target and stayed there for an extended period." What that result means is that the Bank of Canada should have put less weight on the Taylor Rule's advice than it actually did.]