Dan Gardner of the Ottawa Citizen is one of my very favourite journalists. I first came across his work during the 2008 election campaign, in which he distinguished himself by writing what turned out to be the only article on carbon taxes that made no reference to Stéphane Dion's accent. His piece was a beacon of intellectual curiosity in what was otherwise a very bleak media landscape, for which I was almost pathetically grateful.
Happily, he has also found time to write books. His first book, Risk, dealt with the psychology of how we handle uncertainty, and is well worth tracking down, buying and reading. His most recent book, Future Babble, notes that one of the ways we deal with uncertainty is to rely on the predictions of experts. Unfortunately, experts aren't very good at forecasting, but no-one seems to notice or care. Many of the examples Dan uses are, inevitably, from economics, so this is a good book for economists to read and to take seriously.
I think it's worth elaborating on this point a bit. If experts can't produce reliable forecasts, then how can they be experts? The answer is that their expertise is in their ability to make conditional forecasts about a given phenomenon, holding everything else constant - the famous 'ceteris paribus' qualifier. And when we are able to control for these factors, economists' predictions aren't all that bad: when Hurricane Katrina knocked out 10% of North America's refinery capacity, the resulting increase in gasoline prices were pretty much in line with available estimates for gasoline demand elasticities. What economists can't do is predict the arrival of hurricanes that knock out 10% of refinery capacity.
The problem facing economic forecasters is that they are invariably asked to produce unconditional forecasts. This would be a daunting task, even if they could make conditional forecasts with a fair amount of precision. We might be able to call one or two coin tosses. But as the forecast horizon gets longer, there are more coin tosses to call, and accuracy deteriorates. Dan makes the point that meteorologists can make reliable forecasts for the next 24 or 48 hours, but not beyond horizons of 10 or 14 days.
These inherent uncertainties are an integral part of a proper forecast, and it's hard to fault an analyst who goes to great lengths to document the ways and likelihoods in which a certain projection could go awry.
The real targets of Future Babble are those who eschew the ways of responsible forecasting and who make confident, unconditional predictions based on not much more than their brand of Personal Credibility. There are lots of them, and the most delightful parts of the book are those where they get smacked down. I'll resist the temptation to go over them; it would be like giving away the best jokes of a comedy. And some of them are pretty funny.
Even though many of the examples are from economics, the real culprits are what Declan once called 'newspaper economists': the sort of people who issue press releases and who are quoted in the media for really no better reason than because they issue press releases.
In a recent column, Dan drew the distinction between how Bank of Canada Governor Mark Carney talked about the Bank's forecast and how a Master of the Wall Street Universe talked about his. Carney's was nuanced, guarded, and conditional. The Wall Street Master of the Universe was emphatic and unconditional, and made for much better TV.
And that's the real lesson of Future Babble. What people want from forecasters is certainty. Conscientious forecasters cannot provide it, because the future is inherently uncertain. But as long as suckers people are willing to pay for a forecast that promises certainty, there will be snake oil salesmen people willing to satisfy that demand.