This is a story about how something that turns out to be wrong can be published in a top journal, and what happens next.
The Freakonomics team tell good stories, so I'll let them begin. In 2005, they wrote an article in Slate magazine lavishly praising the work of a young economist called Emily Oster.
While the ratio of men to women in the West was nearly even, in countries like China, India, and Pakistan, there were far more men than women. Sen charged these cultures with gravely mistreating their young girls—perhaps by starving their daughters at the expense of their sons or not taking the girls to doctors when they should have. Although Sen didn't say so, there were other sinister possibilities. Were the missing women a result of selective abortions? Female infanticide?
But now another economist has reached a startlingly different conclusion. Emily Oster is an economics graduate student at Harvard who discovered a strange fact. A pregnant woman with hepatitis B is far more likely to have a baby boy than a baby girl. Oster was suitably intrigued. She set out on a vast data mission to determine the magnitude of that relationship.
She found that:
The regions with the most hepatitis B were the regions with the most "missing" women. Except the women weren't really missing at all, for they had never been born. If you believe Oster's numbers—and as they are presented in a soon-to-be-published paper, they are extremely compelling— Hepatitis B can account for roughly 75 percent of the missing women in China. Oster's analysis does show that economics is particularly useful for challenging a received wisdom—in this case, one that was originally put forth by another economist.
Oster's work resonated with Dubner and Levitt because it supported two of economists' most cherished beliefs:
1. Individuals making rational choices will usually produce good outcomes. This is why economists generally believe in the efficiency of markets and a limited role for government.
If individuals making rational choices decide to, say, abort female fetuses, it's hard to believe that freedom of choice leads to good outcomes. If, instead, skewed sex ratios are caused by Hepatitis B, our faith in individual choice is unscathed.
2. Economists are smart. Smarter than other people who are easily swayed by incendiary arguments. Levitt and Dubner put it a little more tactfully in Freakonomics when they say "the conventional wisdom is often wrong."
There's just one problem with this whole heart-warming story. Oster's analysis, as published in the Journal of Political Economy (editor: Steven Levitt) [ungated], was wrong. Studies of thousands of people with and without Hepatitis B infections find that the virus causes only a very small change in the probability of having a boy. Lin and Luoh published a rebuttal in the American Economic Review. Earlier this year, Oster, Chen, Yu and Lin published a paper acknowledging Hepatitis B does not explain male-biased sex ratios in China.
Emily Oster, after all of this, remains a respected economist. Her creativity and intelligence is evidenced by her other work, for example, her recent paper arguing that cable television has improved women's status in India (now that's a cool paper). Creative people tend to come up with lots of ideas - not every one works out.
But Levitt's reaction to the whole affair was somewhat lacking in introspection. He praised Oster: "She also has done something incredibly rare for an academic economist: she has admitted she was wrong."
Yet I have not read any admission from Levitt that he, as editor of the Journal of Political Economy, made a mistake when he accepted her work for publication.
Someone arguing in Levitt's defence might say "well, no one could have known that Oster's hypothesis would turn out to be wrong." Could they? In 2005, the year that Oster's paper appeared in the JPE, Monica Das Gupta published a rebuttal in the Population and Development Review. She describes the results of a 1993 paper by Zeng et al, one cited by Oster:
...the sex ratio at birth varies sharply by the sex composition of the living children the woman already has.... Zeng et al. show that the sex ratio at birth was normal (1.056) for first births. For second births, it was strikingly different depending on whether the first child was male or female: women whose first child was a son had a low sex ratio (1.014) for the second child, while those whose first child was a daughter had a very high sex ratio (1.494) for the second child.
To produce a pattern like that, Hep B has to be one heck of a smart virus. So the first point is: anyone with even a passing familiarity with the literature would know there was something suspicious about the Oster results.
Second, Oster's evidence was, by generally accepted research standards, weak. Basically the paper took a bunch of different countries, and looked at the relationship between the ratio of boys to girls in the population and the prevalence of Hepatitis B. It just happens to be the case that countries with high rates of Hep B - China, for example - also have skewed gender ratios. If you had replicated Oster's methodology and substituted "percentage of calories coming from rice" for "Hep B prevalence" you might have found that rice consumption causes missing women. Cross-country studies are particularly subject to this kind of problem, so many researchers (rightly) view them with suspicion.
Sometimes mistakes don't matter. Earlier this year I speculated that income inequality might by correlated with adultery. I was probably wrong, but I doubt anyone signed up for AshleyMadison.com after reading the blog.
But Oster's mistake had - and still has - the potential to do real harm. Type "missing women" into http://scholar.google.com. When I did that just now, the number four hit was to Oster's 2005 JPE article attributing 75 percent of China's gender imbalance to Hepatitis B.
I don't believe some of the fear mongering about "Bare Branches and Danger in Asia" - the idea that nations faced with "surplus men", that is, men unable to find wives, inevitably resort to war and imperialism. But the current demographic numbers in China, combined with the fact that some younger women would prefer to date a successful older man than a struggling young one, mean that a substantial proportion of young men will be unable to find partners.
And that will lead to profound social changes. Some studies have suggested that competition for scarce brides causes parents of sons to increase savings. Others suggest surplus men increase the demand for sex workers, leading to more prostitution and an increase in HIV infection rates. I've spoken to colleagues from India and asked "what does this gender imbalance mean for your daughter?" and heard stories about rape and sexual assault.
It's hard to predict what will happen - the gender imbalances currently emerging in parts of Asia have few historical parallels.
But I can say one thing for sure: providing an easy scapegoat in the form of Hepatitis B infection makes it easier for people to avoid taking a long hard look at the cultural and social factors that cause parents to abort, abandon and neglect girls. And that is a serious harm.
Professor Levitt, if you are reading this: I would like to see an Errata added to Oster's 2005 JPE piece - on the front page, where no one can miss it - saying "the research in this article has since been advanced in subsequent work, published in Economics Letters."