Burn the mathematics

Paul Krugman thinks that economic discourse on the blogosphere is a good thing. He's right when it comes to economic theory. 

Economic theory is deductive: assumptions + fancy stuff = conclusions. Ultimately, the path from assumptions and the conclusions doesn't matter, as long as it's logically consistent. As Alfred Marshall said, translate mathematical results into English, illustrate by examples that are important in real life, then "burn the mathematics."

Yet it's easier for editors of academic journals, and for referees, to judge the fancy stuff than the plausibility or attractiveness of a model's underlying assumptions. Mathematics is either wrong or right; technically sophisticated or elementary. Plausibility is more subjective: the assumptions I think are attractive ones to make about the shape of the probability density function might be unappealing to you, and vice versa (others have made this point before, but I can't remember who or where).

The strength of the econblogosphere is that it allows frank, open and honest debate about plausibility - if a model's assumptions can't be justified in plain English, no one will buy the model. Similarly, if the conclusions are unconvincing, and contrary to basic intuition, the blogosphere will say so. At academic conferences, and in the pages of journals, we pay the emperor all due respect - the emperor is powerful, and might reject our next paper if we say too much. In the blogosphere, there is less gatekeeping, so the little boy can say the emperor has no clothes. The emperor might stop linking to the little boy's blog posts, but if the little boy speaks the truth, people might eventually listen.

The blogosphere is less effective, I think, when it comes to evaluating empirical research. The formula for empirical research is data + methods = conclusions. Blogs, and bloggers, generally don't go deeply into data and methods. Summary statistics and regression results are hard to read in html. A thorough review by a single, consciencious and knowledgeable referee can usually do better of appraising a paper's research methods - spotting the lack of robust standard errors, for example - than a thousand superficial blog comments. (Although the blogosphere plays a valuable role in keeping authors honest).

Plus your typical blogger is a bit of an academic slut (we're not whores, for the most part, because we don't get paid). We love to get hits and page views, and so write what people want to read: papers which show that conventional wisdom is wrong (served with a dose of academic gossip, and perhaps some sex on the side). Yet this leads to novelty bias, publication of the one result that overturns received wisdom, instead of the ten that confirm it.

(I don't know if I would go so far as Stephen Williamson, and conclude that most blogging on economics is journalism.  Bloggers are the movie and restaurant reviewers, life style and op-ed columnists. If a story involves making phone calls, it's best left to your honest working hack.)

Comments

[Sandwichman]

"Plus your typical blogger is a bit of an academic slut..."

Don't kid yourself, Frances. Most of us are sluts. A few of us are whores: e.g., "free market think tank" bloggers. Whores.

[dw]

Great post.

[Frances Woolley]

dw - thanks.
Sandwichman, so true.

[Gordon Barker]

In your article you state

"Economic theory is deductive: assumptions + fancy stuff = conclusions. Ultimately, the path from assumptions and the conclusions doesn't matter, as long as it's logically consistent"

This is not true. If you have never heard of it I would suggest reading about "Phlogiston theory". It was a theory that held sway (in chemistry) literally for centuries, was logically consistant from all points of view and equally, totally bullshit. Logical consistancy is not a proof of validity and in my observations, current economic theory has never matured beyond being "Phlogiston".

[Patrick S. O'Donnell]

Although I don't share her larger economic views (I'm more or less Marxist in economics, she's not), I think Deirdre (formerly Donald) McCloskey has well-covered the problem with a "scientistic" sort of mathematics in neo-classical economics (e.g. qualitative theorems) generally as part of her greater examination of the "rhetoric" of the discipline in several of her books, but in particular Knowledge and Persuasion in Economics (1994) and the little pamphlet, The Secret Sins of Economics (2002). One sin, which I would describe as betwixt and between the "venial" and the "mortal," is owing to an inability to appreciate or remember the fact that a "'statistically insignificant' number can be very significant for some human purpose," and a "'statistically significant' result can be insignificant for any human purpose."

Bloggers with expertise can forthrightly expose and discuss what she defines as the "numerous weighty sins requiring special grace to forgive but sins not peculiar to economics," namely, an appalling degree of institutional and historical ignorance, cultural barbarism, high-school versions of positivism and ethics, and professional if not personal arrogance that lends itself to a belief in non-democratic forms of incrementalist social engineering. Her website is here: http://www.deirdremccloskey.org/index.php

Other, "sins" (assumptions animating neo-classical market liberalism) are addressed by John Quiggin (who blogs at Crooked Timber) in his book, Zombie Economics: How Dead Ideas Still Walk Among Us (2010).

[DavidN]

Gordon Barker: ‘This is not true. If you have never heard of it I would suggest reading about "Phlogiston theory". It was a theory that held sway (in chemistry) literally for centuries, was logically consistant from all points of view and equally, totally bullshit. Logical consistancy is not a proof of validity and in my observations, current economic theory has never matured beyond being "Phlogiston”.’

I don’t understand how this critique couldn't be applied to any science, and not just economics. If economics is ‘Phlogiston’ then so is every other discipline.

[W. Peden]

It's worth noting that phlogiston's early traction wasn't due to its logical consistency (logical consistency is actually a very low standard) but the fact that it was based on plausible and well-established assumptions in chemistry. If you know that burning an object reduces its weight and you work from the plausible assumption that this means that some substance must be removed, then it is very reasonable to conclude that there is a common substance that is removed in all cases of burning given how burnt objects have similarities.

Phlogiston theory's predictive power came from the fact that one could use it to correctly predict the similarities between rusting and burning, as well as the fact that burning produces a gas.

Weeding out (interesting) incorrect theories involves a lot of subtle experimental ingenuity and meticulous clarification of theories/hypotheses. This is why mathematisation is generally very good, even if one virtue doesn't make a virtuous paper: without measurement and clarity, a theory like phlogiston can be quite unconquerable.

[W. Peden]

Also-

"Logical consistancy is not a proof of validity"

- depends what you mean by 'validity'. Sometimes people use that term to mean consistency; othertimes they use it to mean 'soundness' or even just 'truth'.

[UnlearningEcon]

'Ultimately, the path from assumptions and the conclusions doesn't matter, as long as it's logically consistent.'

It's interesting you say this because what's perhaps most lacking in economics teaching is the path from the assumptions to the conclusions. Teaching a model generally goes like this:

- Assumptions

- ????

- Diagram/Maths

I honestly have very little idea what the impact of most assumptions in economics is on the analysis.

[Frances Woolley]

Gordon Barker:

Phlogiston theory illustrates my point exactly.

The problem with the theory was that it started with incorrect assumptions, namely that there existed a fire-like element called "phlogiston".

Once you start with wildly incorrect assumptions, no amount of beautiful and elegant logic is going to get you anywhere.

W. Peden - true, one shouldn't underestimate the difficulties of falsification - and here the point that I made about empirical research is relevant. Someone has to go through the experiment with a fine tooth comb and say: did you control for this?

Patrick - good suggestions for further reading. Deirdre McCloskey must have made a similar point about judging theoretical work by technique not content somewhere, but I can't recall where. Perhaps I'll find it later. I do think that the profession is actually getting better at distinguishing between economic and statistical significance - e.g. you can't publish a paper that uses logit or probit and can't report marginal effects (i.e. meaningful estimates of economic impact) as opposed to the hard-to-interpret estimated coefficients.

[Frances Woolley]

UnlearningEcon: "what's perhaps most lacking in economics teaching is the path from the assumptions to the conclusions."

That's a pretty damning indictment of the state of economics teaching. There's some truth in it - for example, sometimes you'll see models that make assumptions that are totally inconsistent, something like. assuming decreasing returns to scale and monopoly (only less obvious). It's because people don't really understand what their assumptions are buying them.

Part of the problem, I think, is the degree of abstraction. This post was inspired, in part, by some of Nick Rowe's recent posts, e.g. the one where he asks, "O.k., we have this concept called savings, what does it actually mean?". It's something that's never talked about.

[Eric Pedersen]

I actually think there's a similar argument to be made about empirical results: you can use all the fancy stats you want (and I say this as a huge fan of fancy stats!) but the assumptions you're making about the data-generating process should be statable (and actually stated) in plain language, and anyone should be able to reasonably evaluate your model for themselves, as long as you plot the model output together with the raw data. Andrew Gelman makes the point that graphics with both model and data are an essential part of statistical analysis many times, but a good intro is here. Proper plotting also, I think, deals with concerns like 'statistical vs. economic significance'. Generally, from a well done plot, the reader can figure out how significant a given effect size is for themselves.
In many ways, I'd say tables of coefficients and complex statistical terms are analogous to complex mathematical theorems; you need to have them, if only for conciseness and replicability, but it's often easy to spot where an analysis went wrong as long as the author is honest about plotting relevant data.

[Frances Woolley]

Eric - I agree 100%. Data issues, e.g. treatment of missing observations, often matter far more than lots of things people get all bent out of shape about, e.g. use of probit v. linear probability models, or multicollinearity.

[john]

Great post,

did you guys see this:
http://gregmankiw.blogspot.com/2012/01/home-for-holidays.html

Some of it is similar to your point Frances. It is also very funny.

[DrJim]

I think it's important to recognize that we should only "burn the mathematics" after we actually do it. Otherwise we just say things are true because we want them to be true.

[Frances Woolley]

Dr Jim - agreed. That was a deliberately provocative title - my point is about the value of having conversations about assumptions and the conceptual underpinnings of models, but math is necessary to work out the implications of those assumptions.

[Dave Giles]

Frances - nice post. Of course, it's quite possible to be concerned about the mathematical/econometrics niceties, but still appreciate and promote the importance of data quality and relevance.

For instance see my blog post from last year at http://davegiles.blogspot.com/2011/03/curious-regressions.html

[Frances Woolley]

Dave: Great slogan: "Plot it or Perish".

[Determinant]

Eh? What happened Frances? Normally we try to be polite around here and avoid the heated rhetoric of the macro threads.

BTW I like to think that I have made a point or two around here and I am not an economist, I just play one in blog comment threads. I like to think of myself as part of the Maple Leaf Peanut Gallery.

But this bears repeating:

Bloggers with expertise can forthrightly expose and discuss what she defines as the "numerous weighty sins requiring special grace to forgive but sins not peculiar to economics," namely, an appalling degree of institutional and historical ignorance, cultural barbarism, high-school versions of positivism and ethics, and professional if not personal arrogance that lends itself to a belief in non-democratic forms of incrementalist social engineering. Her website is here: http://www.deirdremccloskey.org/index.php

I hang around here specifically to tear threads containing those issues to shreds. ;)

[Greg Ransom]

Markets and the overall economy don't equalibrate or move toward consistency without learning.

Learning is open-ended, causal, and contingent.

In oher words the cental causal component in economic science is NOT deductive, and hence economic theory as a contingent, causal, explanatory science is NOT deductive.

QED

The assumption otherwise is pseudo-science.

[W. Peden]

Greg Ransom,

I'm not quite sure how you're using "deductive". If you mean something like necessary i.e. the opposite of contingent, then I agree.

If you mean deduction in the sense of inferring no more from an argument than what is in the conclusions (non-ampliative inference, in logical jargon) then I think that is how most economics proceeds (including great stuff like Hayek's work). Even then, of course, as Hayek pointed out there is an ancillary inductive element in that we need premises based on our knowledge of other minds, which in turn comes from introspection and contingent inferences about the minds' of others.

"The Counter-Revolution in Science" certainly cured me about some of my more positivistic ideas about economics. It's one of my favourite books by Hayek.

[Greg Ransom]

Peden, I'm a philosopher. I was trained to use "deductive" to mean a logical relation.

Contingency, esp. explanatory contingency, requires merely alternative causal possibilities. Darwin's mechanism explaining design-like biological order is contingent because we can imagine other possible causal mechanisms, e.g. God, magic, improbable but possible luck, Martians did it, etc.

Hayek's mechanism explain design-like global economic coordination is contingent because we can imagine other possible causal mechanism, e.g. God, magic, improbable but possible luck, Martians did it,

The point Kuhn and Wittgenstein make, is that the more firmly we adopt and begin to live within an explanatory nexus, the more conceptually bound our concepts become -- contingency begins to fall way from the relations of our concepts, and they begin to come for us in a tightly bound package of inter-defined terms that we can hardly imagine not having the seemingly "deductive" conceptual relations we take them to have.

[Greg Ransom]

This point also applies to our perceptions (as Kuhn and Wittgenstein point out). What began as an inference -- e.g. that house with trash out front and a broke window must be unoccupied -- become a direct perception. In time, even before your attention has really focused on it, you directly perceive the house _as_ an unoccupied house.

"The point Kuhn and Wittgenstein make, is that the more firmly we adopt and begin to live within an explanatory nexus, the more conceptually bound our concepts become -- contingency begins to fall way from the relations of our concepts, and they begin to come for us in a tightly bound package of inter-defined terms that we can hardly imagine not having the seemingly "deductive" conceptual relations we take them to have."

[Frances Woolley]

Greg: "we can hardly imagine not having the seemingly "deductive" conceptual relations we take them to have"

The question, then, is whether the form of discourse (blogging v. journal articles v. talking to our own kids) makes any difference to our willingness to consider alternative views of the world?

[W. Peden]

Greg,

I'm with you except for defining contingency in terms of our imagination. J. S. Mill got a lot of things wrong, but he was right to emphasise that psychological necessity and logical necessity are different i.e. the fact that we cannot imagine something doesn't mean that it's logically necessary that it doesn't exist. That doesn't work the other way around, though, i.e. we can't imagine that "P and ¬P" because it's logically necessary that "¬(P and ¬P)".

Frances Woolley,

"The question, then, is whether the form of discourse (blogging v. journal articles v. talking to our own kids) makes any difference to our willingness to consider alternative views of the world?"

I think it depends on the degree to which one views one's interlocutors as equals. People tend not to take their kids' theories of the world too seriously and tend to take things in journal articles seriously. Where blogging falls on that spectrum will depend very much on the bloggers and the commentators, I think.

[Frances Woolley]

W. Peden: "People tend not to take their kids' theories of the world too seriously and tend to take things in journal articles seriously."

I disagree. I know people who have been completely unconvinced of the existence of say, gender discrimination - and who could not be swayed from this view by 1000 articles in the Journal of Heroic Regressions - but whose world view suddenly underwent a 180 degree change when their child experienced challenges in the labour market.

[W. Peden]

I hadn't thought of that. Then again, I don't have kids!

[Wonks Anonymous]

"a "'statistically insignificant' number can be very significant for some human purpose,""
I thought the point of statistical significance was to tell whether something really exists or is just a fluke result of a small sample size. If it doesn't really exist (the null hypothesis), why would it be significant for any human purpose?

[Jacques René Giguère]

There is a continuity in the value of newspapers, scientific journals and blogs. There is a continuity in their audiences and their commenters. And the seriousness of a blog does't necessarily imply the same about comments. Krugman sometimes publishes some very wonky post which are commented by trolls. WCI sometimes encounter brief tempers tantrums but not trolling.
There are very serious blogs about econometrics for example. they are the equivalent of discussion periods andcoffe breaks ar learned conferences.
There are others which are coffe breaks at works.
Some are pure trash, even in "respectable" publication.

Sluts? Frances, Frances, Frances. I had always thought of you as the strong-but-loving mother-of-the-blog and am now discovering your dark freudian secrets. Oh gosh, my mom makes the crosswords in Allô-Police...

[Determinant]

I disagree. I know people who have been completely unconvinced of the existence of say, gender discrimination - and who could not be swayed from this view by 1000 articles in the Journal of Heroic Regressions - but whose world view suddenly underwent a 180 degree change when their child experienced challenges in the labour market.

I have met a lot of people like that. I'm one of them too, after all my labour market travails of the past few years.

On a more theoretical note, Barrie McKenna posted an excellent piece in Economy Lab yesterday about how economics, both micro and macro, gets the theoretical roles and expected behaviour of both consumers and firms wrong and how this leads to misdiagnoses of economic behaviour and malaise.

I really agree with him because I have seen first hand, repeatedly, the kinds of behaviour he describes. For me the single most important question in economics right now is "what went wrong with firm's capitalization and value extraction activities?" In order to answer that you need to challenge or throw out a great many comfortable assumptions about the world and how it behaves.

[kio]

excellent points. The blogosphere is an imaginary part of a complex number. Looks unnecessary before one has to resolve problems in the complex number domain (aka real life). It provides more expertise than just testing assumptions - the variety of bloggers is almost full and includes people testing math and plots as well, where it's possible.
Obviously, feed back is absent and practically no criticism from bloggers gets into academic world. And it is not because the criticism is not right ...
Amazingly, if to take a review paper from macroeconomics (say, a paper by Whelan and Rudd on inflation http://www.federalreserve.gov/pubs/feds/2005/200566/200566pap.pdf) where all inflation models are proved to be wrong, it is hard to understand why acedemics do not listen the blogosphere, which tells the same story.

[hosertohoosier]

In political science, we generally talk about four kinds of methods, which each serve distinct purposes.
1. Formal models allow us to construct logically consistent arguments, and produce counter-intuitive hypotheses
2. Statistics allow us to test whether or not those hypotheses generalize
3. Case studies enable us to test causal mechanisms behind our theories (and to generalize, though less well than stats)
4. I guess there are also experiments, which can serve the same purpose as 2, 3, or 4 (but may not generalize outside the lab)

I note that there doesn't seem to be a lot of #3 in academic economics. You guys are good at coming up with models, and pretty good at testing them statistically. However, you don't often look at whether things are happening the way they are for the reasons you intimated (instead, if the results concur with the stats, you guys are happy to declare the model confirmed).

So I wonder if folks like Paul Krugman are responding to the underproduction of case studies by the discipline. It would seem to me that these kinds of discussions could be done more effectively "inside the tent" as it were, so maybe you should clone a bunch of Barry Eichengreens, and stop starving economic history.

[Frances Woolley]

kio: "practically no criticism from bloggers gets into academic world"

Sometimes things take time - I'm figuring that one day some smart PhD student is going to go through Nick's blog posts, write them up as nice formal fancy models, and get them into all sorts of high-powered journals. So there is an indirect influence on current researchers, and on the next generation. Partly this is coming from people knowing the blogs are out there, partly this is coming from the google effect - it's quite possible to type some random economics topic into google and get WCI on the first page of hits. My concern is not so much that there is no influence, but that the influence isn't recognized or acknowledged.

hosertohoosier - It is true that economists aren't great at case studies, but there is much much more concerns than there used to be about establishing causality through statistical methods. So people look at situations where there's a policy change in one state (province, country) but no policy change in a neighbouring, similar, state (province, country), and use "difference in difference" techniques to try to establish causal impacts of policy changes.

But, yes, economists' extreme reluctance to put any stock in what people say - economists not only think actions speak louder than words, they frequently figure words don't speak at all - means we rarely go and ask people "well, what happened here and why."

Also if we wanted to do case studies, we would have moved next door to the business school and gotten ourselves a substantial salary increase.

There may also be some gatekeeping going on - if we admitted case studies to the respectable economics journals, we'd see an invasion of sociologists and political science - look at what's happening to the top political science journals these days, with all sorts of cross-dressing economists stepping in!