« Does Canada's Electoral System Under-Represent Minorities? | Main | A Lament for Public Policy »


Feed You can follow this conversation by subscribing to the comment feed for this post.

Ecologists went through similar debates once upon a time, looking for a single best "diversity index". The starting point is usually data on how many species there are at a site, and how abundant each of them is. How can we summarize the "diversity" represented by those data? We could just count the number of species, but that neglects the fact that a site with one common species and lots of really rare ones doesn't seem very diverse--it's not really all that different from a site with just one species. Which led to lots of arguments about how best to collapse the entire species-abundance distribution into a single number that would summarize its "diversity". Which of course is impossible because you can't capture every feature of most frequency distributions with a single number, so any single-number summary ends up throwing away some information and behaving weirdly in some applications. As in economics, the most popular indices retain their popularity because they have straightforward interpretations, and because they are traditional.

Jeremy - interesting. Once again, ecology and economy have so much in common.

- what are the popular indices in biology?
- I think there's a difference between *seeming* straightforward and *being* straightforward. E.g. the Gini. It's more sensitive to changes in some parts of the distribution than others, but I always have a really hard time remembering which parts, or working it out from the picture. Put another way, a straightforward graphic interpretation, i.e. the difference between the line of equality and the Lorenz curve, doesn't imply a straightforward real world interpretation.

Should point out that I used the Gini because it was the only statistic readily available for the purposes of my post. Which of course begs Frances' question.

Stephen - supply creates its own demand?

Yep. Everyone looks at the Gini because everyone looks at the Gini.

hi Frances,

In biology the fallback ones are, shannon, simpson.
> wiki, in depth

Correct use of 'begging the question' on a blog?! What next? Superluminal neutrinos?

edeast - "Related to diversity indices are many income inequality indices, such as the Gini index and the Theil index. Generally these measure a lack of diversity, but the only difference with the measures mentioned above is a minus sign.
The Theil index in particular is the maximum possible diversity log(N) minus Shannon's diversity index. It is the maximum possible entropy of the data minus the observed entropy. The Theil index is called redundancy in information theory."

So the Theil index of inequality is essentially the same as the Shannon index of diversity - so presumably the Shannon has those nice decomposibility properties of the Theil?

Yes think so, except for being the residuals of each other.
The theil write up explains what the equivalent measure is in biology.

What edeast said--the two most popular diversity indices in ecology are Shannon "information" (called that because the index was originally developed in information theory), and the Simpson index (the greater the diversity, the greater the probability that two randomly-sampled individuals will be different species).

In ecology, issues of the decomposability of different diversity indices mostly come up not in the context of looking at the diversity of species at a single site (although they do come up there). Instead, they mostly come up when trying to understand the total diversity across a collection of sites. One component of that total is within-site diversity, another component is the among-site diversity (do different sites have the same or different species?) Debate centers on how precisely to measure within- and among-site diversity, so that total diversity decomposes into either the sum or the product of within and among-site diversity (much debate is about whether we should prefer an additive or multiplicative decomposition). It turns out that some popular single-site diversity indices have no obvious, decomposable multi-site extension.

In ecology, all these debates are kind of old-fashioned (as well as totally played out), and these days only really excite a small and fairly closed circle of people, who are viewed with some bemusement by everyone else. Different indices behave differently, so there is no one best one. Period.

Jeremy - "In ecology, all these debates are kind of old-fashioned (as well as totally played out)"

I suspect that might be in part because ecology ended up at a better equilibrium, i.e. with a diversity measure that is decomposible, as opposed to one that isn't. It might also be that there aren't as many underlying conflicts within ecologists about values, norms, policy implications.

As mentioned by others, I fully agree the Gini is used because it is used. I think it is also because (as Frances' points out) for the most part it is "close enough" to providing the answer to the question. If economists are anything, they are efficient with their own mental energy. Without intending to misdirect the thread, I think this is why GDP persists as a measure of economic well-being. It is relatively easy to calculate and understand, has a long history of comprehension by non-economists, and is already embedded in the mental tool-box.

If economic analysis is intended to a) explain the world, b) inform and/or recommend decisions, does the use of other inequality measures change the conclusions?

In other words, would a different index lead to different conclusions / actions when comparing UK 1913 with Brazil 1980?

Frances, policy implications.

The second part of the in depth blog post I linked to; was inspired by a paper in the 90s where the authors realized the futility of saving all species so came up with a measure of keeping some of a similar species. Measures diversity. Ecological economist trying to pin a value to biodiversity with odds of finding a cure. Similar species have similar probabilities.

The blog post is by tom leinster who proves some general properties of indices, he later writes a paper on it http://arxiv.org/abs/0910.0906 and mentions the Herfindahl–Hirschman Index, which is apparently equivalent to the simpson index.

And now the guys have been bashing Tsallis vs Renyi entropy together, in which simpson/shannon are a subset of.

Leinster's aware of the between, among, global diversity differences. It's just intersting to watch mathematicians try to prove the general cases.

The comments to this entry are closed.

Search this site

  • Google

Blog powered by Typepad