Economists don't get SSHRC money, grad student edition

The other day a colleague was explaining how SSHRC-funded Canada Graduate Scholarships are awarded at my university. "It's mostly driven by GPA," he said. "Grades in economics are so low that your students don't get a look in." 

I spent some time messing around with the SSHRC awards engine. His suspicion that relatively few Canada Graduate Scholarships go to economists is well-founded. Taking all masters and doctoral Canada Graduate Scholarships together, and looking at the total awards over the past three years, fewer than two percent of scholarships went to economics students. SSHRC funded twice as many philosophy students as it did economics ones. Three and a half times as many political scientists. There are almost ten times as many psychology students receiving Canada Graduate Scholarships as there are economics students.

Moreover, it's getting worse. There is a gradual downwards trend in the number of Canada Graduate Scholarships going to students studying economics. 

See for yourself. Here is a breakdown of the number of awards by discipline. The "adjusted" numbers take out award holders whose discipline is "not specified", making it easier to see trends over time.

Here is a breakdown of the amount awarded. The "amount awarded" differs from the "number of awards" because doctoral awards are worth more than masters-level awards.

A spreadsheet containing the numbers used to generate these tables, as well as the actual numbers and dollar values involved, can be downloaded here.

Why aren't more awards given to economics students?

Part of the story could be that economics has seen less grade inflation than other disciplines, so our grades are lower - and the difference is widening over time. This paper by Paul Anglin and Ron Meng gives evidence that grade inflation is occurring, and that some disciplines are inflating grades more rapidly than others.

Part of the story may be size of the potential Canada Graduate Scholarship applicant pool in economics. Casual observation suggests that economics attracts relatively more international students than some other disciplines, and international students are not eligible for Canada Graduate Scholarships. Unfortunately publicly available education data typically lumps together "social and behavioural sciences and law" making it impossible to compare the number of awards by discipline to the size of the potential applicant pool.

Part of the story may be that economics students write lousy research proposals, and economics professors write lousy letters of reference for their students. And part of the story may be that the economics profession as a whole is pretty ineffective at lobbying SSHRC. This is just speculation however.

But the bottom line is: no one who is involved in the awarding of Canada Graduate Scholarships takes students' job prospects into account when making those awards. Hence students can, potentially, end up being subsidized to undertake programs of study that generate little or no private or social return.

And why aren't students' employment prospects taken into account? I can only speculate, but I would guess a number of factors are at play: Committees have little or no reliable information about individual students' future prospects. Scholarships are awarded by university professors, who have little interest in inquiring about their doctoral students' futures - they might not like the answers. Academia is characterized by enormous inertia: any change creates losers, and those losers will fight tooth and nail against any change. 

So the status quo prevails.

Comments

[Taylor_wright]

I think another potential factor could be the level of existing financial support for grad students across these programs. I haven't looked at the numbers nor do I know if such numbers exist in Canada, but anecdotally it appears that economics graduate students receive better funding packages than do other disciplines (my guess as to why is that the student pool that economics draws from likely have better outside offers, and so it is required in order to attract students). As a result, committees could be giving funding to other areas because economics students are already doing relatively well in terms of financing their degrees.

On your point about economics professors writing lousy references: I'm not sure if you mean lousy in the sense that economists tend not to be good writers or lousy in the sense that they are relatively less emphatic in their praise than other disciplines but I have a little story: I had a conversation once with someone who was doing the filtering at the faculty level for SSHRC and this person told me they had never seen so many "top five percent" students. In fact, I was told that in one program the applicants were nearly all internal and so this person and the other economist present calculated that over 20% of the applicants from the in-house program were considered top 5%. A bit of a puzzler for them.

I'm also thinking that it can be difficult for non-economists to evaluate the quality of the research proposals written by economists (this is no doubt not helped by economics students writing poor proposals). Not only because we tend to be more quant-y than other disciplines (though it seems the differences in that respect have been shrinking) but also because the types of questions we tend to ask and the scope of the questions we tend to ask likely are very different than those in other disciplines. For example, a proposal with a research question in the style of Steven Levitt/Freakonomics (or what some might disparagingly call "cutenonomics") probably has a better shot at being well understood since it more closely resembles the work done in other disciplines. Contrast that with a proposal where someone wants to estimate market power in the brewing industry (to take a recent NBER IO paper).

Anyways, those are some of my ramblings on this ¯\_(ツ)_/¯.

[Frances Woolley]

Taylor, thanks for those comments. In response -

"As a result, committees could be giving funding to other areas because economics students are already doing relatively well in terms of financing their degrees." I don't really think that's what's going on here. In my experience, academic committees tend to work more on the "to those that hath shalt be given" principle. Though it could be that relatively more econ PhD students are taking themselves out of the applicant pool by working full time.

"lousy references" Thanks for expanding on this point. I suspect both factors are at play.

"I'm also thinking that it can be difficult for non-economists to evaluate the quality of the research proposals written by economists (this is no doubt not helped by economics students writing poor proposals)." Allan Gregory, who is at Queen's, raised this point on twitter - his experience is that the relatively poor quality of econ research proposals is a factor in his university's internal committee.

[Craig Burley]

How many eligible grad students are there in economics departments as opposed to these other fields?

As for the implication of Taylor's comment that the types of questions economists ask are less dumbed-down than in other fields: spare me.

[Frances Woolley]

Craig - # of eligible grad students is unknown, unfortunately. But even if it were known - see this nice little piece by Marc Frenette on detailed earnings by field of study: http://www.statcan.gc.ca/pub/11-626-x/11-626-x2016056-eng.htm.

The average age-adjusted earnings of people graduating with a masters in economics are $112,519 for men, $81,987 for women. With a masters in sociology: $68,619 for women (no data for men). With a masters in history: $79,406 for men, $69,039 for women. A general psychology masters? For men, $74,091, for women, $66,128

Why are we not, as a matter of policy, doing a little bit to nudge students into the degrees that yield a slightly higher return on investment?

[Jon]

Hi Frances,

Awesome post, thanks for putting this data together.

I did my MA in economics but I actually received my SSHRC as a philosophy student (my proposal was a crossover philosophy and economics topic). I was a double major, and I decided to apply for my SSHRC through the philosophy department because the faculty was much more supportive, including SSHRC writing workshops and several professors offering to review my proposal. I was also confident I would get stronger references (even though my econ grades were higher).

So in my experience it wasn't about GPA (it was a lot easier to ace econ tests than consistently write A+ philosophy papers), but just completely different department cultures. The philosophy faculty really advertised the SSHRC deadlines and actively encouraged us to apply. In general it seemed like they really wanted their students to succeed and from what I could tell they wrote glowing and detailed reference letters for us. They also had a culture of open doors so we could drop by and get advice anytime. I should say I had some extremely kind and encouraging economics professors who I am very grateful to, but overall the difference between the departments was stark in terms of their openness, encouragement, and support of students.

Just the experience of one student at one university, but it resonates with some of the issues you raised.

[Frances Woolley]

Jon - thanks for this - I hope every econ prof who drops by this page reads your comment.

[John Chant]

Great post, Frances. I know at one time a single Psychology program and also a single English department, both at the same university, had more doctoral fellows than all of Economics departments combined.

The GPA argument, from my experience, works against economists. But that's not all.

Another thing that works against our doctoral students is the difference between programs in Economics and others. An economics PhD combines course work with research while many others are pure research.

Discussions go like this. For applicants are the end of first year: we can't give this student an award because she's not even started her dissertation. At the end of second year: we can't give this student an award because he has made so little progress on his dissertation.

As a result, screening the awards at the university level works against economics. I can't see how cross discipline comparisons make any sense, given the differences among them. We are outnumbered by disciplines that have research only programs.

Frances, as you know this experience is quite dated but I don't think things have changed.

[rsj]

It seems strange that there aren't set asides for each discipline. It's unlikely that each discipline has the same culture/grading standards/ability to write proposals. So why not dole out the grants based on some amount allocated for that discipline.

I don't see what is gained by having Law students compete with history students for graduate funding. Decide how much you want to fund history and law and then pick the best in both.

[Erik Lund]

A cynical person, of whom there are certainly none around here, would suggest that SSHRCs are given out according to traditional quotas, with minor variations from year to year just to give an impression of randomness. Such a cynical person might have been told as much, in so many words, on several occasions. This hypothetical person might, hypothetically, be talking about history, but another hypothetical person might infer, if this anecdote weren't entirely hypothetical, that the same applies in other fields. (Insert topical joke about counterfactuals here.)

God only knows how it was first established that, say, UofT gets two PhD grants in economics each year, but changing that would mean taking grants away from other disciplines, and that ain't on.

[Frances Woolley]

John - good to hear from you! I don't think things have changed that much - the only thing is that there are more empirical doctoral theses these days, which might make it easier to explain the methods and motivation to people in other disciplines.

rsj, Erik - interesting to read your two comments side by side, rsj being surprised that there aren't set asides for each discipline (which is, formally speaking, true for the Canada Graduate Scholarships), Erik saying that effectively there is a quota system, based on historical entitlements, and maintained by pervasive loss aversion (no one is willing to "give up" one of their scholarships so that a student in another program can get one).

All very informative!

[Chris Kam]

I have sat on the faculty-level committee that recommends / awards doctoral and MA SSHRCs at my (major Canadian) university several times. The incentives to grade inflation are tremendous, and woe to the department / discipline that does not bend to them. It is just that GPA is the one statistic that is easy to obtain and ostensibly comparable across disciplines. (It is ironic that it's the economists who miss this! In contrast, I've yet to see a course grade of below 92 out of our Ed school!) Furthermore, it's dead easy to publish & publish early in in one's career in many disciplines in arts & social sciences. Sure, much of this is claptrap -- but it makes for a long and seemingly impressive CV. In economics, in contrast, a grad student typically develops and carefully polishes 1-2 major papers to show on the job market; the ideal is quality over quantity. So in these key respects economics grad students are essentially bringing knives to a gun fight. It is a sad state of affairs, but economists ought to be able to figure out the strictly dominant strategy in this sort of game, right? Even the political scientists have!

[Frances Woolley]

Chris: "economists ought to be able to figure out the strictly dominant strategy in this sort of game, right?"

Let's think about the gains and losses for an economist from grade inflating.

If the prof inflates grades, he will attract more students, and students will give him higher teaching evaluations. Plus the student may be able to win SSHRC and other graduate scholarships, and get into a better grad school.

If the prof "holds the line" on grades, he will gain a reputation with his departmental colleagues and students for toughness, and only his best students will be able to go onto graduate school. This will, in turn, enhance the prof's reputation with colleagues at other universities.

So giving out low grades can definitely be the dominant strategy - it all depends upon how much weight one puts on different pay-offs.

[Christine]

I think John Chant and Chris both have excellent points. In econ students rarely have a research plan early on, and when they do work towards one big hit rather than knterim contributions. They may be better off applying early - but then grades and references matter more.

[Frances Woolley]

Christine - agreed! The comments on WCI are always good, the ones on this post have been even more insightful, informative and constructive than usua.

[Stephen Gordon]

I've been on the internal SSHRC pre-screening committee here, and students in other fields always seem to have 2 or 3 publications by their 2nd or 3rd year - their MA thesis, plus co-authorships with their advisors. I try to explain that it's extremely rare for a PhD student in economics to have even one publication when they graduate, but I don't know if that argument holds much sway at the adjudications in Ottawa.

And then there's the grade inflation point. I see a lot of straight A or A+ files in other fields, almost never in economics.

[Colin Percival]

The use of GPAs for awarding scholarships creates problems well beyond tri-council awards. At my institution (SFU) we have $1.5M/year of undergraduate funding which is currently awarded on the basis of GPA (and some basic qualifying criteria, but GPA is the competitive factor); unsurprisingly, disciplines which award higher grades see more of their students receiving scholarship money.

The situation is bad enough that we're hoping to develop a "scholarship GPA" which adjusts for differences in grading criteria; it's not trivial (you can't just adjust for the average grades in a course; for example, our Business courses give out slightly above-average grades, but have very above-average students) but a regression on faculty, basis of admission, and degree progress seems to work fairly well at distinguishing "easy course" from "good students".

I'd be happy to exchange notes if you want to explore these issues at your institution(s).