Should professors tell students exactly what they expect?

Imagine, for a moment, that students acquire valuable human capital during their time at university. Imagine that the grades on a student's transcript reflect his or her level of human capital. Imagine that, every term, a professor uses examinations, term papers, and other assignments, to measure how much human capital each student has acquired over the course of a term.

Conventional wisdom suggests that the best way to go about that assessment is to have clear expectations. See, for example, these helpful suggestions from University of Waterloo's Centre for Teaching Excellence:

Transparent marking criteria. Students should know what is expected of them. They should be able to identify the characteristics of a satisfactory answer and understand the relative importance of those characteristics. This can be achieved in many ways; you can provide feedback on assignments, describe your expectations in class, or post model solutions on a course website.

This sounds eminently reasonable. If professors expects answers to true/false/uncertain questions to contain a diagram, they should say so. If they expect students be familiar with IS/LM analysis, they should say so. If they expect term papers to be a certain length, or use a certain referencing format, they should say so. Clear expectations mean that students are evaluated on what they actually know, rather than on how well they anticipate the whims and idiosyncrasies of their professors' marking schemes.

Moreover, when a professor tells students want she wants, she is more likely to get it. This term I'm teaching a course that involves students carrying out a research project. I marked the first draft of the students' papers using a rubric that looks like this, and handed back the rubric with the students' papers:

 

(Two asides. 1. I use rubrics because I find it hard to evaluate essays, and breaking up the marking into small chunks e.g. "literature review" makes it much more manageable. 2. Most students are not far into their regression analysis in the first draft of the paper, which is why the weighting for that component  is so low).

I'm now marking the students' final papers, and I can see the difference that having clear expectations made. The students have produced essays that contain the essential elements of an applied microeconomics research paper, because I told the students what those essential elements are, and created an appropriate incentive structure. The essays are not perfect, but some of them are really not bad. 

How could one ever object to setting out precise expectations, and developing rubrics?

One problem is that expectations anchor students. The minimum becomes the maximum, and limits what students achieve. "I've got three economics journal articles. That's enough, time to move on."

Also, overly explicit expectations are an inadequate preparation for the real world. In a work environment, no one will say "I expect your research report to be spell checked and have page numbers. And, by the way, it should not be plagiarized". Part of the human capital gained in a university education is knowledge of the unwritten rules, the social norms and conventions, of intellectual life.

A related point is that part of understanding the course material is understanding what's important and what's not. When students are told precisely what will be on the exam, their ability to identify key concepts - to figure out what matters - is not tested. Instead, what's tested is students' ability to read and follow instructions. The students who succeed are the ones who are able to successfully imitate the model answers posted on the website, not the ones who come up with creative and innovative ideas.

Another downside to setting out rubrics and statements of expectations is that they create a contractual atmosphere. I suspect one reason professors are sometimes reluctant to set out clear expectations is that they want to avoid arguments with students along the lines of, "you said I needed three journal articles in my literature review, I've got three journal articles, why haven't I got an A?"

It's also a lot of effort to draw up a rubric, and if you draw up the rubric before you start grading the papers, you may get it wrong - i.e. put a heavy weight on an element that almost all students have failed to complete satisfactorily, and end up with an unacceptably low average. So there are serious drawbacks to circulating a grading rubric before an assignment is submitted. 

 So I can see both the advantages and the disadvantages of setting out precise expectations for students. And yet...

Passive aggression is the besetting sin of academic life. "The indirect expression of hostility, such as through procrastination, stubbornness, sullen behavior, or deliberate or repeated failure to accomplish requested tasks for which one is (often explicitly) responsible." (Wikipedia) If you're an academic, this probably describes a good number of your colleagues, and possibly also - if you're being totally honest - yourself.

One manifestation of passive aggression is "avoiding direct or clear communication". If you are one of those professors who resists putting grading criteria in the course outline, or setting out explicit outcomes, take a look inward, and ask yourself, "Am I truly taking a principled, pedagogically sound stand here? Or am I resisting bureaucratic dictates in the only way I can?"

Comments

[Joseph Savon]

"an unacceptably low average"?? Presumably you give weight to an element because, in your professorial opinion, it has high human capital and is appropriate to the course level. If 'almost all' prove unsatisfactory, it may indicate,inter alia;
-element is unsuited to course level
-it ain't the brightest class
-students didn't allocate effort to match rubric weight
-your pedagogy slipped a little )^_^(
none of these devalue pre-circulated rubrics
Without clear criteria, stated in advance, it's difficult to believe that evaluation of essays will ever be accepted as 'fair & accurate'
On a slight tangent, based on personal experience, I think the 'average' student appreciates a weighting fn even on analytic exams, where there is always a 'right answer': it helps allocation of effort when there's a strict time limit.

[John Dougan]

What I would have appreciated in school was clear expectations of some less formal aspects: "Paper should pretend Freud and Marx are correct descriptions of their respective domains.", "Questions in class shouldn't show the lecturer to be wrong.", "Posted office hours are a snare and a delusion." among others. Granted, these didn't all apply to all of the individuals in question, but all had at least one.

[Frances Woolley]

John - excellent! Also: "That high school English teacher who encourages you to write your innermost thoughts in your journal? Don't."

[Adam Chapnick]

A number of the challenges with your rubric (albeit not all) can be managed by expanding the rubric to better reflect the difference between A+, A, B, and C level work. For example, rather than lit review - 10 with a number of criteria, you could include brief descriptions of different quality lit reviews:

A+ --> draws from more than 3 relevant econ articles; identifies original connections and contrasts among the articles; organization is both clear and based around the most significant ideas; considers significant empirical and theoretical contributions.

A --> draws from at least 3 relevant econ articles; identifies connections and contrasts among the articles, some of which are original; organization is based around ideas; considers empirical and theoretical contributions

B --> draws from 3 econ articles, at least two of which are important to the field; identifies connections and contrasts; organization is based at least somewhat around ideas; considers empirical or theoretical contributions

C --> draws from no more than 3 econ articles, most of which are not important to the field; identifies connections or contrasts; organization is by article; unable to recognize the difference between empirical or theoretical contributions

[Frances Woolley]

Adam, yes, that's much better, but also much more effort!

Though this also illustrates the dangers of being super-explicit with expectations. E.g., with your "A" criteria - how can I assess if a connection between articles is original if the articles aren't in my field? Is there an objective criteria by which to decide which are "the most significant" ideas? Which are significant contributions? With regards to your B level criteria - why is important to the field here and not for the As? Do I want to get into an argument with a student about whether or not an economic article is "important to the field" - and how can undergrad or even masters level students be expected to know which articles are important to the field and which ones are not?

Also how do you compare one student who has three fantastic articles but does a lousy job of analyzing them, and another student who has three lousy articles but does a fantastic job of synthesizing them?

In some ways what you've suggested is an aspirational rubric - what I would, in my dreams, hope to get from undergraduate students - rather than a practical one. At the same time, it's really good to share aspirations - to say what a beautiful literature review would look like.

[Donald A. Coffin]

This is a nice discussion of a fairly difficult issue. And the comments add value; the problem I always had with developing rubrics was the need to make them specific enough to be useful and flexible enough to allow for me to be (pleasantly) surprised.

[Blissex]

«Imagine, for a moment, that students acquire valuable human capital during their time at university. Imagine that the grades on a student's transcript reflect his or her level of human capital.»

That's pretty hard to imagine for me: the overwhelming value of a university degree is the degree itself, or more precisely, in getting admission to the university that gives the degree, and there are books and papers that show the compelling quantitative evidence for that.

The grades on the transcript are mostly a formality, and smart students spend a lot of effort getting admission to a valuable university, and thereafter go through the motions, and make the least effort needed to get the degree, as they have "clear expectations of some less formal aspects" as another commenter says and this relates to:

«Part of the human capital gained in a university education is knowledge of the unwritten rules, the social norms and conventions, of intellectual life.»

Indeed, and the students who care about the content of their courses are usually a minority, the insecure, or the swots and the nerds, and the proposal here to make evaluation criteria more transparent may be good for them though, as they tend to get hang up about these things.

Pious homilies from entities like "centres for teaching excellence" are meant mostly as marketing exercises, to make-believe that quality of teaching really matters to careers or that "people" like potential employers really care about it.

[cm]

Blissex: From anecdotes, a number of people seem to take the signaling value of a degree (especially at the doctorate level) as coming from or possessing a culture of achievement, of not giving up or settling for a lower outcome when facing adversity, but making extra effort to overcome it. And probably assuming that this will last at least for a few years while the initial work track record "proof of merit" is established and/or disillusion sets in.

But one may consider this included in "social norms and conventions".

[Blissex]

«the signaling value of a degree (especially at the doctorate level) as coming from or possessing a culture of achievement, of not giving up or settling for a lower outcome when facing adversity, but making extra effort to overcome it.»

I must agree that for a doctorate or an MBA that signaling value matters in some countries, in the USA for example; but in several other countries doctorates in particular result in lower lifetime earnings and more difficult careers, as they signal something else instead, and many employers other than universities don't want "otherworldly" "know-it-alls" :-). It can cut both ways :-).

It depends on context indeed; I also admit that in some countries transcripts and marks matter significantly at least in some professions as in those cases employee insecurity, and swottiness and nerdiness, are valued by many employers.

Apart from whichever signaling value they ascribe to credentials, in general employers care little about the actual content of the sources, beyond a fairly low minimum, for most non-professional degrees. Put another way most employer HR departments want to make their jobs simple by delegating staff selection to a single metric delivered for free to them by universities :-).

Usually students, or at least their mothers, know well which metric (admissions, transcripts, ...) employers actually value as a signal.

[Frances Woolley]

Blissex:

"That's pretty hard to imagine for me: the overwhelming value of a university degree is the degree itself, or more precisely, in getting admission to the university that gives the degree"

Even more precisely, getting admission to *and graduating from* the university that gives the degree.

In a signalling world, a professor's job is to sort more able from less able students. We add value by identifying and failing low ability students. . Hence the arguments for/against setting out clear expectations are totally different in a signalling world. Does lack of clear expectations improve the informational content of the "completed university" signal? One could, in fact, argue that setting out clear and detailed instructions that any idiot could follow means that any idiot can pass - and the signalling value of a university credential is diminished.

With signalling, the entire "clear expectations help students learn" discussion becomes moot.

cm - people who start university but fail to complete a degree have worse outcomes by almost any measure than people who never start university in the first place. So I think there is something to the giving up/effort story.

[Jacques René Giguère]

Frances to cm: would it be because those who never tried have a better understanding of their abilities, a cognitive trait they can use all their life or are those who fail to complete are so marked by failure they get discouraged for a long time?

[Jedgar Mihelic]

Now, this is time consuming, but one way to look at it is if you read all the papers through once without a pen in hand, you can see where there is systemic weakness and then construct your rubric from there (or if the class is large enough, just a sample). Then you can reread with your weights and then look on what you might want to focus on as a teacher in terms of setting those professional norms. (Repentant former graduate student in Literature).

[Donald A. Coffin]

Jedgar--The problem is that the usual practice is to construct the rubric *in advance* and *share it with your students* so that they are not in the dark about what they are supposed to be doing. This is especially the case when the activity (e.g., a statistics-based research paper) is something they've never done before. A colleague (and former teacher) of mine who did this developed his research paper rubric from his experience as a journal referee. He had the students read journal articles and apply the rubric to what they read. Then he had them begin the process of writing their own research papers (this was, actually, in a PhD program).

[Squeeky Wheel]

I just gave a third year programming final exam. At this school, it is typical to give a textual description "Program must compute the daily 5 year - 5 year ahead expected inflation from the given data" followed by the correct output "Correct output is 1.0, 1.1, 1.2, 1.1, ..." One of the problems on the exam given yesterday had only the textual description. Several students freaked out asking how to ensure correct output without knowing the correct output. Of course, outside exams, if I have a program producing the correct output, I don't ask someone for a new one! It is helpful to give students some indication of what is expected, but at some point they have to see what problem solving looks like.

[Frances Woolley (for Karl Skogstad)]

Jacques Rene - "those who never tried have a better understanding of their abilities" Possibly. Or they're willing to make a decision and stick with it. Definitely something like that.

Donald: "He had the students read journal articles and apply the rubric to what they read."

That's a really neat idea.

[Ruth Cameron]

Frances - do you have any reference for your comment that students who start university but fail to graduate have worse outcomes? I find that very interesting and would like to read more.

[Toby Handfield]

Here's a different sort of expectation that I have: 5-10% of students in a large class will fail by the end of semester.

I don't think that's a good expectation to share with students! Indeed, there is probably good pedagogical reason to cultivate the perception that I believe all the students will do very well -- even though this is a misperception of what I believe. Given what we know about stereotype threat, etc, there is lots of reason to think that students will achieve more if we can convince them that we have high expectations of them, even when those expectations are unrealistically high.

[Frances Woolley]

Ruth - I was looking at the relationship between life satisfaction and education in the canadian community health survey just the other day - the people who have some university/college but not a certificate have lower levels of happiness than people who have completed high school but not gone onto university. This and related phenomenon - that there's something about graduating rather than just simply attending school - are often termed the "sheepskin effect". I looked for a nice reference for you, though, and couldn't find one quickly.

Toby - " Indeed, there is probably good pedagogical reason to cultivate the perception that I believe all the students will do very well"

This is a fascinating idea - that laying out unrealistic expectations may increase student learning.

[cm]

Blissex: "I must agree that for a doctorate or an MBA that signaling value matters in some countries, in the USA for example; but in several other countries doctorates in particular result in lower lifetime earnings and more difficult careers, as they signal something else instead, ..."

Yes I was talking about the US, and my opinion is this is part of a more general phenomenon of "brand awareness" (if not "brand focus").

"Usually students, or at least their mothers, know well which metric (admissions, transcripts, ...) employers actually value as a signal."

And here we go for the killer argument ... mom knows better. No, I don't actually think in general mothers have a good/better than others assessment of this. But you are certainly right when broadening the argument to "social environment counselors".

[cm]

Frances/Jacques:

"cm - people who start university but fail to complete a degree have worse outcomes by almost any measure than people who never start university in the first place. So I think there is something to the giving up/effort story."

"Frances to cm: would it be because those who never tried have a better understanding of their abilities, a cognitive trait they can use all their life or are those who fail to complete are so marked by failure they get discouraged for a long time?"

BTW short introductions - I'm a software developer with CS degree, and my understanding of other domains is of the armchair/life experience variety - no other formal "education" beyond mandatory college classes.

"Failing college/uni/tertiary education" is correlated with pretty much two things (may be considered more than two):

* Life adversity (family issues, health problems, having to abort college for financial reasons, unexpected mother-/fatherhood, etc.) - these are causations that very likely have impact on general socioeconimic success.

* Being talked/coerced into college despite lack of inclination or ability - I know first hand examples of talented peers who were high-achieving throughout high school, and then in uni reverted to mean. Following cases with analysis:
* An arguably brilliant and generally nice guy who was socially abrasive/inept to the point of deliberately letting "authority issues" escalate to levels most people wouldn't - he was good, but got into a situation where he couldn't complete his PhD probably because of limitations related to this
* Another IMO above-average smart guy who excelled through high school under the pressure of his parents (and while living with them) - when on his own, his motivation to deal with adverse (socially imposed) conditions plummeted, and he pursued an "easy/secure government job" definitely below his "skill" level - perhaps the smart choice in the long term (I may have known several individuals in similar straits who took a lower-risk and nominally "lower-skill" exit)

On the second point, I'm from former East Germany, and what I convey happened in the 90's during our social revolution and reunification, which injected another dimension of uncertainty and disturbance into everybody's circumstances. Otherwise people may have made other choices (in a presumably starkly different social environment - hypothetical as it may be).

Jacques: There is self-selection, of the 'cowardly' or 'wise' type (for my lack of coming up with better descriptions), i.e. people avoiding things where they perceive either little chance of success or little chance of *enjoying* the (so-called?) success. E.g. there are certain advancement opportunities (BS or even real) where you are asked to put in significantly more work for an imaginary or tangible payoff - then there are the dimensions what you are "guaranteed" to get out (probably conditional on "succeeding") vs. entirely speculative payoffs, e.g. the opportunity to participate in the next level of audition/competition, etc. And not unimportantly, what *other* people (much likely those who encourage you to participate!) are getting out of it. And comparing to peers who "don't participate".

So maybe it is not so much that people get unfairly discouraged but plainly see that some things are not for them, psychologically, or there is no perceived advantage. Maybe they could have been successful, but the uncertainty or abhorrence of the imagined lifestyle weigh more heavily.

Of course a lot of this is probably my own projections.

[Jacques René Giguère]

"Sheepskin effect": You're only as good as your last record. You can go out of HS with a bang: "Mom I graduated!" or out of college with a whimper:"Mom, I failed at university..."
There are medals for bravery and others for service but there is no diploma for two years of study (though some universities will tranform your fisrt two years into a certificate) :"Mom I have my degree!"

[mike]

Yep, telling students exactly what to expect will prepare them for real world analysis. Not!

[Too Much Fed]

Definitely.

Don't say short answer when you mean short essay.

[cm]

Jacques: I think it is something else. People are pushed into a narrative that doesn't come from themselves but is externally imposed. Left to their own devices, most (at least many) are directionless. There has been a loss of "community" and other forms of social structure that gives individuals and groups a direction. Most people go to college because that has become a minimum requirement for what was passing as "good" jobs not requiring back breaking labor only a few decades ago (i.e. the prior generation's experience), and it is heavily pushed on them by everybody. Of course they are interested mostly in the degree paper! Because that's what the employers are interested in also.

Most jobs don't actually require this level of teaching, outside of technical specialties that are probably a minority of jobs overall. Most jobs require some foundations, but most skill and experience is acquired on the job. Back in my day, only around 10% or so of pre-HS students would (be allowed to) go on to high school and then college directly. A larger proportion would be put through a combined vocational+high school program. Some number of them would also go to college eventually, "upgrading" e.g. from a technician role to an engineer role (from an individual perspective, perhaps out of desire, but in the bigger picture because of a social need in a society moving from industrialization on to higher technology). This also made for some subculture of people taking pride (and frankly sometimes turning into a sense of superiority) in having learned a trade as opposed to those who just went to college without on-the-floor practical experience.

Now much of the "technical" work, together with manufacturing, has been offshored and continues to be. What remains (so far?) in country are (in no particular order) administrative roles, sales, operations, healthcare, legal, law enforcement, construction, transportation, durable goods maintenance/repair, etc.: the whole infrastructure of keeping the local economy and workings of society going - job functions that are location or language/culture specific.

Most (not all) of these job roles have in common that they are relatively light on "classroom teachable" skill - they are more of the vocational/learning on the job type. I want to make it clear that I *don't* mean "low skill" - but by far most skill acquisition is practical, not from textbooks. "More college" will not help here, certainly not to address the "training" aspect.

[Frances Woolley]

cm: I agree with you about university being a means to an end for many students - and that's fair enough. I do think there are more "classroom teachable" skills out there than your comment suggests - whether or not universities are actually teaching them is another matter entirely.

[cm]

"Classroom teachable" is also not really what I meant - more accurate would be that the subject matter is not structured and formalized by scientific theories, either because it cannot be, or just because nobody would attempt it, or it is not general/workplace independent enough. And thus it is arguably outside the purview of uni/college teaching - their function is not to train the workforce for specific companies or institutions.

There is still plenty scope for classroom teaching in the vocational domain - but there also the taught material must be general for the domain. Also when companies/institutions offer internal classes (because eventually the staff *needs* this training for the organization to work), these classes will be held in classrooms. But they will be specific to the organization's need.

Yet another aspect of the whole "why college" problem is that in many fields the required "skill"/foundation level *has* risen - but the human material hasn't "improved" to kids can be taught a higher level in the same number of years. Then just in principle the lifetime years of learning/training must be increased. Whether what is currently practiced is very effective is another question.

To some extent you can replace breadth/depth on the basics with higher level material (e.g. drilling multiplication tables vs. higher level concepts), but this is very limited - higher level concepts cannot be understood without enough basics.

Now (or since a while?) cursive handwriting is being phased out - it's now placing letter by letter. Who writes anymore, right? Now you need "computer skills", i.e. presumably navigating touch screens and "apps" provided by major internet companies, as that's what replaces "traditional" teaching methods apparently.

[cm]

When I was in high school, calculations transitioned from pen+paper and slide rule to electronic calculators - I still got to see both. A few years later I started hearing complaints that graduates had difficulties with, or were plainly unable to do written division and mental arithmetic in general. Now again, who needs that, where this skill would have been necessary, now computers and software packages are used.

[rsj]

I think a lot of University professors assume that the process of applying for a job is like the process of applying for graduate school. You need transcripts, letters of recommendation, even an application fee. They also flatter themselves by thinking that they have so much power over a student's future career.

But in my experience, I don't think anyone looks at transcripts. Maybe things are different in Canada or in certain sectors of the economy -- I can only speak for myself. I'm sure, for example, if you are applying for an internship at an investment bank, they would probably only hire from a few schools and they might ask for your transcript. If you are graduating a vocation school or a school that directly teaches the skills of your career, such as medical school, then the same might apply.

In my world, there is first an HR layer that screens employment applications, and in this layer they want everything standardized, if possible. E.g. it's better to fill out the information on the employer's website and then upload a cover rather than upload a resume, as that allows for easier machine processing. If you have to upload a resume, then there are software tools to auto-read them, and those without the tools spend a lot of time reading stacks of resumes in HR departments across the country. Please keep your resume to 1 page. No one cares about your hobbies or whether you are a well rounded person. No one wants a
lot of details beyond keywords. The HR people look for certain keywords and attributes, such as "X years of experience", "Legal right to work in the country", and knowledge of the relevant skill sets. Moreover, they only go by what you put in your resume. There is no second step, where they reach out to you and ask for more information. If they reach out to you, it is to set up a phone screen, in which you will be interviewed.

Phone screens are followed by in person interviews and a hiring decision. In tech, the in person interview can be an all-day thing, in which they bring in different people to talk to you for an hour each for the day. They ask you to write code in front of them, or they give you problems to solve. Some might ask for examples of previous work. They might give you a take home assignment to be completed and brought to the interview. Or it might be two half day events. Then they make a hiring decision.

Once the hiring decision is made, there may be for some jobs a background check, with your hire contingent on the check, but the check happens after you sign the employment contract. The background check does not look at your school grades, it's a search of courthouse records for felony convictions and verification of employment history and the accuracy of what you put on the resume. Which employers do the background check and how detailed it is depends on the industry. Do not lie on your resume, because you might lose your job.

Point being, you are hired for your skills, and since the median worker is in their mid 40s, and since with the exception of trade schools, colleges don't directly teach any of the skills on which people are hired, and since your proficiency in those skills at the time of interview is going to be vastly different than your proficiency in those skills when you were in college, it just isn't useful data for an employer. I think professors flatter themselves when they think that the grades they give out are going to allow or disallow their students to get a job. No one cares about the grades they give out, as long as the student graduates.

The one exception to this might be internships. I guess it's possible to care about grades during internships, but no internship program I was ever involved with asked for grades or transcripts. They asked for the student's major and interests. Grading is just too different between the various schools, and internships are low risk for employers. Here, too, the in person interview and the phone screen is what determines if you get the job, and the school you are attending together with your major determines whether you get the interview.

If you stay in the educational universe, grades are useful data for graduate school, law school, medical school, or a post-doc, who knows, maybe even grant applications, but very few people care about grades outside of the education universe.

[Frances Woolley]

rsj: "If you stay in the educational universe, grades are useful data for graduate school, law school, medical school, or a post-doc, who knows, maybe even grant applications, but very few people care about grades outside of the education universe."

I don't disagree with your characterization of the world. But I would say that the exceptions you list create a huge space within which grades matter. Take, for example, internships. At Carleton, we have an undergraduate co-op program for economics students. A good percentage of the students who participate in the co-op go on to get jobs with the federal government. But only students with a certain minimum grade point average are admitted into co-op. Giving a student a C rather than an A might make that student ineligible for co-op, and have a big effect on their life chances. Likewise, just about every prof has been petitioned by a student who says "if you can just increase my grade from a B+ to an A= I will be able to keep my scholarship, and that's worth XX thousand dollars to me..."