Bang your head (Peer grading edition).

10 03 2013

While most people here and elsewhere seem to have appreciated my essay about the futility of peer grading from Inside Higher Ed last week, I have seen enough serious critiques that I want to defend myself here.  While I certainly understand why any superprofessor would want to teach the best MOOCs they can teach, I nonetheless offer what follows as a reality check.

Debbie Morrison, who is open-minded enough to read this blog even though it might well be the polar opposite of hers, rebuts my argument by citing research which suggests the circumstances under which peer grading can be effective:

1) When learners are at a similar skill level.

2) When assignments are low stakes [i.e. when a course is taken for professional development of personal interest…]

3) Where credit is not granted

4) When learners are mature, self-directed and motivated.

5) When learners have experience in learning and navigating within a networked setting [if the review is completed in an open and online setting.]

6) Learners have a developed set of communication skills.

The breakdown in peer grading occurs when the learning environment cannot provide the conditions as mentioned above.

Now that’s all well and good, but the sheer massiveness of a MOOC combined with Coursera’s obligation towards its investors to eventually turn a profit pretty much assures that every single one of those conditions will be violated at one point or another.  More importantly, the rich university administrations that produce course content as well as the poor university administrations that long to replace their faculty with videotaped superprofessors and poorly-paid teaching assistants have every incentive in the world to break every one of those conditions too.

The other critique I particularly appreciated appeared very late in the week at the bottom of the comments to my original article.  Its author signed in as “EnglishTeacher,” and went through the same peer grading process I did as a student in Jeremy Adelman’s course.  They write:

“No one has ever claimed that MOOCs do, can or should replace the full learning experience available to those fortunate enough to be able to be in an engaged on-campus classroom.  And no one has ever claimed that students can take the place of humanities professors.”

Actually, Daphne Koller of Coursera just claimed that students can do a BETTER job at grading essays than humanities professors about a week and a half ago.  With respect to taking over the rest of any particular professor’s job, if the superprofessor provides all the content and student peers do all the grading, what exactly is left?  Not bloody much.  Yes, we can go from desk to desk like my high school math teacher used to do while we worked through our algebra problems, but what kind of wage is that going to get us (particularly as most professors don’t have union representation like so many secondary school teachers do)?

I admire everyone who wants to experiment with new technology to make higher education better for their students.  I really do.  Unfortunately, while those people bang their heads against a wall as part of a futile quest to build a better mousetrap than the one we already have, the powers that be will still be doing their best to make us all technologically unemployed whether robots can do our jobs any better or not.

In the end, the value of peer grading comes down this:  Who can do a better job at grading students essays, peers or professors?  If the answer is “peers,” then why do professors exist at all?*  If the answer is “professors,” then why are so many people wasting their time trying to figure out a way to make the wrong answer right?  As David Golumbia has explained:

MOOCs are being deployed specifically as part of an economic argument whose consequences for liberal arts education are designed to be explosive: they are designed to make liberal arts education emerge as too expensive for us to afford.

Peer grading, like the MOOCs it facilitates, is designed to make the unacceptable acceptable.  It is a strategy created to fit the contours of permanent austerity rather than for the benefit of our students.  So while I agree that pigs look better with lipstick on them, that doesn’t mean the pig becomes any less porcine.

Should anyone choose to keep banging away at the peer grading problem anyway then be my guest.  Just remember that you have been warned, not just about your prospects for ultimate success, but also of the larger political context in which that banging must inevitably occur.

* By the way, I expect the resignation letters of all humanities professors who answer this way to be tendered as soon as they have time to pack up their offices.



9 responses

11 03 2013
Anne Corner

This is a good article and I was particularly interested in Debbie Morrison’s criteria. There is no way Coursera can meet these criteria so all my comments from a couple of posts ago still stand. You are of course right: Who can grade better – peers or professors?

11 03 2013
Peer Grading and MOOCtopia | Gerry Canavan

[…] Jonathan Rees on the peer-grading beat: 1, 2. […]

11 03 2013

I agree that moocs are dreadful for all but the usual tiny minority of highly motivated students. But you’ve missed a larger point about peer assessment, I think. It’s not that the students do it better than us – they don’t. And they don’t because they don’t actually have any idea what the standards are, or how to apply them. They think grading is either absolute and oracular (right/wrong) or completely arbitrary (what does Professor X want). They have zero awareness of judgment as cultural practice, or metacognition about their participation in it. The same could be said of many of their teachers, of course.

Like anything else, in order to understand that huge midrange of conventions, standards, genres, rhetorics, strategies and so on, they need to engage them actively. It’s so they can become responsible judges that they have to practice judging, not because they are already.

11 03 2013
Jonathan Rees

That’s funny, Carl, because I have a whole post on the threat of MOOCs to academic culture (Complete with E.P. Thompson reference!) that’s been sitting in draft since last week. In my own defense, I can only fit so much in one post. Maybe your comment will inspire me to finish it.

11 03 2013

Cool! Looking forward to it.

16 03 2013
Niall Beag

This kind of links to a point I was about to make. Peer review does, as you say, encourage students to think about judgement and metacognition. The problem is, strong students do this anyway, and weak students fail to do this. Peer grading risks simply rewarding the strong students again without developing the weaker ones.

Now, you may say that learning by doing is the best way here, but it’s a very blind form of learning by doing, because we have no model to follow in a fully peer-graded environment such as Coursera.

Consider that in a lecture on any topic you’re given a process, a structure, a concept or a pattern, and you’re shown how it works with several different worked examples. On once this has been done are you asked to apply that process, structure, concept or pattern to solve a problem, create a new work or analyse an existing item.

Why should the act of peer-grading be any different? If it’s part of the learning process, it should follow the established practice, and the worked examples in this case would surely have to be the teacher marking the student’s work.

The earliest things I read about peer review were about constructive growth: allow students to get more feedback by ensuring they were not stuck with only the teacher’s time. It was supposed to be additional feedback, and a further skill for the students at no extra cost. Sadly, it has been perverted and co-opted into the neverending quest for cutbacks, where there’s a lot of quid, and very little pro quo….

11 03 2013
Why and When Peer Grading is Effective for Open and Online Learning | online learning insights

[…] Update: Response post from Professor Rees on More or Less Bunk, here. […]

11 03 2013

Why not just hire professional graders at the fraction of the cost of a professor? This is exactly what the software industry did years ago when it became apparent that having developers write their own code was a waste of their time, but waiting for users to discover and report bugs was too late in the process: hire a professional class of testers as intermediaries.

13 03 2013
What if flipping the classroom does more harm than good? | More or Less Bunk

[…] started me down this train of thought was the line I wrote the other day about my old algebra teacher, Mr. Manzer, going from desk to desk as we worked through our math […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: