I’ve been trying very hard not to sound smug lately. While the anti-MOOC bandwagon was once somewhat lonely, it now seems that just about everybody hates MOOCs now, the public, faculty, college presidents and even campus chief information officers. When I read stories like this, I try to remember that folks like Aaron Bady and Siva Vaidhyanathan were writing about the stupidity of MOOCs just as early as I was, and they were doing it far more eloquently than me as well.
What I hope has stood out in my coverage of this subject on this blog has been my tendency to get down in the weeds and explain exactly how MOOCs work. This was the product of my actually taking one. That led directly to one angle of MOOC criticism that I don’t think comes up nearly as often as it should: the obvious flaws in peer grading. Again, peer review is one thing – I use that strategy myself sometimes. However, letting students grade each other’s papers remains a fundamental dereliction of duty. After all, they don’t pay us to chat on Twitter all day, do they?
Yet peer grading survives. Indeed, a team of writing instructors at Ohio State seem to think they’ve made a big stride towards solving the problem with this strategy:
One way to improve peer grading in MOOCs could be to let students grade their peers who graded them.
That’s what a team of writing instructors at Ohio State University decided last spring when they were designing a massive open online course on rhetorical composition, known as WExMOOC.
They built a custom peer-grading system designed to assess not only the quality of the essays submitted by their MOOC students, but also the quality of the feedback that other students in the course contributed after reading the essays.
While the obvious question here is, “Who’s going to grade the grades of the graders?” and I love Frank Pasquale’s line about “graders all the way down,” this would help solve one problem I pointed out in IHE back in March:
Comments were anonymous so the hardest part of the evaluative obligation lacked adequate incentive and accountability.
Know that you’re grades are going to get graded and maybe – just maybe – you’ll do a better job.
Unfortunately, this doesn’t solve the even more obvious problem with peer grading: Students in a writing class aren’t qualified to grade writing (Otherwise, they wouldn’t have enrolled, would they?). Here’s me back in March again:
Good grading technique is difficult enough for graduate students to learn. Because of the size of the course I think I can safely assume that many of my fellow MOOC students inevitably had no history background at all, yet the peer grading structure forced them to evaluate whether other students were actually doing history right.
The implicit assumption of any peer grading arrangement is that students with minimal direction can do what humanities professors get paid to do and I think that’s the fatal flaw of these arrangements.
So why does peer grading continue? Follow the money. When you start with the assumption that your writing class has to have tens of thousands of people in it, then you structure your MOOC around being massive rather than how effectively it will teach writing. That’s why teaching a writing-based MOOC will always be a Devil’s bargain, but, unfortunately, so far the only people who pay the price are the students.
More or Less Bunk 
Hi Jonathan
I am willing to make a little bet, say USD “X” that MOOC’s or their successors (we can settle on a definition) won’t go away. This is not a defense of MOOC’s but rather a studied observation and assessment.
a) the EU with its transfer between institutions is considering inter-institutional credit for MOOC type “courses”
b) In the old “MegaUniversities”, the party delivering the materials, a lecturer, was not the party that graded the tests. This is similar to an Oxbridge experience where one can sit in on lectures and read for the degree or be evaluated on specific skills separately.
c) with the shift to competencies, it may not matter where the student gains the content knowledge to then obtain recognition for skills. It’s not the issue of MOOC’s that is the problem, it’s the idea that we don’t need one of each expert at each institution to build a separate variation on a theme. If one goes to the English department course catalog at a major university, one finds “n” courses on Shakespeare. The need for “y” professors of US History 101 is the issue at hand.
d) Today students can get credit for a college course by taking an equivalent in secondary school or taking a course from an adjunct who may be an ABD or less. In graduate level courses one might look towards a Ph.D.- maybe, as there are examples of ABD’s and post-docs at that level, also.
MOOC’s are symptoms of a more complex etiology which transcends “money” (a real issue) and lies at the heart of the image of “old main” and the enrobed professor holding forth on the lawn surrounded by eager young minds. There is the same ego of the Luddites and the 3rd pilot in a modern electronic cockpit. Hello students, my name is HAL.
MOOC’s are a “chew toy” or a pacifier to divert attention away from the underlying issues at hand.
[…] Peer grading (still) can’t work […]
[…] Jonathan Rees on Peer Grading […]
[…] (For opposing views on peer evaluation, the backbone of MOOC “interaction,” read Jonathan Rees* and Ben […]