I’ve been trying very hard not to sound smug lately. While the anti-MOOC bandwagon was once somewhat lonely, it now seems that just about everybody hates MOOCs now, the public, faculty, college presidents and even campus chief information officers. When I read stories like this, I try to remember that folks like Aaron Bady and Siva Vaidhyanathan were writing about the stupidity of MOOCs just as early as I was, and they were doing it far more eloquently than me as well.
What I hope has stood out in my coverage of this subject on this blog has been my tendency to get down in the weeds and explain exactly how MOOCs work. This was the product of my actually taking one. That led directly to one angle of MOOC criticism that I don’t think comes up nearly as often as it should: the obvious flaws in peer grading. Again, peer review is one thing – I use that strategy myself sometimes. However, letting students grade each other’s papers remains a fundamental dereliction of duty. After all, they don’t pay us to chat on Twitter all day, do they?
Yet peer grading survives. Indeed, a team of writing instructors at Ohio State seem to think they’ve made a big stride towards solving the problem with this strategy:
One way to improve peer grading in MOOCs could be to let students grade their peers who graded them.
That’s what a team of writing instructors at Ohio State University decided last spring when they were designing a massive open online course on rhetorical composition, known as WExMOOC.
They built a custom peer-grading system designed to assess not only the quality of the essays submitted by their MOOC students, but also the quality of the feedback that other students in the course contributed after reading the essays.
While the obvious question here is, “Who’s going to grade the grades of the graders?” and I love Frank Pasquale’s line about “graders all the way down,” this would help solve one problem I pointed out in IHE back in March:
Comments were anonymous so the hardest part of the evaluative obligation lacked adequate incentive and accountability.
Know that you’re grades are going to get graded and maybe – just maybe – you’ll do a better job.
Unfortunately, this doesn’t solve the even more obvious problem with peer grading: Students in a writing class aren’t qualified to grade writing (Otherwise, they wouldn’t have enrolled, would they?). Here’s me back in March again:
Good grading technique is difficult enough for graduate students to learn. Because of the size of the course I think I can safely assume that many of my fellow MOOC students inevitably had no history background at all, yet the peer grading structure forced them to evaluate whether other students were actually doing history right.
The implicit assumption of any peer grading arrangement is that students with minimal direction can do what humanities professors get paid to do and I think that’s the fatal flaw of these arrangements.
So why does peer grading continue? Follow the money. When you start with the assumption that your writing class has to have tens of thousands of people in it, then you structure your MOOC around being massive rather than how effectively it will teach writing. That’s why teaching a writing-based MOOC will always be a Devil’s bargain, but, unfortunately, so far the only people who pay the price are the students.