“What’s happening right now is that xMOOCs are moving backwards into replicable content from the interaction and assessment pole while textbooks are are moving forward into interaction and assessment from the replicable content pole.
The end result of this is not necessarily massive classes. It’s broadly used courseware — software that provides much of the skeleton of standard classes the way publisher texts do today. In other words, the best way to think of a MOOC isn’t really as a class brought to your doorstep — it’s more a textbook with ambitions.”
Whenever I get involved in one of those big publisher focus group thingies, I always make the same suggestion. Instead of being forced to enter their chosen universe, I ask for their services à la carte. I think this dates back to the early days of my career when I got sick of carting thirty-year-old maps into my classroom and started accumulating plastic overlays for the elmo. [If you don’t know what an elmo is, you can click here. I’m shocked to see that there are any of these things still around.] I wanted maps and pictures that reflected what I talked about in class already, not what some publisher thought I should be discussing. Therefore, I had to accumulate overlays from a wide variety of U.S. History survey textbook publishers to create something essentially personalized to meet my needs.
The Internet has rendered that collection obsolete for me now. I can create my own PowerPoint slides (with very little text) faster and with a much greater selection of possible pictures than I ever could have imagined when I started out in this business. To me, this is what teaching with technology is all about. More choices. Better control of my own time. Three cheers for progress!!!
Unfortunately, the giant publishers and my own employer for that matter are either unwilling or unable to give more options unless I enter their particular technological universe. Time saving is invariably the incentive for making the move to any technology, but this seems especially true for moving any aspect of education online. This is from Anya Kamenetz, writing for NPR:
But instructor time remains the most expensive resource, and it’s often scarce and rationed, especially in the online realm.
[Andrew] Smith Lewis [of Cerego, an edtech startup], along with other ed-tech people I talked to, framed the next generation of computer enhanced learning as a way to free up professors to do what they do best — not to replace them.
Professor [Jeff] Hellmer [of U-T Austin], for one, was so taken with the Cerego platform that he decided to incorporate it into his live, in-person classes, starting this summer. He sees it as a labor-saving device: The machine will handle the shoveling in of facts, while he does the cultivating of the students’ mental gardens.
Pardon me if I’m not quite so trusting. I want to maintain control of the technology rather than let the technology control me because if my technology doesn’t reflect my own teaching priorities I might as well be signing my own unemployment compensation request form. No matter how often edtech entrepreneurs insist that they want to do things right, there’s a class of administrators who have already shown time and again that they want to do things wrong. You know, the ones who brought us adjunct labor and huge executive salaries.
Of course nobody should trust these administrators to use MOOCs responsibly either. I’ve already discussed the prospects of the just-in-time professor here, but the idea of creating a MOOC-y textbook or a textbook-y MOOC raises the prospect of eliminating teachers entirely for those students who can’t afford a college education that includes them. If this sounds ludicrous to you, what exactly is a self-paced, on demand MOOC then? A video game with scholarly content.
Unlike the early days of this blog’s technological turn, my somewhat paranoid rantings are hardly alone anymore. As Andrew Leonard wrote in a little-noticed piece at Salon on Friday:
Early returns on MOOCs have confirmed what just about any teacher could have told you before Silicon Valley started believing it could “fix” education. Real human interaction and engagement are hugely important to delivering a quality education. Most crucially, hands-on interaction with teachers is vital for the students who are in most desperate need for an education — those with the least financial resources and the most challenging backgrounds.
Of course, it costs money to provide greater human interaction. You need bodies — ideally, bodies with some mastery of the subject material. But when you raise costs, you destroy the primary attraction of Silicon Valley’s “disruptive” model. The big tech success stories are all about avoiding the costs faced by the incumbents. Airbnb owns no hotels. Uber owns no taxis. The selling point of Coursera and Udacity is that they need own no universities.
“No more pencils. No more books. No more teacher’s dirty looks.” The whole thing has the feel of some kind of teenage revenge fantasy against educators everywhere, but it still makes perfect sense. If textbooks have ambitions to be courses and MOOCs have ambitions to be used like textbooks, what’s left for the professors to do then? If we faculty let this happen all in the name of our own convenience, then we have nobody but ourselves to blame.
“White-collar professionals, too, are subject to routinization and degradation, proceeding by the same logic that hit manual fabrication a hundred years ago: the cognitive elements of the job are appropriated from professionals, instantiated in a system or process, and then handed back to a new class of workers–clerks–who replace the professionals.”
This post has more views than any other on the history of this blog. I have no idea why. OK, I think it’s because it shows up pretty high on the results list when somebody Googles “Coursera” or “Udacity,” because it gets about 100 hits a day these days, but why that post gets all the traffic as opposed to anything else that I’ve written about MOOCs is indeed a mystery to me. Even though I consider it to be a rather mild denunciation of inert MOOCs that students only absorb passively, it has attracted a fair number of hostile comments by now. I figure as long as passersby don’t insult me or write something that’s prima facie offensive, I’ll approve such comments for display.
Yet I can’t help but wonder what kind of people search the Internet for anti-MOOC blog posts that they can denounce their authors. My friend Vanessa has suggested in the comments to that post that MOOC providers have hired paid shills. I’m not sure I’d go that far, but there certainly are a fair number of people who are so excited by all the free content that they now have access to that they get very defensive on behalf of MOOCs and MOOC providers. Inevitably, these must be mostly people with college degrees already and therefore already know how to learn. While they certainly have a right to defend their technological baby, what they’re really saying in the comments to that post is, “All I care about is how this new technology affects me.”
Me? I’m worried abut everybody else. The professors who might potentially get displaced by MOOCs certainly includes me, but I’m also concerned about the students who don’t know how to learn yet and might end up left with no other option but MOOCs in our shiny online educational future. I’m even concerned about the poor superprofessors who had no idea what they were signing up for when edX or Coursera or whomever offered them the opportunity to become famous regardless of their actual merit as online teachers. While some students who had nothing before MOOCs may be happy about them, I would argue that the rest of these groups are mostly bored by the experience. Luckily, the evidence has begun to appear for me to actually prove this point.
I. “A spoonful of sugar helps the medicine go down.”
Longtime readers know that I’ve continually made the distinction between MOOC producers and MOOC consumers. Harvard and MIT produce MOOCs. The University of Maryland system has recently begun consuming them, integrating MOOC content into a wide range of content across many campuses. Ithaka released a Gates Foundation-funded study on these test runs last week.
As you might imagine given the funding source, the results are generally positive. All but one of the of the instructors said that they would use MOOCs inside their own classes again if given the opportunity. However, there are plenty of warning signs even in this positive study. For example, from p. 18:
Finding and adapting online content to use for a hybrid course posed the greatest challenge for faculty partners. MOOCs illustrate the priorities of their creators, and these are not necessarily the same priorities that other faculty have for their individual students. Moreover, academic departments develop degree program curricula as a whole, making deliberate decisions about when and where certain content should be taught and competencies assessed within specific courses. To integrate a MOOC into an existing class is not necessarily a simple case of choosing what pieces to include or exclude. Even with online course materials that are a fairly close fit with the pedagogical approach of the instructor and needs of the students, the local instructor may need to re-conceptualize or restructure his or her existing course to fit with the online content.
If a MOOC-consuming professor has to fit a square peg into a round hole, which is going to change first: the MOOC or their class? Since the consuming professor can’t go back and change the superprofessor’s work, I think you know the answer to this question.
So why would anybody voluntarily subject themselves to that kind of de-skilling? Of course, there are incentives for professors who decide to farm out an important part of their job to superprofessors. This is from p. 26 of the study:
Based on the data we collected, the clearest potential for time savings appears to be in the amount of time instructors spent delivering courses. Those in the treatment sections spent just over half as much time in class as those teaching the control sections. In one large course designed by a single course coordinator, instructors in the MOOC-based sections told us they saved considerable time because they did not need to prepare content to teach and they spent less time in class. Once the initial investment is made in preparing the course, instructors may be able to spend less time planning what to teach and actually coming to class.
On the one hand, I’d certainly like to spend less time preparing and coming to class, but I also understand that what I bring to that process is part of the reason I get paid a living wage. Not being burnt out (at least not yet), finding interesting ways to deliver the content I spent seven years in graduate school studying (and more time since refreshing) is one of the most interesting aspects of my job.
More importantly, I understand these MOOCish developments as part of a broader effort to destroy faculty prerogatives across the board. Most faculty understand this. That’s why the executive summary of the study notes:
It was clear that administrative leadership was essential to stimulate faculty interest.
This leadership will, of course, include professional and monetary incentives. A spoonful of sugar helps the medicine go down, the medicine go down, medicine go down…
Don’t do it, kiddies!It’s a trap! Once you’re finally done cleaning up your room, Mary Poppins is going to kick you out on the street, and find a new set of less-whiny children who are less likely to make any demands upon her at all.
II. Pity the poor superprofessor.
Yeah, I’m going to hit this whole #massivelearning thing onemore time. I hadn’t planned to, but this post by Aposotolos K. includes the message that Coursera sent Paul-Oliver Dehaye’s students and it’s just too disturbing for me to ignore. Here’s a quote from AK’s excerpt:
Unfortunately, Prof. Dehaye had not previously informed Coursera of this part of his pedagocial approach: Deleting course material is not compatible with Coursera’s course concept, where students all over the globe decide when they want to watch a particular course video. Prof. Dehaye’s course included experimental teaching aspects which led to further confusion among students.
Coursera and the University of Zurich decided on Friday, July 3rd, to reinstall the course’s full content and paused editing privileges of the instructor until final clarification on the issue would be obtained.
[Emphasis added]
When I wrote a post entitled “Dear Superprofessors: Your MOOC Isn’t Yours,” I was only arguing that this kind of thing could happen in theory. Now we’ve seen that it has happened in fact. A superprofessor has been locked out of his own course. Yes, I know what you’re thinking: “Professor Dehaye had no business teaching a MOOC in the first place.” That may very well be true (in fact, I kind of argued that point here), but think about this in the abstract for a moment.
According to Coursera, Dehaye wanted to try something really innovative and Coursera shot him down. Think how boring life would be if you had to get a bunch of bureaucrats to approve every innovative teaching technique that you wanted to try. It would be like living in a corporate university with a pencil-pusher stationed right there in your classroom. Yet this is precisely what Coursera’s model appears to be. Here’s AK again, this time in his own words:
Coursera has a “concept” of what MOOC teaching and learning looks like, and they are packaging it with their LMS. I guess the only sanctioned way to design and teach on Coursera is the Coursera way. This to me is quite problematic from a pedagogical stance; and I am sure others have written about this before and will continue to write about it.
If you consider Coursera to be just a platform, and a platform to be neutral (both problematic assumptions, if you ask me), then they shouldn’t care how Dehaye ran that MOOC. But Coursera, it seems, has a “course concept,” which suggests a certain structural uniformity across all of that company’s product. Break out of that norm and they can end your career as a superprofessor faster than you can say “self-paced learning environment.”
While superprofessors are the public face of the MOOC concept, the #massivelearning debacle demonstrates that the people really holding the power in MOOC world are the clerks. What else did you expect when higher education gets treated as if it were nothing but a series of isolated, standardized content factories? It wouldn’t surprise me if Coursera started to time whoever ends up doing the dirty work that superprofessors abandon with stopwatches to make sure that they’re grading to the company’s universal standard of educational productivity.
III. Happy talk.
I’m not sure I would have picked up Matthew Crawford’s Shop Class as Soulcraft again if it weren’t for my recent obsession with American Pickers. Among the many kinds of antiques that Mike and Frank are obsessed with are motorcycles, and Crawford uses his motorcycle repair business as a metaphor for work in general. I’m going to use my growing interest in them as a metaphor for teaching.
Until recently, I couldn’t care less about motorcycles – even the old ones. What the guys from American Pickers have shown me is the link between the old motorcycle industry, the bicycle industry and even the automobile industry. While I couldn’t have cared less about bicycles either and only cared a little about old cars, the abstract connections they keep making continuously reminds me of my favorite history book of all time: From the American System to Mass Production, by David Hounshell. Now just because I called this my favorite history book of all time, I’m not recommending that you all go out and pick up a copy. It’s a slog. Yet in the hands of the right instructor, it can be a major eye-opener.
A few semesters ago, I taught a graduate course devoted entirely to industrialization. I assigned Hounshell, Bill Cronon’s Nature’s Metropolis (against the explicit wishes of my department chair who had told me that it was unteachable),Richard White’s Railroaded, David G. Schuster’s vastly under-appreciated masterpiece Neurasthenic Nation and a few more such studies. While there may have been a couple of students who wanted to kill me by the end, the majority of the people sitting around that table told me how much they enjoyed the class even though they hadn’t expected to like it at all. My guess is that the difference here was my enthusiasm.
When I’m teaching a subject that I find interesting, my eyes light up and my normally-rapid speech naturally slows as I try to explain in great detail why something that I think is cool is, in fact, cool. I’ve actually taught Hounshell to undergraduates [Twice!], but I didn’t say go read Hounshell and come back with a five-page report in two weeks. I explained it as we talked about the text with the book open, adapting my spiel to the kinds of inevitable questions that I faced from the class. The same way the American Pickers got me excited about motorcycles, I think I managed to get a lot of students excited about assembly lines.
Certainly, the people who leave me hostile comments on that post have that kind of enthusiasm for their MOOC content. However, I’d argue that this kind of teaching will only work in a MOOC setting on the kinds of students who are excited about the material already. The result for most students will be indifference and boredom. We can actually see this dynamic at work in that Ithaka study. This is from the Executive Summary again:
Despite the similar student outcomes produced by the two course formats, students in the hybrid sections reported considerably lower satisfaction with their experience. Many indicated that they would prefer to have more face-to-face time with instructors.
And that’s for MOOCs embedded inside regular face-to-face courses. Imagine how hostile most students would be if the MOOC is all they had. Actually, that’s easy to imagine as this would explain the 90% average dropout rate that most MOOCs have. Sure, a cMOOC would probably have a better track record with student satisfaction, but that’s not nearly as efficient as the passive xMOOCs that the Gates Foundation wants to test. The technology of austerity is not interactive because interactive costs time and money. This won’t work if you’re only measuring educational “efficiency.”
In summary, the only people who are happy by this kind of result are lifelong learners with no skin in the game and the clerks. Is it really worth disrupting everybody’s higher education to make just these two groups happy?
Thanks to the great Sarah Kendzior, I picked up a copy of Gone Girl when I was in Target yesterday. While I’ve barely started it, I can tell it’s going to be really, really good. What I think made me do it was that until I read Sarah’s piece on the death of malls, I didn’t realize that the book had economic themes, like this quote from the beginning few pages:
“I’d arrived in New York in the late ’90s, the last gasp of the of the glory days, although no one knew it then. New York was packed with writers, real writers, because there were magazines, real magazines, loads of them. This was back when the Internet was still some exotic pet kept in the corner of the publishing world–throw some kibble at it, watch it dance on its little leash, oh quite cute, it definitely won’t kill us in the night. Think about it: a time when newly graduated college kids could come to New York and get paid to write. We had no clue that we were embarking on careers that would vanish within a decade.”
I love good magazine journalism. I have an almost religious devotion to the New Yorker since my parents were subscribers. Yet I also read the Huffington Post and give content away for free at this very blog. Does that make me a bad person?
Of course, it’s easy to imagine some bitter magazine writer muttering to themselves, “You’re next, professor.” That, at least, is what the disrupt higher education crowd would like everybody to believe. Longtime readers know that I constantly go back on forth on the question of whether or not my profession will “vanish within a decade.” Today, I’m definitely in “I will survive mode.”
These days, two out of three students attending on-campus programs receive some form of generous subsidy or discount, while their online counterparts, generally ineligible for such assistance, foot the full sticker price even though they do not benefit from all the amenities of the revered campus life, do not take up parking spaces, inflict wear and tear on facilities, or take up as much instructor time. Instead of embracing these online learners who produce considerable incremental revenue for institutions, colleges and universities are penalizing them, which has troubling implications not only for students’ bank accounts, but also for universities’ own vaunted views of fairness. By introducing e-tuition, which is appropriately lower than the on-campus price tag, universities could easily capitalize on the scale, brand extension, and new revenue synonymous with online learning while maintaining far more equitable pricing for online students.
Never. Gonna. Happen. How do I know? The entire existence of online courses is predicated on the notion that they’re just as good as the courses on campus. Because offering online courses cheaper than face-to-face courses strongly suggests that these courses are somehow inferior to the courses that are offered on campus, universities will do nothing to discredit their brands.* Besides that, many of them have undoubtedly already spent the money they expect to take in through online courses on new administrators and climbing walls in the gym.
Best’s example of a university that’s bucking this trend is the online computer science MA at Georgia Tech. If you remember though, this is the MOOC-ish MA in which people spend most of their time watching video, with minimal outside help. They’re only discounting the sticker price because they’re hoping to make it up in volume and, as Chris Newfield has arguedpersuasively, their revenue estimates are probably inflated. That would make this an exception that proves the world.
The assumption that college is too expensive is certainly correct. The problem with this article is that it assumes that online education is the way to solve that problem. As I’ve noted before, it is possible to do some really interesting things with education online. However, if you’re just doing it to save money so that your university can keep more money for other things, your online courses are going to be awful. As a result, nobody will learn anything and they’ll all end up unemployed. The negative feedback loop will then lead to a real crisis in higher education, brought on by the people who thought they were saving it.
Maybe this time I’ll stop changing my mind. The future is a lot closer than it was when I first wrote that post, and the evidence of discounting still hasn’t surfaced. What’s going to make them change their minds?
* Of course, the good online courses aren’t inferior to the ones offered on campus, but if you discount them the market will still likely treat them that way.
I never took a course in the history of technology. My dissertation (and very poorly read first book) were about labor relations in the American steel industry. While overdosing on industry trade journals, I quickly realized that how steelworkers labored depended upon how steel was made and that the best way to distinguish what I was writing from the many studies that had come before was to get the technological details right.
This proved to be a terrible strategy. While I’m quite sure that I did indeed get the technological details right, the people who read my manuscript never recognized this since they had all read or written books that got them wrong or never covered them at all. The worst comment I ever got (which, of course, I remember to this day) was “Rees knows nothing about the technology of the steel industry.” I begged to differ, but what could I do about it? Nothing.
I wrote Refrigeration Nation because I enjoyed reading old trade journals to get the details right and because I wanted to examine the technology of an industry that nobody else had written about. Surprisingly, when I picked my second book project that description included the refrigeration industry. Actually, refrigeration is not one technology, but many: ice harvesting equipment, large scale industrial refrigerating machines, electric household refrigerators and others. If you read the book (and I certainly hope you do), you’ll see I spill the most ink writing about the transitions between one technology and another.
These transitions can be painfully slow. Ice harvesting didn’t die until around World War I. The ice man still delivered machine-made ice door-to-door in New York City during the 1950s. Even today, you can still buy what is generally known as “artisan ice” for people who really want their drinks to be special. Perhaps this explains why I’ve always been so suspicious of Clayton Christensen’s theory of “disruptive innovation.” Everything I’ve ever studied that you’d expect to disappear in the blink of an eye when in competition with better technology always managed to hold on for decades.
By now, you’ve probably already read Jill Lepore’s absolutely devastating takedown of disruptive innovation in what I presume is this week’s New Yorker. [It appears rather late in my neck of Colorado. Thank goodness this one is outside the paywall!] If you still haven’t let’s just say that Lepore is unimpressed by the work of her Harvard colleague:
Disruptive innovation as a theory of change is meant to serve both as a chronicle of the past (this has happened) and as a model for the future (it will keep happening). The strength of a prediction made from a model depends on the quality of the historical evidence and on the reliability of the methods used to gather and interpret it. Historical analysis proceeds from certain conditions regarding proof. None of these conditions have been met.
And remember, there’s plenty of excellent evidence for the pace of technological change in countless American industries. You’ve never read an Alfred Chandler takedown because Chandler actually consulted this stuff. Christensen apparently not so much.
Since I don’t have a team of fact checkers at my disposal, I’m just going to concentrate here on the industry Lepore covers that I know best: steel. Here’s Lepore:
In his discussion of the steel industry, in which he argues that established companies were disrupted by the technology of minimilling (melting down scrap metal to make cheaper, lower-quality sheet metal), Christensen writes that U.S. Steel, founded in 1901, lowered the cost of steel production from “nine labor-hours per ton of steel produced in 1980 to just under three hours per ton in 1991,” which he attributes to the company’s “ferociously attacking the size of its workforce, paring it from more than 93,000 in 1980 to fewer than 23,000 in 1991,” in order to point out that even this accomplishment could not stop the coming disruption. Christensen tends to ignore factors that don’t support his theory. Factors having effects on both production and profitability that Christensen does not mention are that, between 1986 and 1987, twenty-two thousand workers at U.S. Steel did not go to work, as part of a labor action, and that U.S. Steel’s workers are unionized and have been for generations, while minimill manufacturers, with their newer workforces, are generally non-union. Christensen’s logic here seems to be that the industry’s labor arrangements can have played no role in U.S. Steel’s struggles—and are not even worth mentioning—because U.S. Steel’s struggles must be a function of its having failed to build minimills. U.S. Steel’s struggles have been and remain grave, but its failure is by no means a matter of historical record. Today, the largest U.S. producer of steel is—U.S. Steel.
Two other factors that Lepore doesn’t mention (which makes me think that Christensen didn’t either) are environmental regulation and foreign competition – the second being the more important of those two to the overall fate of the industry. The success of minimills also required a huge decrease in the price of scrap steel. What these other factors suggest is that any hard and fast rule of technological change will inevitably fall victim to the unpredictability of people. My old advisor used to call this the social system of production, and practically the entire subfield of the history of technology is predicated on this notion rather than Christensen’s brand of technological determinism
For example, if I remember right, Chandler’s last book (I get the titles mixed up) is about the various quirks in the path of industrialization across international borders. In my work, the most important factor determining the speed at which one refrigerating technology transitions to another is its reception by consumers and amazingly enough lots of refrigeration consumers just hate “progress.” Just to namecheck a great book that I happen to be reading right now, in Seeing Underground, Eric Nystrom describes the effect of political factors – especially lawsuits – on the quality of mine maps. In Butte, Montana, at least, the more lawsuits there were the more precious metals they eventually found.
Of course, my interest in Christensen comes from his pronouncements about higher education. Lepore does very little with them in her article, but that shouldn’t stop anyone from applying the same logic that I just did here. There is no scientific law of the jungle that fates universities to go entirely online or die off. If people value direct human contact and the educational advantages it brings, they should be willing to pay – or force their governments to pay – for universities to teach in face-to-face settings. Like I wrote in Inside Higher Education a really long time ago now, all this talk about inevitability is just a way to shut down discussion so that the educational traits that we once valued will be abandoned more easily.
The great service that Lepore has performed is to metaphorically take the fight over those values to the source of the attacks against them. Like MacArthur at Inchon, she has landed behind enemy lines and will hopefully force the enemy to pull back and defend ideological territory that they thought they had already conquered. Those of us currently at risk of becoming victims of creative destruction can only hope she succeeds.
The advantage of having these eclectic interests is that it makes it possible for me to draw some connections that other people might miss. So let me begin by briefly summarizing two rather amazing articles that I read this morning and then trying to pull them together. First, the distinguished Atrios guest poster alumni and Corrente blogger Lambert Strether published a higher education post over on Naked Capitalism that really is quite epic. I’m not sure there’s all that much here that I didn’t know already, but it is certainly very helpful to see it in one place.
He begins with a discussion of the actual privatizing of public universities, citing this Bloomberg piece:
After gaining greater independence, many public universities have increased tuition, raising fears that West Chester would follow suit.
“For any university that leaves the state system, tuition and fees will likely go up — creating an added burden for students and their families,” Frank Brogan, chancellor of the Pennsylvania State System of Higher Education, said in a statement opposing the bill when it was introduced.
The independence drive is analogous to the rise in K-12 education of charter schools… Like charters, breakaway universities want less red tape and more freedom to experiment with academic programs.
I had actually heard that the University of Alabama (of all places) actually increased faculty salaries after doing something similar, but anybody who puts their faith in the majority of college administrators (or even a significant minority) to do the same thing is deluding themselves. As Lambert goes on to point out, the usual effect of more revenue at a corporate university is for it to become even more corporate, despite the fact that their students still need to depend upon public assistance through student loans in order to attend there at all. Welfare is to Walmart as student loans are to the corporate university, especially the for-profit corporate university, but even the ones that you’d have thought were better than that too.
Skipping a lot (please do read the whole thing), Lambert concludes with an analogy to Naomi Klein’s Shock Doctrine:
It’s almost like there’s a neo-liberal playbook, isn’t there? No underpants gnomes, they! Defund, claim crisis, call for privatization… Profit! [ka-ching]. Congress underfunds the VA, then overloads it with Section 8 patients, a crisis occurs, and Obama’s first response is send patients to the private system. Congress imposes huge unheard-of, pension requirements on the Post Office, such that it operates at a loss, and it’s gradually cannibalized by private entities, whether for services or property. And charters are justified by a similar process.
Having read that book more than once now, what I want to point out here is that this Shock Doctrine-style privatization actually began a long time ago. No, you don’t have to get a charter to operate as if you were corporate, all you have to do is outsource large sections of your core mission to private companies, just like the U.S. government did in Iraq.
Consider, for example, education technology. This is where the second great article I read this morning comes into the picture. While I normally wouldn’t be caught dead reading the Educause Review (since David Noble called them out as corporate stooges about fifteen years ago), they might actually be getting better as they published this remarkable article by Jim Groom and Brian Lamb called “Reclaiming Innovation,” [h/t David Kernohan].
As you might imagine, my favorite part is where they go after the Learning Management System. For purposes of this post, the key argument of their five-point LMS condemnation is #4:
The expense of enterprise LMSs is an inexhaustible drain on institutional resources. Even when they are operating at optimal efficiency, the commitment required to maintain them represents an immense set of challenges. And any technologists who have been involved in a migration from one system to another, or in significant upgrades of the same system, can testify to how time-consuming and troublesome these processes will be.
In other words, all the techies you’re hiring to keep the thing operating could be going to keeping tuition low. More importantly for my audience here, all that money going to Blackboard could be going to raising faculty salaries or even just giving adjuncts a living wage. And the really insane thing is that none of these G.D. things make anybody (students or faculty) who uses them particularly happy!!! They are solutions that solve nothing. In fact, what they mostly do is create new problems.
Read the whole thing to see Groom and Lamb’s elegant solutions, but what I want to point out here to the education technology-inclined is that no matter how convincing you happen to be in your advocacy for open source anything, you’re still going to have to overcome the neo-liberal mindset that Lambert describes so well. You want to design something truly innovative and all they offer you is an electronic beast of burden. Worse yet, their ultimate neo-liberal wish is to use that beast of burden to put you out on the streets – to put you out, put you out, put you out of misery. Like the coal companies in Colorado one hundred years ago, they’re more interested in mules than people and that goes for students as well as faculty.
You may have noticed my general failure to avoid discussing MOOCs lately. “Just when I thought I was out, they pull me back in.” Actually, that’s not an entirely accurate assessment. There’s my latest for Chronicle Vitae, which is entirely MOOC-free. And sometimes instead of writing exclusively about MOOCs these days, I find myself writing about things that are MOOC-ish (MOCs, POCs, XOCs, etc.) or, like that Academe article of mine, I write about MOOCs in a wider context of technological threats to faculty prerogatives.
The more I think about it, the more I think that this last subject is where the real battle for the future of higher education will occur. While Coursera might love to stuff MOOCs down our throats, administrators of ill will are much more likely to use a wide range of technological tools to change higher education for the worse by making most faculty irrelevant. After all, the vast majority of us are too busy or too old school to follow every little twist and turn in education technology. That’s why it should be easy to slip something by us.
Which is why I’m making this announcement: I am no longer anti-MOOC (and not just because I like DS106). Anti-MOOC is so 2013. I am now anti- “the misuse of technology to destroy higher education by usurping faculty prerogatives.” Of course, that INCLUDES the vast majority of MOOCs, but really the threat we face is so much bigger than MOOCs and their ilk.
In order to spread the word about what’s going on, I’ve decided to get my act together and take it on the road. Yes, I’ve just started working up a presentation for interested faculty everywhere (and am teaching myself Keynote in order to do it) which I’m tentatively calling, “Educational Technology, Budgetary Priorities and Academic Freedom.” Anybody interested in booking me to present this analysis for their event need only contact me at the e-mail address here on the right.
Does this mean I’m selling out? The answer to that question is, “Sort of.” If you happen to have money to pay for my services, I will accept it. However, if you are an impoverished faculty group (and of course I know the vast majority of faculty groups are very impoverished), I’ll go anywhere and speak just for expenses, just like all the speakers I know through AAUP do all the time.
PS If you need a reference, contact the nice people at the Connecticut AAUP. I had more fun speaking there last year than I ever thought possible, and all they gave me was a personalized poster (which I will treasure for the rest of my life or until I get replaced by a robot, whichever comes first).
“This conjunction of an immense military establishment and a large arms industry is new in the American experience. The total influence — economic, political, even spiritual — is felt in every city, every State house, every office of the Federal government. We recognize the imperative need for this development. Yet we must not fail to comprehend its grave implications. Our toil, resources and livelihood are all involved; so is the very structure of our society.”
“A Course Manager is a teacher, mentor, and technical reviewer in one. You should take pride in ensuring that your students receive the best possible learning experience by motivating and working with them 1-on-1, mentoring them as they develop their portfolio, reviewing course materials, and giving insightful feedback to the Course Development Team.”
“Where I come from,” I tweeted, “they call course managers “professors.” But when I went back and looked at the ad again I noticed that no specific content knowledge is required to be a Udacity course manager at all.
Does anybody else see a problem with this?
Udacity, Post-Pivot:
If you’ve been paying attention to MOOCs for some length of time, you undoubtedly remember the infamous Sebastian Thrun Fast Company interview in which he basically called all of his company’s courses crap. It even gave birth to its own hashtag, #thrunpivot. In this new interview, he doubles down on that proposition:
The MOOC that we created at Udacity was our first attempt to democratize education and we learned from it. Like everyone, we made mistakes. We learned we can drastically boost learning outcomes by adding a service layer around MOOCs. It has a huge impact on completion rates and learning outcomes. Many people in the industry would say, ‘We told you so.’
What, pray tell, is a “service layer?” Living breathing human beings who will help guide students through the corridors of knowledge:
At the very beginning you do a Google Hangout and someone from Udacity talks to you. It’s our internal fleet of mentors [who provide coaching through the class]. When we make a class, we have a very different model from a classic MOOC. The team trains mentors specifically for the one class.
I was so troubled by our [former] completion rates. When I called a MOOC a lousy product I wasn’t kidding. [With this new model] we have literally gotten a [course] completion rate of 60 percent.
It would be interesting to know what the difference is between a course manager and a mentor. I’m guessing the course manager serves as the mentors’ boss. Yet some of those course managers are actually part time. Either way, if “the team trains specifically for one class,” who teaches the team? Certainly it can’t be the superprofessor, right? They’re too busy preparing the lectures and otherwise serving humanity. Do they just watch the same videos that everybody else does before they get released to the class?
No matter what, this whole set up is most decidedly not automated education. It’s cheap. It’s online. But it’s not automated. People who need to be trained require money for their labor and the source of that money has to be the students. That’s why Thrun says:
If you’re affluent, we can do a much better job with you, we can make magic happen.
Pardon me while I go vomit.
The New Profitable Non-Profit Model:
While it would be really interesting to contrast Thrun’s new model with a new Boston Globe article in which Clayton Christensen (and a co-author) restate his now very old ideas, I’d rather compare it to another article you may have seen, this one about the University of Southern New Hampshire because I think there’s very little daylight between this and the new Udacity. And Jesus, if this story doesn’t give the average college professor the chills, I don’t know what will:
Delilah Caldwell, a philosophy instructor at Southern New Hampshire University, may well represent the future of higher education’s teaching force.
As one of the first full-time faculty members at Southern New Hampshire’s online college, Ms. Caldwell taught 20 online courses last year: four at a time for five terms, each eight weeks long. The textbooks and syllabi were provided by the university; Ms. Caldwell’s job was to teach. She was told to grade and give feedback on all student work in 72 hours or less.
Second of all, the 72 hours or less is my favorite part. Suppose you actually want to have a life AND write half-decent comments on your students’ papers. What do you do then? Stupid me, who’s actually going to be dumb enough to assign papers if they’re facing a 72 hour turnaround time on all student-submitted work?
Yes, the academic assembly line workers at the University of Southern New Hampshire get paid relatively well (compared to adjuncts), but I bet the course managers at Udacity do too. The problems here go well beyond that in both cases. No academic freedom. No research. Very little control over your own class. [In the case of MOOCs that goes for both the mentors and to some extent the superprofessor too.] These things aren’t just important to the faculty involved. They’re vitally important to the quality of the course. Happy, knowledgable teachers teach better than peons on an academic assembly line.
Technological enthusiasts may be asking me right now, “Where’s your study on this?” I don’t have one. Neither do the MOOC people. As a recent study I saw via George Siemens has suggested:
To date, there has been little evidence collected that would allow an assessment of whether MOOCs do indeed provide a cost-effective mechanism for producing desirable educational outcomes at scale.
Gee, you’d think somebody would actually bother to study that BEFORE they decided to disrupt higher education. So why do such courses exist then? Are faculty so desperate to be superprofessors that they’re willing to act now and ask questions later? Are students simply pining for them? What if disrupting higher education isn’t such a hot idea after all? I think the reason that both these online facsimiles of real college college courses exist is, to paraphrase Eisenhower, the MOOC/online education – administrative industrial complex.
“Our toil, resources and livelihood are all involved”:
The beauty of the Military-Industrial Complex (if such a word is even appropriate for use with such an awful thing) was that all that defense spending offered the economic benefits of being on a permanent wartime footing, but only occasionally did anybody have to go out and kill anybody. After the Manhattan Project, all those incredibly expensive nuclear weapons were never used at all.
What made Eisenhower’s warning about the Military-Industrial Complex so powerful was that here was a guy who knew. The great regret of his presidency was that he hadn’t done enough to stop this reckless spending, and his warning was supposed to help prevent that spending from continuing too far into the future. But the forces that stood to gain the most from that spending, generals and defense contractors, thrust it upon a willing America anyway because it was in their interests, if not the interests of America at large.
I’m beginning to think that administrators and edtech providers of all stripes, MOOCs or otherwise, have an evil tacit bargain all their own. Move college online, the deal goes, not because it will do anything in particular for education, but because it will help backfill all the government funding you’ve been losing over the last few decades. As an added benefit, it will certainly help you cut labor costs as your formally highly-paid, influential teachers can be replaced by an online army of the under-employed, or worse yet, robots.
Meanwhile this revolution is being sold to students for reasons of cost and convenience. As an added benefit, administrators and their private sector clients can make it seem as if such courses will help make college more effective at solving the structural inequalities inherent in our modern economy than they really are. Here’s Christensen and co-author on precisely that point:
Education technology companies and alternative learning providers — not just MOOCs — are finding disruptive footholds by targeting these non-consumers. They note that graduates from even well regarded colleges are struggling to launch their careers, make it into the workforce, or transition between jobs. Innovators are, therefore, beginning to address this widening gap by identifying what employers need and building those skill sets into their curricula.
Why not just sell the college to Subway so that they can turn it into a sandwich university and just get it over with?
Yet if the MOOC you’re taking sucks, why would it make a difference if you’re learning the exact skills that employers want or not? After all, you wouldn’t be learning them particularly well. Even if the MOOC you’re taking didn’t suck, the fact that so many people can learn those skills at the same time will only drive down the wages that graduates would earn for having them. If those skills are best practiced online, our students would then be facing the same kind of job market that new Ph.D.s are, and that’s not good news for anybody.
In short, why would anybody pay to have “magic happen” if they’re never going to get a chance to make a decent living using the skills they learn? If there aren’t any journalists willing to ask Thrun that question, maybe his investors should.
While I was making my way home from Atlanta on Sunday, a whole bunch of my virtual and actualfriends were still at the Organization of American Historians annual meeting discussing whether blogging is scholarship. While I’m sorely tempted to weigh in on this question myself, I think I’d rather follow Mike O’Malley’s example and consider exactly what scholarship is. Or to put it a slightly different way, what and who is scholarship for? Or maybe just why scholarship?
What’s sent me down this path before I even saw O’Malley’s post is this rather amazing article from Smithsonian (which I found via Rebecca Schuman, who’s probably still laughing her ass off about this days after she first read it):
“There are a lot of scientific papers out there. One estimate puts the count at 1.8 million articles published each year, in about 28,000 journals. Who actually reads those papers? According to one 2007 study, not many people: half of academic papers are read only by their authors and journal editors, the study’s authors write.
But not all academics accept that they have an audience of three. There’s a heated dispute around academic readership and citation—enough that there have been studies about reading studies going back for more than two decades.
In the 2007 study, the authors introduce their topic by noting that “as many as 50% of papers are never read by anyone other than their authors, referees and journal editors.” They also claim that 90 percent of papers published are never cited.”
Of course, the flies in the ointment of this discussion are tenure and promotion standards. Early-career scholars with blogs want blogging to be scholarship because that will make tenure easier to attain. I know that sounds bad, but really what’s the use of running the normal academic peer review gauntlet if it’s likely that only three people will read the result?
Coincidentally, this discussion and this article happened at the same time that I have to worry about precisely this sort of thing once again. Yes, I’m a tenured full professor, but as anybody among the somewhat more than three people who read this blog regularly know our administration here at CSU-Pueblo is trying very hard to move the vast majority of professors at this institution from a 3-3 (or 9 credit) to a 4-4 (or 12 credit) teaching load. While I was once optimistic that there would be enough exceptions to that standard that most active scholars on campus would be able to avoid it and continue their research apace, I am not anymore.
Here’s why: A few weeks ago, our Provost published his new research standards at the back of a grant application form for a single semester of release time. To my knowledge, he did not consult our faculty senate or any faculty members whatsoever before doing so. Here is a selection from that document (no link because it was e-mail only, e-mail attachment only to be exact):
“At CSU-Pueblo, faculty are expected to teach 12 credit hours per semester (and engage in research/scholarly/creative activity, and perform service). I emphasize that regular scholarly activity is expected of faculty who teach a 12 cr hr teaching load per semester. Awarding equivalency time to conduct research/scholarly/creative activity, above and beyond the usual expectations that we have of faculty, requires careful justification – even moreso at a public institution, in an environment with significantly constrained resources.”
Here’s what it says about release time for scholarly activity in our faculty handbook:
“After consultation with the faculty and Chair of a department, the Dean shall recommend to the Provost all requests for release from teaching. Faculty members released from teaching assignments shall devote a minimum of three (3) clock hours per week for each semester hour of released time to tasks associated with such release….Release from teaching to engage in sponsored research, University supported scholarly or creative activity, University service or other approved activities may be authorized by the Provost dependent upon the availability of funds and program needs.”
In other words, we’re going from an environment in which the vast majority of faculty members received that one course release to an environment in which we all have to prove that we’re not ripping off the taxpayers of Colorado and we still might not get that course release anyway. Furthermore, there’s been no hint that the standards on our annual performance reviews will be amended at all to reflect this rather significant change in policy.
While I’m fortunate enough to have no need to submit this blog as proof of scholarship, other faculty members on campus might not be quite as productive as I’ve been lately. Here’s the gauntlet that we all have to run to get one of 20 or so release time “fellowships” to pay for our adjunct replacements (as described in that policy statement I referenced above):
“The Provost will not approve equivalency time for research/scholarly/creative activity for Fall 2014-Spring 2015 if there is not a demonstrable peer-reviewed work product within the previous 2 or 3 years, depending upon the amount of equivalency time requested.”
It so happens that I approve of the peer review process. In most cases it has significantly improved the work that I’ve published, but as anybody with actual experience in peer review knows this slows things down to an unimaginable degree. For example, I wrote on article to mark the centennial of the Ludlow Massacre for Labor during my sabbatical a year and a half ago in order to make the anniversary itself, which is this very week. It’s accepted, but won’t be published until the fall, months after the anniversary is over.
Will more than three people read that article? Labor is a very good journal so I think so. However, even before I read that Smithsonian article I had become increasingly convinced that most academic journals are utterly useless. The value of blogging (or God forbid practicing actual journalism) is that you’re almost instantly guaranteed a much wider audience than publication in even the most respected academic journals will ever give you. Shouldn’t the point of scholarship be to influence the way the world works? If so, how can anybody justify a narrow fixation on peer review if almost nobody reads the results?
What troubles me most, however, is my administration’s demand for a “demonstrable peer-reviewed work product” within a two to three year window. My last book took me (on and off) thirteen years. Nevertheless, I still want to write more books. Not only that, I want to write more books that people will actually read. I’m currently close to being under contract to write two more comparatively quick refrigeration related books using my surplus research. Both will be peer-reviewed (or at least extensively peer-edited). After that, however, my Harvey Wiley biography is going to take a huge amount of time for me to finish because his papers are all back East and that extra class I’ll be teaching starting this fall isn’t going to speed that process up any.
As you might imagine, this whole situation makes me incredibly sad. If the only solution to this problem is to write short, crappy, purely academic work that reads like the instructions for the Holy Hand Grenade of Antioch and only three people ever read it, I don’t know if I want to play this game anymore.
What’s old is new again in edtech land. While that’s always been true to some extent, what’s new now is that this constant effort at reinvention has begun to take MOOCs as the status quo to be contrasted against rather than what they once were, namely the bright, shiny new thing that will save us all.
You say you want evidence for this trend? The folks who produce one particular online program tweeted this at me last week, presumably because the author used my Slate article as a jumping off point:
As a social phenomenon, access to education in this way – that is available for everyone, for free – is unprecedented and changing the way we live, work and learn. No one wants to move away from that or undo the huge steps forward we have made. But, as we have seen, it is not a perfect system. Something needs to change to utilise this power to its best advantage, to take what we have learned and move it a step further. Students need interaction with their teachers and fellow students. They need support. What we have seen so far is that MOOCs fail to address the need for communication as a learning tool.
Their solution? “[A] combination of online learning and personal interaction.” Don’t get me wrong: That’s certainly an improvement over MOOCs, but something like that’s been available for about twenty years now. They’re called online classes. You know…the non-massive ones. Certainly online classes are not all the same, particularly since the more student/teacher interaction they foster the better. However, to claim that personal attention is somehow an exclusive selling point of this one provider requires a rather selective reading of edtech history.
Nevertheless, others have actually invented their own new acronym for doing what some people have been doing for ages now. My much-valued commenter and online friend Contingent Cassandra sent me this link:
Small Private Online Courses (SPOCs) on the other hand, are purposely focusing on class size as a sort of opposite of the Massive Open Online Course (MOOC). A University Business article emphasizes that this isn’t a new model, but one that may be finding a broader audience as school and corporate partners offer specialized curricula to small groups of (17-20) learners. These numbers mean that the kinds of support often missing in MOOCs and other large classes – such as personalized feedback and coaching, and opportunities for real-world experience – are more readily available.
That sound you hear is a whole slew of dedicated online instructors hitting their heads against their desks over and over again. Certainly offering online students this kind of personal attention beats what they’d get in MOOCs, but when you get right down to it that’s not a very high standard, is it? The other important question is how long can these Small Private Online Courses can stay small. When will the profit motive that even public universities now express regularly get the best of any instructor’s best intentions?
Leaving the substantial minority of people who do really innovative teaching online aside, the question then becomes how should we judge online education as a whole. What does online education get right that we can’t do in face-to-face classes? What does it get wrong? More importantly, why does it get what it gets wrong wrong? New UC Chancellor Janet Napalitano (of all people) may have hit the nail on the head here:
“There’s a developing consensus that online learning is a tool for the toolbox, but it’s harder than it looks and if you do it right, it doesn’t save all that much money,” Napolitano told about 500 policy and education experts at a speaker series sponsored by the Public Policy Institute of California….
Online courses may indeed prove to be useful, she said, but more as a way to augment upper-division work for students who are already deeply engaged in their subject matter.
Let the people who have already learned how to learn learn online. Give all the students who don’t know how to learn yet the attention they deserve. More importantly, let them get all the attention that they can get in a classroom setting before you give them the option of entering the brave old world of online education. When online education at all levels of instruction becomes the only option for the vast majority of students, higher education will have failed us all.
Reinventing the wheel here is hardly a pedagogical imperative. It’s not even a financial imperative, since (as Napolitano points out) online education doesn’t really save universities all that much money. Just because you can teach students online doesn’t mean you should teach students online, especially in massive open online courses that offer no individual attention at all unless students win a lottery or beg for it.
When all is said and done it’s not the teacher/student relationship that’s broken. What’s broken is the political economy of higher education that has convinced some people to consider even the worst forms of online education an imperative in the first place.
Here’s a little-known fact about me: I used to be a Walmart blogger. That blog is still updated occasionally by my friend Jeff in Cleveland, but a few years back I came to the decision that I could do more to help my own profession through blogging than I could the Walmart workers of the world (although they all still have my general support). Yet that experience was hardly a total bust. For one thing, it explains why I’m a vegetarian. It also explains why I once talked on the phone with the British journalist Simon Head.
I don’t remember exactly why he contacted me. I think I had written something about Walmart that implied that I knew more than I really did. What he ended up doing is schooling me on the evils of Walmart’s computerized scheduling system. The long and short of it is that Walmart schedules its workers on the basis of when its computer system predicts that customers will be in the store. If there aren’t enough people there to justify paying you, then you stay home. To make matters worse, Walmart demands that its employees be on call for work at any shift, any time, thereby making other jobs (or even going to school) that much more difficult. Since Head first wrote about that system, it’s become absolutely commonplace in workplaces of all kinds. It’s one very important reason why so many cheap employers today can employ all part-time workers and not be understaffed.
Head has a new book out now, Mindless: Why Smarter Machines Are Making Humans Dumber. You may have seen an excerpt from it about Amazon in Salon last month. I’ve read the whole thing now, and I can tell you it’s almost certainly going to be the most important book I’ll read all year. In essence, it is an update on Head’s earlier work on Walmart. Rather than just use the computer to dictate when your employer needs you to work, technology is now powerful enough for management to use it to dictate exactly how workers do their jobs. For example, here’s a bit from that excerpt about what Amazon warehouse workers do all day:
Amazon’s system of employee monitoring is the most oppressive I have ever come across and combines state-of-the-art surveillance technology with the system of “functional foreman,” introduced by Taylor in the workshops of the Pennsylvania machine-tool industry in the 1890s. In a fine piece of investigative reporting for the London Financial Times, economics correspondent Sarah O’Connor describes how, at Amazon’s center at Rugeley, England, Amazon tags its employees with personal sat-nav (satellite navigation) computers that tell them the route they must travel to shelve consignments of goods, but also set target times for their warehouse journeys and then measure whether targets are met.
All this information is available to management in real time, and if an employee is behind schedule she will receive a text message pointing this out and telling her to reach her targets or suffer the consequences. At Amazon’s depot in Allentown, Pennsylvania (of which more later), Kate Salasky worked shifts of up to eleven hours a day, mostly spent walking the length and breadth of the warehouse. In March 2011 she received a warning message from her manager, saying that she had been found unproductive during several minutes of her shift, and she was eventually fired. This employee tagging is now in operation at Amazon centers worldwide.
If you read Head’s entire book, you’ll see that these same principles are now being applied to white-collar jobs of all kinds. Work with a computer, you may be subject to this kind of Taylorism no matter what your particular income level happens to be.
In the book, Head notes the effect of this kind of management system on British universities. Unfortunately, he does not go the extra step of reporting (or even just predicting) the effect that these kinds of programs might have upon teachers involved in online education. Nevertheless, the implications should be obvious. Teach online and your every interaction – heck your every keystroke – is subject to scrutiny if you use a system that your employer controls. I’m not saying this is happening now everywhere, but it is easy to imagine that this will be happening somewhere soon. Even the best online educators will be subject to this kind of scrutiny if their employers care more about efficiency than they do about education.
Can this really happen? Well, look at MOOCs. Here’s Jim Groom, reviewing some recent history in that area:
MOOCs, as Siemens and Downes imagined them, are one of the few sources of true innovation you can point to in educational technology in recent history, and it was born from a higher ed/government relationship. Yet, within a couple of years the MOOC movement had become increasingly denatured and over-run by corporate boosterism that was redirecting the logic of experimentation and possibility to a rhetoric of how broken higher education is, and how Silicon Valley (poaching superstar faculty from Stanford with the allure of million of dollars) has come to its rescue. What was remarkable to me as I watched the MOOC experiment transform into a corporate takeover was how quickly and completely the alien pods took over the experimentation before it could breath. before it could even develop it was already a fully formed disruptive solution to a moribund institution. Innovation lost.
As I know I once read Marc Bousquet write somewhere, many universities admire the for-profit online approach and have rushed into it not to improve education, but to improve their own bottom lines. You may be convinced your online class is the best online class that can be given online (and it might very well be better than a lot of in-person lecture-only classes at giant state universities), but how long will your employer let you keep teaching it in the inefficient way that you’re teaching it now? I’m not saying this disaster is going to befall every online instructor, but it will certainly befall some of them as budgets grow tighter in higher education worldwide. The question then becomes, “Where will the line between the lucky and the unlucky get drawn?”
I’ve started to feel as if I walk that line every day. Our recent troubles here with our research downloads has reminded me that nothing lasts forever. This is particularly true for those of us who work at universities with administrations that do not value what skilled faculty bring to the educational table. If I am not swept up in a wave of digitization that will allow my job to be Taylorized, the CSU-System could still simply invest its resources in creating new campuses where a surveillance state can be constructed at birth.
As longtime readers know, I keep a close watch on this job of mine. I keep my eyes wide open all the time. But what happens if this kind of vigilance makes no difference?
"Rees has written a solid, comprehensive account of the technological creation of cold chains in the United States. I wish this book had been available for me to read when I was doing my own research."—Mansel G. Blackford, Ohio State University.
More or Less Bunk 
Recent Comments