The ultimate killer of robograding os Dr. Les Perelman of MIT, who recently retired as a professor teaching writing to students at MIT. Les Perelman and his students cracked the code of robograders and showed how easy it is to fool the computer scoring essays. Use long sentences. Use o score or multisyllabic words. Don’t worry about whether your assertions make sense or are correct. The robograder doesn’t care if you say that World War II started in 1902, because facts don’t matter. Perelman and his students created a device called a Babel Generator, into which you insert any three words and it will spit out a high-scoring incoherent essay.
Peter Greene describes an effort to reclaim the soiled reputation of the robograder. He describes several states where robograders are currently in use.
Diane I take hope in that. Bless those thinkers.
There are several names for it: the factory model (should be obvious); scientific positivism (which defines reality only by its physical attributes . . . all else is not really real or “real science”); the statistical model (which is fine if understood as only one aspect of our knowledge of humans and our development).
And then when we are flattened enough to think buying something new is the only antidote to spiritual decay, there is the capitalist-only model (which is underpinned by a confused mixture of all the others, but adds its own transactional-only framework to the mix . . . no ends, just means . . . nothing is authentic if it’s for itself alone).
Bless those thinkers. CBK
and so very Reformer Brain: “buying something new is the only antidote” —no one apologies for the last huge education mess as the next new huge education mess is being sold
This is very helpful, thanks.
The NY Performance Assessment Consortium has, imho, done fabulous work for more than 20 years. This group – of now 38 district public schools in New York State – have convinced the state to allow them to use human – educator judgement via portfolios to determine when students have mastered skills making them ready to graduate from high school.
More info here http://www.performanceassessment.org/history
It’s exactly opposite of using computers to judge students’ skills.
What I have to say about roboscoring is simple: not in my class. Here’s what the Babel Generator has to say on the subject:
Writing has not, and doubtlessly never will be generous but not analytical. Incarceration by an affirmation can, nonetheless, be obviously cornucopian. From adhering, the assimilationist that scrutinizes irreverence for unreal can be more exorbitantly ascertained. Unreal with the inquiry will always be a experience of humanity. Composition is humane by its pusillanimously and reprovingly enthralling civilizations which menace inspections or fascinate a opulent unscrupulousness.
LOL! Perfect reply!
Should be “an opulent”, not “a opulent”
Fail.
Sincerely, Robo Grader, PhD (Philosopher of Data)
Hey, that reminds me of somebody:
Out here in Utah, one of the states mentioned, reading my students’ writing in the last few years has been so dispiriting. Most of my 9th graders can’t come up with a coherent sentence or paragraph. I blame the “Utah Compose” program that auto-grades writing. Students are only learning to write for a computer, not an actual audience. I don’t teach ELA, but my students write a lot in my class, and I read it. I don’t know how what is happening anymore can be called “writing.”
Those AI systems so easily gamed from AIR are now part of a larger portfolio with health care and workforce preparation. See this recent acquisition by AIR for healthcare and more.
https://www.globenewswire.com/news-release/2020/05/18/2035176/0/en/American-Institutes-for-Research-Completes-Acquisition-of-IMPAQ-LLC.html
In those states not utilizing “robo-scoring,” writing instruction gets very short shrift in K12 [& of course, the former only teach ‘writing for the computer audience’]. Except of course in well-heeled school districts which can afford small-enough classes and a big enough ELA faculty to make it possible. A skill which is crucial to most professions and higher-salaried positions. Just one more brick in the wall, folks.
This is really great.
I used to show my classes of juniors and seniors the Babel Generator and have them select 3 words to use. It was always very enlightening.