Peter Greene just keeps writing hit after hit. This
one explains what VAM means and why it works well in
manufacturing but not in dealing with human beings.
He explains how
Pennsylvania measures teacher quality: PVAAS uses a
thousand points of data to project the test results for students.
This is a highly complex model that three well-paid consultants
could not clearly explain to seven college-educated adults, but
there were lots of bars and graphs, so you know it’s really good. I
searched for a comparison and first tried “sophisticated guess;”
the consultant quickly corrected me—“sophisticated prediction.” I
tried again—was it like a weather report, developed by comparing
thousands of instances of similar conditions to predict the
probability of what will happen next? Yes, I was told. That was
exactly right. This makes me feel much better about PVAAS, because
weather reports are the height of perfect prediction.
Here’s how it’s supposed to work.
The magic formula will
factor in everything from your socio-economics through the trends
over the past X years in your classroom, throw in your pre-testy
thing if you like, and will spit out a prediction of how
Johnny would have done on the test in some neutral universe where
nothing special happened to Johnny. Your job as a teacher is to get
your really Johnny to do better on The Test than Alternate Universe
Johnny would. The only thing that goes wrong is that it
doesn’t work. Students are not inanimate objects like pieces of
steel. So he concludes: This is one more example of a
feature of reformy stuff that is so top-to-bottom stupid that it’s
hard to understand.
But whether you skim the surface, look at the
philosophical basis, or dive into the math, VAM does not hold up.
You may be among the people who feel like you don’t quite get it,
but let me reassure you– when I titled this “VAM for Dummies,” I
wasn’t talking about you. VAM is always and only for dummies; it’s
just that right now, the dummies are in charge.
See? All that’s required for VAM to work is believing
that the state can accurately predict exactly how well your
students would have done this year if you were an average teacher.
How could anything possibly go wrong??
“when I titled this “VAM for Dummies,” I wasn’t talking about you. VAM is always and only for dummies; it’s just that right now, the dummies are in charge.”
A fine observation. 🙂
I have been in conversation with RI’s Deborah Gist about the use of VAM in RI. She told me that what we are (or will be) using in RI is NOT VAM, but rather a “growth model”. Can someone tell me the difference between VAM and a growth model?
There really isn’t any difference. I’ve spent a fair amount of time digging into the information relating to the Massachusetts “Student Growth Percentile” which is called a “growth model” rather than VAM, and what it amounts to is exactly what Peter Greene describes — through a very sophisticated complex regression analysis it predicts how Johnny would have done compared to his peers if he had a big enough group of peers with exactly the same score history and then assigns him a percentile that is supposed to indicate how Real Johnny actually did with respect to those virtual peers. In my school, two teachers who team teach full-time (in the same classroom, with two classes of students combined) have obtained very different median SGPs for the same year, and I have seen another teacher’s median SGP swing by 30-40 points from one year to the next depending on whether she was teaching students with disabilities or not.
So it could in essence be a way that allows reformy types who are confronted with the problems of using VAM to say, “Oh, we don’t use VAM here — we use a growth model!”
Sound about right?
Ron,
Bruce Baker goes into detail on SGPs on his schoolfinance101 site.
VAM, SGPs it’s all porcine excrement simmered in the hot summer sun spreading throughout the nation its wonderful aroma.
Ron Poirier & LindaM & Duane Swacker: exactly.
The rebranding of the same old putrid wine in new bottles.
One of the most striking features of the charterite/privatizer movement is the dense cloud of smug arrogance that surrounds all their attempts at reviving failed ideas and practices like merit pay for teachers and management by the numbers and so on.
They truly and honestly feel and think that all it takes is for them to make a pronouncement, proof by assertion, and then everyone will worshipfully fall on the ground in obeisance.
Nothing sadder than folks who believe their own hype and pr and slogans.
Thank you for your comments.
😎
In Massachusetts, the Dept of Ed is requiring district-determined measures for those subjects not tested by state tests, & the original proposal was that districts should do pre- and post-tests, and order the results by “growth”, placing teachers with kids in the bottom third into low, teachers of kids in the middle third into “moderate,” and teachers of kids in the top third into “high”. And their SGPs are percentiles — last year, they announced in a webinar that I attended that they have realized that using percentiles means that 50% of the kids will always be below the 50th percentile even if the state scores are all growing, so now they are calculating SGP using historical virtual peers to raise the SGPs a little. (I’m not joking — I have the powerpoint in which they said this.)
We teachers in New Mexico are assured by the VAM dummies that apples + oranges = pears.
I’m still waiting to see how They are going to work the “grit” metric into the VAM algorithm for agricultural bovine analysis.
Richard Feynman’s comment applies: