Audrey Amrein-Beardsley posted a guest blog by a rising star in the Academy, Jimmy Scherrer of North Carolina State University, who previously taught in LAUSD.
Scherrer wrote:
“As someone who works with students in poverty [see also a recent article Scherrer wrote in the highly esteemed, peer-reviewed Educational Researcher], I am deeply troubled by the use of status measures—the raw scores of standardized assessments—for accountability purposes. The relationship between SES and standardized assessment scores is well known. Thus, using status measures for accountability purposes incentivizes teachers to work in the most advantaged schools.
“So, I am pleased with the increasing number of accountability systems that are moving away from status measures. In their place, systems seem to be favoring value-added estimates. In theory, this is a significant improvement. However, the manner in which the models are currently being used and how the estimates are currently being interpreted is intellectually criminal. The models’ limitations are obvious. But, as a learning scientist, what’s most alarming is the increasing use of the estimates generated by value-added models as a proxy for “effective” teaching…..”
“Typically, research studies on teaching and learning are framed using one of three perspectives: the behaviorist, the cognitivist, and the situative. Each perspective is associated with a different grain size. The behaviorist perspective focuses on basic skills, such as arithmetic. The cognitivist perspective focuses on conceptual understanding, such as making connections between addition and multiplication. The situative perspective focuses on practices, such as the ability to make and test conjectures. Effective teaching includes providing opportunities for students to strengthen each focus. However, traditional standardized assessments mainly contain questions that are crafted from a behaviorist perspective. The conceptual understanding that is highlighted in the cognitivist perspective and the participation in practices that is highlighted in the situative perspective are not captured on traditional standardized assessments. Thus, the only valid inference that can be made from a value-added estimate is about a teacher’s ability to teach the basic skills and knowledge associated with the behaviorist perspective.”
This, he writes, is “intellectually criminal” and “intellectually lazy.”
Tell it! VAM is Junk Science.
Diane Ravitch's blog 
Honestly, I talk to people outside of teaching and you would lose them after the word “status measures”. The public listens to sound bytes like “schools are failing” or “teachers need accountability”, not reason and analysis. That is why the Reformers are winning. They have the money to buy the public’s perceptions. Just like what is happening in Reynoldsburg and the Kasich/Koch puppets on the school board. Try to explain how the board there is destroying schools and all you get are glazed eyes and blank stares. Ohio is failing and no one really pays attention. Disgusted.
“The conceptual understanding that is highlighted in the cognitivist perspective and the participation in practices that is highlighted in the situative perspective are not captured on traditional standardized assessments. Thus, the only valid inference that can be made from a value-added estimate is about a teacher’s ability to teach the basic skills and knowledge associated with the behaviorist perspective.”
Agree on the VAM scam, but also think this scholar needs to look beyond math and ELA for examples of the problems with testing for learning.
Consider the insistent use of the terms “academic” and “rigorous” and “impactful” as adjectives to describe valued qualities of teaching and learning.
I think these overused terms are intended to tell teachers they should not provide for: (a) students’ demonstrations of learning by participating in a semi-structured project or task and/or (b) students’ inventing and structuring a project to represent what they have learned.
Those options, a and b, are regarded as soft and fuzzy and personalized demonstrations of learning that teachers of a “progressive persuasion” are likely to favor. Standardized tests (and everything else) is an efficient way to kill off those values and to dismiss the idea that learning is an experience apart from mastering content in texts and becoming proficient in skills for passing tests (cheap, easy to score, low inference).
I think these overused terms are also intended to cut off attention to learning that may require conjectures, making inferences beyond those closely tied to well-worn paths for a “correct answer,” or one best answer. You can see this in the mandate for “close reading” in the CCSS, the intolerance of inferences drawn from personal experience, the greater attention given to informational texts over literature, the hard-nosed insistence on verbatim treatment of those standards.
What happens to assessment when you move beyond well-mapped curriculum territory and conventional parsing of curriculum and instruction in school? By conventional, I mean the custom of introducing simple basic skills, then working toward conjunctions and connective networks of these, and then addressing more complex tasks or problems (known by the teacher or test-maker to have a limited number of efficient and relevant responses)…the whole concept of “learning progressions.”
Suppose, for example, the assessment challenge is to make up a dance with two other people, or compose and sing a tune with lyrics about clouds, or write a play where trees have conversations with one another; or invent a method of drawing without paper and without a tool that you hold in your hand or paper. Suppose these are invitations not tied to a notion of a learning progression, or a grade level, or “baseline data,” or proficiency.
I don’t know of any clear way to map what students may learn from invitations such as these. I could construct a map of “intended” learning and outcomes, but those exercises would be little more than a demonstration of my analytical skill (or lack of it) in meeting a demand for accountability, not because that process would yield “proper criteria” for judging these performances, or gaining more general insights into learning, or extracting lessons about more “effective” teaching.
This is to say that today’s regime of testing is not just a technical problem of misrepresenting what students learn in school and specifically from their teachers, but an ideological one. Perhaps that was also the point of this post about theoretically informed research.
VAM: The Scarlet Letter
A talk given to the School Board of Palm Beach County, Florida. March 12, 2014. By Andy Goldstein and Ellen Baker.
Thank you Jimmy Scherrer for an excellent article that validates our need to get rid of the toxic CCSS dysfunctional environment that an obsession with testing and data has created. You inspired me to write my message again attacking punitive Behaviorism in schools:
Many of us who have worked in elementary schools for years have observed a disturbing trend toward Authoritarian Behaviorism. Authentic teaching has diminished to become test indoctrination. Punitive methods of Behaviorism are being embraced and called Positive Discipline. This pedagogy is a sign of social regression from more primitive thinking. Primitive thinking results from people who are functioning in a survival mode from fear and insecurity. They are “reactors”. Many adults now grew up in unrecognized PTSD Families and functioned as children in a chronic state of hypervigilance from fear and insecurity, so it is understandable they cannot recognize this as psychologically harmful to children. They were conditioned in childhood with coping mechanisms (Avoidance, Denial, Dissociation) that helped them survive chronic stress (often traumatic stress), but those coping mechanisms do not serve them well in adulthood. Anyone who cannot recognize this current environment of punitive Behaviorism as harmful to children is still using those primitive mechanisms of childhood coping. They are not using scientific thinking or common sense, and they cannot empathize with children.
Having grown up in a PTSD Family (ACoA Families & Covert Narcissistic Families) with a father with unrecognized PTSD after WWII and a mother with (unrecognized & undiagnosed) Anxiety/Depression, I was led into a career of counseling following my younger brothers suicide. Workaholism or Alcoholism is often a symptom in these families. Like most Narcissistic Families of the boomer generation in the Old South, we looked like the typical All American Family. Successful hard working parents, smart, high achieving, obedient children who learned (from walking on eggshells) never to question authority and to keep up appearances at all costs. I am sharing this personal example because I can see a frightening increase in Covert Narcissistic Families from my personal experience as well as years of working with young children. We only need to observe the statistics from growing rates of relationship trauma, child abuse, divorce statistics, mental illness, and our rapidly growing prison population to notice we have dysfunction that cannot be denied or avoided any longer. Our schools must become safe havens for children, not places of punishment, fear, and intimidation.
After observing the impact of this increasingly punitive school environment on younger children for years, and after observing that chronic stress is now becoming traumatic stress for many children, I started writing articles last year comparing the CCSS school environment to that of a dysfunctional family. The dynamics are the same. Dysfunction is on a spectrum, and is impacted by environmental stress. Our school environment of CCSS is similar to the poisonous pedagogy that was used to “train” children in Germany for decades leading up to WWII. Schwarze Padagogik. Although in mainstream American society now, it is not being perpetuated intentionally, but unconsciously. It is a method of indoctrination since it creates an environment of entrapment and domination. It stifles children’s healthy development of identity and produces children who are obedient and codependent to authority (even abusive authority). It leads to anxiety, depression, and personality disorders. Punitive methods of Behaviorism are recognized by the Behavior Analyst Certification Board (BACB) in the US as psychologically harmful to children. Why is the BACB silent on this? Why is NAMI not more vocal on this? Why are we allowing punitive Behaviorism to be used in schools when it has been scientifically proven to be harmful and cause permanent psychological damage to children. Why are we using the same methods of obedience training for children as that used for dogs and circus animals and calling it “teaching”.
How can we get people to step out of the dysfunction to recognize the big picture?
Child abuse is not a partisan issue. We need legislation that requires schools to “Do No Harm” by outlawing punitive Behaviorism. We need schools that have empathy. We need rehab for a nation.
For those who will dismiss this as hyperbole. Please do your homework first:
Here are some books I recommend, or you can do your own research since there are numerous scholarly abstracts and online:
Drama of the Gifted Child by Alice Miller
Understanding the Borderline Mother by Christie Lawson
Childism by Elizabeth Young-Bruehl
The ACoA Trauma Syndrome by Tian Dayton
“The Origins of War in Child Abuse” by Lloyd DeMause is available free audio on this site: http://www.psychohistory.com/
What is NAMI??
National Alliance on Mental Illness
xxxxxxxxxxxxxxxxxxxxx My sense is yes you’re right & you may be overthinking it. For one thing, Americans are woefully [& even willfully] ignorant about mental illness. Even slim experience with it will teach you, for example, that there are no hospitals which treat any combination of physical and mental illness at the clinical level, despite a plethora of neurological evidence connecting brain to body. We are in the 19thc. on this medically.
But to get to your point: the wave of austerity/ authoritarianism/ accountability afflicting the political scene (the usual fallout in hard economic times, on steroids thanks to ‘no excuses’ data-driven digital econometrics), tho it may be directed by for-profit corporate types, depends on appeal to voters’ basest instinct: fear. There is an unapologetic reach-out to those who feel angry and powerless. And who are those people? They include the bullies, over-represented among the poor and ignorant, encouraged by the evangelicals. The same people who don’t talk about or believe in or inform themselves about mental illness– the ones who arm themselves against ‘those people’/ ‘the government’, who take out their lack of social standing by beating their wives, even the ones for whom the ‘only thing taboo about in*est is talking about it’.
Quite. No doubt that VAMpire is responsible for feeding zombie machines firing teachers.
Am fretting and feeling upset. My daughter’s fabulous Latin teacher was forced to give them the final on the 4th day of class as a “pre-test” so that the district could evaluate her at the end of the semester. It goes against everything I believe as a parent and everything she stands for as an educator. I believe that using this exam as a measure of a teacher is harmful to the students who love her so. I am so afraid that she’ll have had enough and leave. What kind of nonsense are we doing to our children.
Who among the administrators will know what the students wrote in Latin? Only the teacher knows.
Exactly. To me it’s showing that the district has a lack of trust in her as a teacher and are simply data hungry. It requires a great deal of work on the part of the students to learn and do well in Latin.
I assume that the students would do terribly on the exam.
If I were the teacher I would keep the exam and give it back to the students after they take the summative final in the spring so they can reflect on how much they have learned.
I am glad you are not their teacher then. It makes no sense to give an exam that is entirely material that you don’t yet know.
Sarah5565,
I think it might well give the students some satisfaction to see how much they have learned in the class.
Sarah, Your district officials probably do not care if giving the final as as a pretest makes the later use of the same test invalid. The experts call this the practice effect.
Teaching economist assumes the teacher can keep the pretest and return it students after the posttest. Unlikely. That would allow the teacher to teach to the specific questions on the test. The point of the pretest and posttest really is to evaluate the teacher, with the students’ scores instruments for that. It this absurd policy really bothers you and other parents, make noise with the district board of education, your state and congressional representatives.
Laura,
I would recommend that they not use the same test for both the pre and post test, but a similar one.
The point of taking the class is to learn Latin and learn about history and the roman empire and read literature. This teacher is the only Latin teacher in the district and the only one who can grade this test anyways. She deserves the courtesy of respect from the district that after very successfully teaching Latin for more than 25 years she knows how.
“Common Core and VAM”
(With apologies to the master of goofy poetry, Dr. Seuss)
I am DAM
DAM I am
Do you like
Common Core and VAM?
I do not like them, DAM I am.
I do not like
Common Core and VAM.
Would you like them
here or there?
I would not like them
here or there.
I would not like them anywhere.
I do not like
Common Core and VAM
I do not like them, DAM I am.
Would you like them in a school?
Do you like them as a rule?
I would not like them
in a school.
I do not like them
as a rule.
I would not like them
here or there.
I would not like them
anywhere.
I do not like
Common Core and VAM.
I do not like them,
DAM I am.
Would you like them with a Carter?
would you like them with a charter?
Would you like them on the news?
Would you like them with a FUSE?
Would you like them with a Chetty?
Would you like them with spaghetti?
Would you like them for a firing?
Would you like them for a hiring?
Would you like them in Louisiana?
Would you like them in Montana?
Would you like them for your pay?
Would you like them for a day?
I would not like them with a Carter
I would not like them with a charter
I would not like them on the news
I would not like them with a FUSE
Not with spaghetti
Not with a Chetty
Not for a firing
Not for a hiring
Not in Louisiana.
Not in Montana
Not for my pay
Not for a day
I would not like them
here or there.
I would not like them anywhere.
I do not like
Common Core and VAM
I do not like them, DAM I am.
Splendid, again. I don’t think apologies are necessary.
I suppose I am the only one to read the quoted passages to mean that the author is arguing that the current system is using the wrong test. Presumable using the right test, with questions from the behaviorist point of view, the cognitivist, and the situative point of view would be just fine with this “rising star”.
You may be confusing yourself with word salad.
Ken,
I agree that the distinctions Jimmy Scherrer are making might not be useful, but it is clear that he thinks using VAM is a valid way to draw inferences about a teachers ability to teach some things.