Archives for category: VAM (value-added modeling)

Bryan Alleman, the technology coordinator for Acadia Parish schools in Louisiana, sent the following story. VAM, as you know, is “value-added measurement,” or “value-added modeling,” which is used in many states to evaluate teachers. If test scores go up as predicted by a computer, the teacher is effective; if they exceed the computer prediction, the teacher is “highly effective.” If they don’t, the teacher is “developing” or “ineffective.” Secretary of Education Arne Duncan has pushed some version of this approach into almost every state as a condition of Race to the Top funding or waivers from the onerous sanctions of No Child Left Behind.

Bryan writes:

Greetings from south Louisiana! I wanted to share with you an interaction I recently had with a Pearson representative at a technology leadership conference I attended in Baton Rouge, LA a couple of weeks ago.

At the conference, I noticed Pearson was present as a vendor. Pearson owns Powerschool, a Student Information System (SIS), that has a small presence in Louisiana. Rumor is that Pearson is selling off Powerschool.

Typically, SIS systems can be very profitable for a company. Working in SIS management for the past 7 years, I wanted to confirm this rumor.

So, I approached the Pearson rep., a nice gentleman, who confirmed that indeed Pearson was selling off Powerschool. Curious, I asked him, why? Imagine my surprise when I heard the following (paraphrasing):

“…Pearson is strongly committed to improving student outcomes and has decided to score every single product it owns to determine the impact on student achievement. Powerschool didn’t score well so we are selling it off…”

So there I stood–mouth agape—at the realization that Pearson has fallen for it’s own scam. They have actually VAM’d themselves. And, as a result, is selling of a profitable product.

The parallels to students, teachers, principals, and whole schools who have also fallen victim to VAM flooded my brain.

Education has lost valuable human capital (I despise that reference typically….we are human resources, not capital) as a result of VAM.

Pearson is losing a valuable product as a result of their own VAM. I wonder how the Pearson shareholders feel about this.

Just amazing.

Bryan

__________________________

“It is the mark of an educated mind to be able to entertain
a thought without accepting it.”
-Aristotle

Diane,

is that Pearson is selling off Powerschool. Typically, SIS systems can be very profitable for a company. Working in SIS management for the past 7 years, I wanted to confirm this rumor. So, I approached the Pearson rep., a nice gentleman, who confirmed that indeed Pearson was selling off Powerschool. Curious, I asked him, why? Imagine my surprise when I heard the following (paraphrasing): “…Pearson is strongly committed to improving student outcomes and has decided to score every single product it owns to determine the impact on student achievement. Powerschool didn’t score well so we are selling it off…”

So there I stood–mouth agape—at the realization that Pearson has fallen for it’s own scam. They have actually VAM’d themselves. And, as a result, is selling of a profitable product.

The parallels to students, teachers, principals, and whole schools who have also fallen victim to VAM flooded my brain.

Education has lost valuable human capital (I despise that reference typically….we are human resources, not capital) as a result of VAM.

Pearson is losing a valuable product as a result of their own VAM. I wonder how the Pearson shareholders feel about this.

Just amazing.

Bryan

__________________________
Bryan P. Alleman
Technology Coordinator
Acadia Parish School Board
P. O. Drawer 309
Crowley, LA 70527-0309
phone: (337) 783-3664 x279
fax: (337) 783-0194
balleman@acadia.k12.la.us
“It is the mark of an educated mind to be able to entertain
a thought without accepting it.”
-Aristotle

,

Audrey Amrein-Beardsley is one of the nation’s leading experts on value-added assessment. She is also one of its most prominent critics. In her blog VAMboozled, she follows the ongoing damage done by VAM to teachers and schools across the nation.

 

In this post, she draws attention to the findings of one of her graduate students, Jessica Holloway-Libell, who studied the use of VAM in Tennessee. Tennessee has employed a VAM model since the early 1990s based on the work of agricultural statistician William Sanders. Jessica’s study has been published in the peer-reviewed Teachers College Record.

 

Jessica found “Evidence of Grade and Subject-Level Bias in Value-Added Measures,” the title of her article.

 

Amrein-Beardsley writes:

 

More specifically, Jessica found that:

 

Teachers of students in 4th and 8th grades were much more likely to receive positive value-added scores than in other grades (e.g., 5th, 6th, and 7th grades); hence, that 4th and 8th teachers are generally better teachers in Tennessee using the TVAAS/EVAAS model.

 
Mathematics teachers (theoretically throughout Tennessee) are, overall, more effective than Tennessee’s English/language arts teachers, regardless of school district; hence, mathematics teachers are generally better than English/language arts teachers in Tennessee using the TVAAS/EVAAS model.

 

Are teachers of fourth and eighth grades better than all others; are math teachers better than English teachers?

 

Amrein-Beardsley concludes that these findings are additional evidence that Sanders’ VAM is inaccurate and biased towards certain grades and subjects.

 

 

William Sanders, a pioneer in the early implementation of value-added measurement in Tennessee, was an agricultural statistician when he realized that children could be measured in their test score growth like cattle or corn, and that teachers could be held responsible for that growth in test scores from year to year. His TVASS system was adopted by Tennessee in 1993. If it worked as its proponents devoutly believe, Tennessee should be #1 in the nation in test scores by now. It is not. It is not even close.

 

Our blog poet, who calls him/herself “SomeDam Poet,” wrote the following ode to Professor Sanders:

 

“If I have seen farther, it is by standing on the shoulders of giants” — Isaac Newton, who invented differential calculus and wrote down what are now known as Newton’s Laws of Motion

 

“If I have seen fodder, it is by standing on the horns of cattle” — William Sanders, who first applied VAM for cattle to teachers and wrote down Sanders’ Laws of Self-promotion and teacher demotion

Surprise: Most of the teachers in Ohio are effective or highly effective.

 

However, there are more ineffective teachers in districts with high levels of poverty.

 

Chicken-and-egg? Correlation?

 

Lesson: if you want to be a highly-rated teacher, avoid high-poverty schools and districts.

 

Second lesson: This is a ridiculous way to evaluate teachers, and the results were predictable and flawed. Also meaningless.

 

How many millions were spent to learn this?

 

 

Audrey Beardsley reveals the answer to the intriguing question: Why is D.C. hiding VAM data? The answer was earlier leaked to blogger and retired math teacher G.F. Brandenburg. Beardsley cites him in this post.

The VAM data show that VAM is junk science. Keep it a secret. D.C. school officials are trying to.

“In Brandenburg’s words: “Value-Added scores for any given teacher jumped around like crazy from year to year. For all practical purposes, there is no reliability or consistency to VAM whatsoever. Not even for elementary teachers who teach both English and math to the same group of children and are ‘awarded’ a VAM score in both subjects. Nor for teachers who taught, say, both 7th and 8th grade students in, say, math, and were ‘awarded’ VAM scores for both grade levels: it’s as if someone was to throw darts at a large chart, blindfolded, and wherever the dart lands, that’s your score.”

The Education Cpmmission of the States will present its James Bryant Conant award to Willism Sanders. Sanders is a pioneer of VAM (also known as value-added measurement or modeling).

 

VAM is probably the single worst feature of corporate reform, the one that is most likely to demoralize teachers, lead to early retirements, and to the decline in new recruits to teaching. Sanders promotes the idea that teachers can be evaluated by the test scores of their students. He pioneered VAM in Tennessee in the late 1980s and today it is a widely used methodology, even though Sanders has copyrighted his methods; it is proprietary and other researchers are not allowed to understand how it works.

 

Although Sanders’ team markets his product with grandiose claims, one need only look at Tennessee to see that it is not near the top of NAEP. After 30 years of VAM, what does Tennessee have to show for its reliance on high-stakes testing? Who would call Temnessee today a national model?

 

Audrey Amrein-Beardsley describes Sanders:

 

“VAMs were first adopted in education in the late 1980s, when an agricultural statistician/adjunct professor [emphasis added, as an adjunct professor is substantively different than a tenured/tenure-track professor] at the University of Knoxville, Tennessee – William Sanders – thought that educators struggling with student achievement in the state should “simply” use more advanced statistics, similar to those used when modeling genetic and reproductive trends among cattle, to measure growth, hold teachers accountable for that growth, and solve the educational measurement woes facing the state at the time by doing so. It was to be as simple as that….”

 

Chalkbeat, which covers education issues in Tennessee, recognizes that VAM is very controversial:

 

“As per the article: “Hailed by many who seek greater accountability in education, [Sanders’s] TVAAS continues to be a topic of robust discussion in the education community in Tennessee and across the nation. It has been the source of numerous federal lawsuits filed by teachers who charge that the evaluation system—which has been tied to teacher pay and tenure—is unfair and doesn’t take into account student socio-economic variables such as growing up in poverty. Sanders maintains that teacher effectiveness dwarfs all other factors as a predictor of student academic growth.”

 

Amrein-Beardsley is stunned that ECS is giving this honor to a man who tried to turn teacher evaluation into a “science” comparable to producing crops. She wonders whether angry teachers might picket the ECS meeting in Denver in late June.

Carol Burris writes of the terrible consequences that will follow implementation of Governor Cuomo’s teacher evaluation plan.

She urges support for the plan created by seven (of 17) dissident members of the Néw York Board of Regents. Almost all are experienced educators who have carefully reviewed research. Cuomo is not an educator and obviously paid no attention to research.

Two more Regents and the dissidents are a majority.

Lester Young of Brooklyn? Roger Tilles of Long Island?

Master teacher Sheri Lederman is suing the State of New York after having received a low rating on the state’s “growth” measure. Her husband Bruce is her lawyer. She has been teaching for 18 years and has earned her doctorate. While only 31% of the students in the state “passed” the Common Core tests as proficient, 66% of the students in Dr. Lederman’s class were proficient. But the state gave her a low rating because, by the state’s convoluted formula, the students did not “grow” enough in their test scores.

The New York State Education Department tried to get the lawsuit dismissed, but their effort was rejected and the case is moving forward.

One of the strengths of the Lederman’s case is the excellent affidavits submitted by experts, as well as by parents and students. You can read the affidavits here. You will be informed by the expert statements of Linda Darling-Hammond, Audrey Amrein-Beardsley, Carol Burris, Aaron Pallas, and Brad Lindell.

Darling-Hammond says that Lederman’s rating is “utterly irrational.”

Amrein-Beardsley says that no VAM rating–given the current state of knowledge or lack thereof– is sufficient valid or fair to rate individual teachers.

You will find the testimony of parents and former students enlightening.

Bruce Lederman, an attorney acting on behalf of his wife, experienced elementary school teacher Sheri Lederman, filed suit to challenge the state’s teacher evaluation system. The New York State Education Department sought to have the case thrown out. Today, the New York Supreme Court ruled that the lawsuit can go forward. Good for the Ledermans!

From Bruce Lederman:

The NY Supreme Court has denied a motion by the NY Education Department to dismiss the Lederman v. King lawsuit, in which an 18 year veteran Great Neck teacher has challenged a rating of “ineffective” based upon a growth score of 1 out of 20 points, even though her students performed exceptionally well on standardized tests.

This means that the NY Education Department must now answer to a Judge and explain why a rating which is irrational by any reasonable standard should be permitted to remain. The NY Education Department argued that Sheri Lederman lacked standing to challenge an “ineffective” rating on her growth score since her overall rating was still effective and she was not fired. A judge disagreed and determined that an ineffective rating on a growth score is an injury which she is entitled to challenge in Court.

Now, Sheri will have her day in Court. A hearing will likely be scheduled in August.

In this brilliant article, Marc Tucker explains why the civil rights community is making an error by supporting annual testing as a “civil right.” He knows their leaders believe that poor and minority children will be overlooked in the absence of annual testing. But he demonstrates persuasively that annual testing has done nothing to improve the academic outcomes of poor and minority children and that they have actually been harmed by the pressure to raise scores every year.

 

Tucker writes:

 

First of all, the data show that, although the performance of poor and minority students improved after passage of the No Child Left Behind Act, it was actually improving at a faster rate before the passage of the No Child Left Behind Act. Over the 15-year history of the No Child Left Behind Act, there is no data to show that it contributed to improved student performance for poor and minority students at the high school level, which is where it counts.

 

 

Those who argue that annual accountability testing of every child is essential for the advancement of poor and minority children ought to be able to show that poor and minority children perform better in education systems that have such requirements and worse in systems that don’t have them. But that is simply not the case. Many nations that have no annual accountability testing requirements have higher average performance for poor and minority students and smaller gaps between their performance and the performance of majority students than we do here in the United States. How can annual testing be a civil right if that is so?

 

 

Nonetheless, on the face of it, I agree that it is better to have data on the performance of poor children and the children in other particularly vulnerable groups than not to have that data. But annual accountability testing of every child is not the only way to get that data. We could have tests that are given not to every student but only to a sample of students in each school every couple of years and find out everything we need to know about how our poor and minority students are doing, school by school.

 

 

But the situation is worse than I have thus far portrayed it. It is not just that annual accountability testing with separate scores for poor and minority students does not help those students. The reality is that it actually hurts them.

 

 

All that testing forces schools to buy cheap tests, because they have to administer so many of them. Cheap tests measure low-level basic skills, not the kind of high-level, complex skills most employers are looking for these days. Though students in wealthy communities are forced to take these tests, no one in those communities pays much attention to them. They expect much more from their students. It is the schools serving poor and minority students that feed the students an endless diet of drill and practice keyed to these low-level tests. The teachers are feeding these kids a dumbed down curriculum to match the dumbed down tests, a dumbed down curriculum the kids in the wealthier communities do not get….

 

 

It turns out that there is one big interest that is well served by annual accountability testing. It is the interest of those who hold that the way to improve our schools is to fire the teachers whose students do not perform well on the tests. This is the mantra of the U.S. Department of Education under the Obama Administration. It is not possible to gather the data needed to fire teachers on the basis of their students’ performance unless that data is gathered every year.

 

 

The Obama Administration has managed to pit the teachers against the civil rights community on this issue and to put the teachers on the defensive. It is now said that the reason the teachers are opposing the civil rights community on annual testing is because they are seeking to evade responsibility for the performance of poor and minority students. The liberal press has bought this argument hook, line and sinker.

 

 

This is disingenuous and outrageous. Not only is it true that annual accountability testing does not improve the performance of poor and minority students, as I just explained, but it is also true that annual accountability testing is making a major contribution to the destruction of the quality of our teaching force….

 

The evaluation systems recently created has serious flaws. Their goal is to fire teachers, and those likeliest to be fired are teachers in minority communities. Meanwhile applications to professional education programs are plummeting. This is a very bad scenario for children and teachers alike; it harms teachers by putting the fear of failure in their minds, and it harms the children by giving them a stripped-down schooling and a revolving door of teachers.

 

Time to think again, says Tucker. I agree.