Gary Rubinstein has been watching the trajectory of the much-ballyhooed Tennessee Achievement School District. It was the pinnacle of reform chutzpah. Give us the lowest scoring schools in the state, said the reformers, and we will turn them into high performing schools in only five years.The basic strategy is to turn public schools over to charter operators.
That’s what they said in 2012. Five years ago. Time’s up.
Gary writes here–with full acknowledgement that “growth scores” are “garbage,” that the ASD is lagging far behind. Why use “garbage” scores? Because that the reformers’ chosen metric. The state has not released the latest scores, but the growth scores are abysmal.
He writes:
“Six years ago, the Tennessee Achievement School District (ASD) was created with the promise that within 5 years they would ‘catapult’ the schools in the bottom 5% to the top 25%. They would do this by either taking over schools or finding charter schools to take over those schools.
“Things were not looking good for the ASD four years into the experiment and then they got a reprieve in the 5th year when the state test results were nullified because of technical snafus.
“The spring 2017 test scores would settle the question about whether or not the ASD would be a success or a failure. But the test scores were not announced at the usual time, over the summer. Instead they released the high school scores a few weeks ago, which were awful for the ASD with less than 1% meeting the standard in math. A few days after that, the superintendent of the ASD, Malika Anderson, resigned after less than 2 years on the job. She had replaced ASD founder Chris Barbic, who resigned after 4 years.
“Well, the 3-8 Tennessee test scores still haven’t been released, but the other day the state released the ‘growth scores’ for the districts. Tennessee is actually the birthplace of the value added growth model and the version of it that they use is called TVAAS.
“The Achievement School District probably made a mistake in making their name something that would likely be on the top of an alphabetical list of scores. Looking at the chart from Chalkbeat Tennessee, it can be seen very clearly, that The ASD students, on average, did not ‘grow’ at least according to the magical TVAAS formula that they have so much confidence in.
“Looking at the individual school results from the state website, we see that 19 out of 29 schools in the ASD got a 1 on their overall growth for 2017. Among those schools was KIPP Memphis Prep.”
Gary will write again when the scores are released, but the prospects are not good for the schools in the ASD.
Meanwhile the ASD concept has been replicated in other states, modeled on the Tennessee ASD. I am not sure how many others have created their own ASD but North Carolina and Nevada are among them.
Diane Ravitch's blog 
Tennessee has mysteriously dropped off the list of ed reform successes and been replaced with Indianapolis.
So someone from outside The Movement should look at Indianapolis. Because I’d bet money they’re exaggerating that if it’s anything like ed reform in the rest of Indiana.
LikeLike
Tennessee was always Arne’s favorite reform paradise because it got the first Race to the Top grant of $500 million. Still waiting for the big Miracle!
LikeLike
When all that Race to the top money got to our school district, it was carefully spent so that nothing we did would be required to be funded when the money was gone. Most of my naturally conservative fellow teachers used RTT as justification for voting for Trump. RTT was a part of the Swamp thing he was talking about.
The so called measurements that Duane so expertly damns below have given TN teachers a DVam good screw for a long time now, with no end in sight. ESSA may have removed the requirement of testing that started all this, but we are staying with it and wondering why it is hard to find good teachers. School is already at the first quarter mark, and a nearby school has 6 unfilled positions.
When you take all that and add it to the Memphis poverty blues, it gets pretty bad. Through it all, if you want to meet some good people, go talk to some of the TEA members I met up at the capital a couple of years ago. They were bright, aggressive, and willing to go to the wall for their community. Hats off to those who fight for right in a tough situation.
LikeLike
ASD was the experiment where charters actually replaced public schools, right? Served the same role public schools serve- taking all comers in a zone?
Because one has to read these schemes very carefully. Ed reformers in Ohio do a kind of finesse with urban charters. They say charters are “located” in urban areas- that’s true. But unlike urban public schools in Ohio charters take about half their students from OUTSIDE urban areas. They STILL don’t outperform urban publics.
Anyone can see this. Charters pull from just about every district in Ohio. Now, obviously, if that’s true and it is, they AREN’T limited to urban areas so NOT comparable to urban publics. Ed reformers ignore this and rely on where the schools are located, NOT the students they serve. It’s deliberately deceptive.
LikeLike
This will be completely ignored by the charter cheerleading squad and the privatization push will continue:
“A new study offers a sobering answer: England’s mass conversion of primary schools to “academies,” which function in some ways like charter schools in the United States, did not produce any academic gains for students. (Incidentally, DeVos met this week with Jo Johnson, a United Kingdom education minister; a spokesperson for DeVos said the meeting focused on higher education.)
And although exporting lessons from other countries is an inherently fraught exercise, the English experience provides a cautionary tale — and aligns with research from the U.S. In short, there’s little evidence that providing schools with additional freedom will, on its own, boost student achievement.”
If our charter experiment turns out like their charter experiment will ANY of the people who cheerled this be held accountable? Or will they all retain their slots at universities and think tanks?
You all know the answer to that. Public schools will be eradicated and we’ll be told everyone was well-intentioned and no one is responsible – they’ll all remain “experts”
https://www.chalkbeat.org/posts/us/2017/09/21/does-englands-rapid-expansion-of-charter-like-academies-hold-a-lesson-for-the-u-s/
LikeLike
Does anyone know the formulas used to measure “growth” for students with disabilities, ELL status or poverty? In NY, even members of the Board of Regents say they have no idea and want to know.
If we don’t know exactly how much they adjust these scores versus other students, educators really have no way of knowing how to succeed.
Some call it “norming” or “scoring on a curve” as they compensate for high-need students by manipulating scores based on thresholds set by subjective decision making by unnamed officials.
Then, they are they kept secret, even in the face of lawsuits.
Is it because the formulas are scientifically invalid? Too blunt? (Within SWD or ELL or poverty status there are many varying levels, but are they all recognized?)
Do the formulas change from year to year? Are they ever audited? Are they updating student statuses in real time?
It’s really bad for charters to fail by their own measures, but maybe then they will see the glaring problems inherent in the idea of standardized test-based “growth” metrics that have too many real world variables to be used for anything important.
LikeLike
“Does anyone know the formulas used to measure “growth” for students . . . Is it because the formulas are scientifically invalid?”
No need to “know the formulas used to measure ‘growth'” because nothing is being measured, not “academic achievement”, not academic growth, NOTHING IS BEING MEASURED! The formulas and the concepts from which they are derived are onto-epistemologically bankrupt. Noel Wilson in his never refuted nor rebutted 1997 dissertation “Educational Standards and the Problem of Error found at: http://epaa.asu.edu/ojs/article/view/577/700 proved that all of the onto-epistemological errors, falsehoods and psychometric fudges in the process of making standards and tests render any results and/or conclusions drawn to be completely invalid (and yes, one should consider that analysis to be in the “scientific” mode).
And here is another way to analyze that supposed “measurement” of academic growth:
The most misleading concept/term in education is “measuring student achievement” or “measuring student learning”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.
There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.
But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??
Who are you to question the authorities in testing???
Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
That supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
C’mon test supporters, have at the analysis, poke holes in it, tell me where I’m wrong!
I’m expecting that I’ll still be hearing the crickets and cicadas of tinnitus instead of reading any rebuttal or refutation.
Because there is no rebuttal/refutation!
LikeLike
Duane,
I never really thought about this much (if at all) until I started reading your posts and realized that you can’t measure what you don’t precisely define.
Since they never define what they mean by academic growth (other than in terms of increased test scores, which is a circular definition), it’s not possible to perform a real measurement, not even in principle.
LikeLiked by 1 person
I agree that the testing is scientifically invalid, but my question applies to believers and skeptics alike – why would the algorithms used to adjust high needs children be hidden from view?
The test makers have a very expensive and involved process of field testing exam elements to ascribe degrees of difficulty and psychometric analysis in order to officially declare this many correct answers equals proficiency level x, this many equals y, etc.
Assuming we are forced to accept that part of testing, where “raw score” proficiency levels are set based on getting the right number of answers right, it still leaves the question about high needs students unanswered. Their scores are adjusted, depending on poverty, language and disability status. But how much? By who? And why is it secret?
This seems it should be a burning question even for supporters of testing.
LikeLike
Why?
To hide the deceptions.
Yep, it’s that plain and simple.
Those “psychometric fudges” as named by Noel Wilson (if you haven’t read his dissertation you really should), what you describe as “degrees of difficulty” and “analysis” are just using statistical jargon to appear scientific and objective. All of the questions you have are not meant to be answered by the testing adherents. Trust them, they are god-like in their psychometric knowledge.
May I suggest you also read my book, “Infidelity to Truth: Education Malpractices in American Public Education” to more completely understand just how nefarious, and I contend, unconstitutional standards and testing as used to day is. It’s available on Amazon.
LikeLike
The reason the VAMmers give is that it’s too complicated for people to understand. Math is hard, say they. Of course, we here know that if it’s “too complicated to understand”, it just doesn’t make sense. I have seen a couple of the formulas. They use skin color as a variable in the algorithm, among other problems. It’s not science. It’s just junk.
LikeLike
Einstein once said that if you can’t explain something in simple terms, you don’t really understand it yourself.
But I agree with Duane that there is something more insidious going on than mere lack of understanding or even a belief that others don’t have the math knowledge to understand.
VAMmers and others won’t reveal their detailed methods because they know that if those methods were subject to inspection by real scientists they would not withstand the scrutiny and would be exposed for the frauds that they are.
Because there is a lot of public money involved, it is fraud both in a scientific and legal sense.
LikeLiked by 1 person
William Sanders, inventor of VAM, was an agricultural statistician. He didn’t understand that measuring children’s “growth” is not the same as measuring the weight of cattle or the height of corn stalks.
LikeLike
I respectfully disagree.
Sanders understood enough about statistics to know that what he was doing was dubious (at best), which is almost certainly why he would not open his methods and computer code up to peer review.
LikeLike
LeftCoastTeacher, if you could share a link or reference to the formulas from any jurisdiction or time period, I would appreciate it. I know skin color – and gender – became a factor in NYC’s system for specifying targeted sub-groups, but I didn’t know it was ever used in a formula to convert raw scores to scale scores so you are making me even more curious!!
LikeLike
This was 3 years ago:
“On the last stop of his back-to-school bus tour through three Southern states, U.S. Secretary of Education Arne Duncan used a panel discussion Wednesday to tackle the education crisis present in so many economically devastated communities across the country.
The event took place at Cornerstone Preparatory Academy, one of 23 schools in the Achievement School District, Tennessee’s turnaround-school effort that is almost entirely located in Memphis.
Duncan’s big blue bus rolled straight into a pep rally in the school’s courtyard, where a group of about 100 elementary, middle, and high school students cheered and waived red pom-poms. ”
Duncan said it should be a “model for the country”. Luckily most of the country ignores the US Department of Education since they’re (now) completely irrelevant to public schools, but what if the country HAD taken his advice?
Is he ever going to be asked about any of it? Or does he just remain an “expert” no matter how many times he’s wrong? When do we get to the part of ed reform where we revisit some of the experiments and they are held accountable?
LikeLike
Arne is doing fine, thank you for asking. Laurene Powell Jobs hired him to work for the Emerson Collective, where he advises her about how to redesign high schools. He is a senior fellow at the usually scholarly Brookings Institution, which assumed he knew something and could write about it. Apparently they didn’t know they would have to bring on Peter Cunningham to write on Arne’s behalf. But Peter is busy pushing school choice for The 74 at Education Post.
LikeLike
I don’t have any problem with cheering charter school students. But where’s the rally for the Shelby County schools, who apparently outperformed the charter “zone”?
Why don’t they get a rally? I thought we were “agnostics” and this was all about “great schools”?
If they’re promoting charters and vouchers they should just say that. You’d have to be blind to miss the bias. This isn’t “science”- it’s marketing.
LikeLike
The whole idea that “growth is the be all and end all” (or even necessarily good) is goofy.
Cancer cells grow exponentially, but no one would suggest we try to enhance that growth.
What deformers should really be talking about is not growth but development — and mental development at that — but of course, that is really impossible to measure.
Growth (in length, weight, volume, etc) is easy to measure in the case of physical attributes, but unfortunately, learning is not a physical attribute. So as Duane Swacker points out on a regular basis, applying a growth model to learning is completely inappropriate.
But of course, it’s relatively simple to mathematize and monetize “growth” in test scores, which is why it was done.
LikeLike
This must be the year the ASD becomes voucher schools.
LikeLike
Growth = regression.
LikeLike
The Atlantic posted this. Thought some readers or Diane would like to call and make a comment. We are the experts on education and need to be heard. Here is your chance.
………
We’d like to hear your stories about education: public, private, school-of-hard-knocks, you name it. Call us up at (202)266-7600 and leave us a voicemail with your story and your answer to the question, “What is public education for?” Don’t forget to leave your contact info.
LikeLike