Mike Petrilli of the right-leaning Thomas B. Fordham Institute thinks that policymakers are wrong to judge schools by proficiency rates. In a thoughtful article called “The Problem with Proficiency,” he argues that it makes more sense to grade schools by whether their students show “growth.”
He offers the example of a school where the proficiency rates (passing rates on state tests) are very low but the improvement each year is impressive.
In his hypothetical, he offers this example:
Our school—let’s call it Jefferson—serves a high-poverty population of middle and high school students. Eighty-nine percent of them are eligible for a free or reduced-price lunch; 100 percent are African American or Hispanic. And on the most recent state assessment, less than a third of its students were proficient in reading or math. In some grades, fewer than 10 percent were proficient as gauged by current state standards.
But, he adds, at the same school “every year Jefferson students gain two and a half times as much in math and five times as much in English as the average school in New York City’s relatively high-performing charter sector. Its gains over time are on par or better than those of uber-high performing charters like KIPP Lynn and Geoffrey Canada’s Promise Academy.”
Now, how would you rate this school?
Gary Rubinstein recognized that Mike Petrilli was responding to the poor showing of many charter schools in New York City on the recent Common Core tests. He wrote a post called “Petrilli’s Desperate Attempt to Save Democracy Prep’s Reputation.”
Matt Di Carlo has often pointed out the problems inherent in grading schools by changes in proficiency rates. In his most recent article, he argued that:
In general, it is not a good idea to present average student performance trends in terms of proficiency rates, rather than average scores, but it is an even worse idea to use proficiency rates to measure changes in achievement gaps.
Put simply, proficiency rates have a legitimate role to play in summarizing testing data, but the rates are very sensitive to the selection of cut score, and they provide a very limited, often distorted portrayal of student performance, particularly when viewed over time. There are many ways to illustrate this distortion, but among the more vivid is the fact, which we’ve shown in previous posts, that average scores and proficiency rates often move in different directions. In other words, at the school-level, it is frequently the case that the performance of the typical student — i.e., the average score — increases while the proficiency rate decreases, or vice-versa.
Critics of the New Orleans “miracle,” on the other hand, have frequently complained that charter champions keep talking about student test score “growth” in the Recovery School District but refuse to admit that the RSD is one of the lowest-performing districts in the state of Louisiana.
Petrilli’s article provoked an extended online discussion among about 50 think tank denizens and policy wonks in D.C. and beyond, who went back and forth about what accountability should look like, how to measure it, etc.
For my part, I find myself alienated from the conversation because I see less and less value in our multi-billion investment in testing and accountability.
This was my contribution to the online discussion, which many in the conversation, no doubt, thought to be from Mars:
It really is going to take a generation to set it all right. So, we better stay focused because if we don’t reclaim those values and put them plainly in action for everyone to see and embrace, then we will destroy ourselves and all we love. Those values made a March on Washington possible 50 years ago and they want to march back into our lives today.
Hi Diane,
If your response is from Mars, I guess I’m Martian too. And so are most of the families I have worked with in my years as a teacher! Your words at the end of this post are so beautiful that I plan to print them out and frame them.
Could you please repost the details regarding your NYU speech on September 11? Also, are tickets needed? I would love to attend (even though it’s a school night). 🙂
Rick.
I think most teachers would say, “Well, duh!!
Mike
Just two words…thank you.
Your “martian” post is BRILLIANT. I cannot imagine any sane person disagreeing with it. Unfortunately in this battle for public ed, we are not fighting sane, rational people.
Thank you for lighting my fire again today.
Diane you continue to inspire me. Keep writing these posts and I will continue to copy them for my young teacher friends. They need to become a little more radical and stand up for their students and themselves. When I have grandchildren, which I hope is sooner than later, I will homeschool them rather than have them experience today’s public schools. The poor children who have no voices or advocates are the ones who will suffer in the end!
At the rate we are going, I don’t see it taking a mere generation to set it right. There is too much at stake for the rich and powerful (i.e. individuals, corporations, politicians, and ed reformers). High stakes testing is an important part of corporate education cash cow business plan. It is a key driver of the business model. High stakes testing with new standards mandates the need for new curriculum, new tests, and new means of delivery (i.e. technology). Schools failures are another important part of the business plan because it is another huge area of revenue generation for them. While it is criminal, students failing provides another important part of their business plan because this failure component feeds the prison pipeline. The politicians who rely on the rich and powerful to get elected and to provide opportunities to build wealth for themselves. The so called ed “reformers” will continue to run alongside pandering for the rich and powerful because it gives them access to power and wealth—so long as they allow themselves to be used. All of this provides the rich and powerful with greater of the same PLUS the ability to whitewash their images by increasing their ability to appear “benevolent” through their colonial philanthropy. (If they really wanted to become outstanding world citizens, they would pay their fair share of the taxes…but that is another conversation.) As those in charge become older, they will engage their children and families in this work…the beast will feed itself by repeating the cycle. Ultimately, the war on education is about class warfare. The war on public education just happens to further the cause. As a nation, we have not reached the tipping point where those exploited and harmed refuse to participate anymore. The reference of the hunger games by another respondent was both appropriate and has me hoping that it doesn’t take our society to get to that point before we say “enough” and take back our power.
I think the part where you noted the extended discussion of how to evaluate schools is key. It demonstrates that there is NO set of metrics that can do this with any degree of precision. (Petrilli’s darkly hilarious. Don’t like the way the metrics came out? Insist on a different set of metrics!)
Yet, we in the education community have to suffer through these idiotic systems that even the experts admit have no real certainty. If a discussion were to be hit from so many angles,there must not be a way to measure school success in a comparative sense.
Michigan just unleashed its new system for evaluating schools. It created a system that makes it near impossible to get into the highest category (unless your a brand new school with no prior statistical measures or an elite magnet school). No one knows how to do this and, like Common Core, there is no field testing before they subject schools and teachers and administrators to these flawed systems.
Is ther no principled way to evaluate teachers or teaching methodologies? That seems to be what you suggest with your claim that there is NO set of metrics that can evaluate schools.
Isn’t the burden on the entity prescribing a set of metrics to defend them, to prove them, to field-test them and show how accurate, prudent and non-disruptive they are?
And now Obama wants to grade colleges by including their graduation rates in the evaluation process. As it is now many H.S. grads are not college ready partly because high schools are graded in part on their graduation rates forcing them to lower their academic standards. Students that are entering college unprepared will graduate college still unprepared for college. Incredible! I don’t understand how a president who is by all accounts a smart guy can propose such transparently stupid and destructive policies?
Aren’t the NCLB waivers that are being touted as a good thing just about federalizing school report cards?
From AYP to SPP | Yinzercation
Diane, I loved your response. I have heard many parents say the same thing. To them, and to me, they type of person our children will become is more important than the type of job they will eventually have. That’s not to say that the type of job is not important, just that some people who are in charge of making the top-down decisions about our system of education seem to have gotten our priorities out of place somewhere along the way.
And of course, none of this testing is relevant in the real world — my recommendation – simplify – as I wrote up here: http://cestlaz.github.io/2013/05/09/Evaluating_Teachers.html#.UhkNLt__umg
Obviously an over simplification, but I think the basic idea is on track.
Not sure you can consider Fordham Institute RIGHT LEANING. They are heavily funded by Bill Gates.
Thomas B. Fordham Institute is a conservative think tank that supports charters, vouchers, and Common Core. Yes, they are funded by the Gates Foundation.
So to me that means they want us to “think” they are a conservative think tank. Just like Heritage that created NAFTA which is why our jobs have moved out of the country. This is my opinion but you cannot be conservative when it comes to education and support Charters/Vouchers/Choice. Charters are part of the plan to privatize education (and make a lot of people richer than they already are) and destroy local parental control (the little we still have). As we see tons of public schools converted to Charters (part of RTTT) we will see elected school boards disappear and with it the hope of any control at all. I am not a fan of elected school boards but more because the public is too lazy to get involved and make sure the bad school board members are removed and replaced with good people. I am also very saddened that not more good citizens run for school board.
Why should we rate schools?
Why should we rate schools?
This Martian has worked in educational assessment for many years and would suggest that my colleagues in the area are mostly Martians as well … and regarded as such by policy makers…
There’s a problem with “Growth,” and it is the same problem that most of our testing ‘schemes’ have – the Bell Curve. The highest-performing students, at the narrow end of the graph, cannot exhibit “Growth” because they have already reached the top of the scale.
A concrete example can be found in Carnegie Vanguard HS, Houston ISD, a magnet-only Gifted-Talented campus. Enter a District scheme for bonu$ carrots, which included a strand for “student growth.” Since a student already testing in the 99th percentile has nowhere to “grow,” the CVHS teachers could never qualify for the top bonus money. Alter the bonus scheme to a teacher-evaluation formula, and suddenly these G-T faculty members appear to be “ineffective” under [William Sanders’ black-box system].
Michael Petrilli? Michael Petrilli?
Why in the world does ANYone give this clown the time of day?
Petrilli likes to pass himself off as “one of the nation’s most trusted education analysts.” And that’s not a joke.
Although it surely is a joke that Petrilli actually has a clue. He doesn’t.
He simply passes off the conventional conservative dogma as “reform.” Little if any of it is grounded in factual reality.
Here’s some of the pap that Petrilli parrots:
He says, despite a wealth of evidence to the contrary, that “our education system is tattered. Some of it is fine, but too much is mediocre or worse.”
He attacks inner-city schools, with hardly a mention of poverty or degraded environmental conditions, and then says also that “our suburban schools are just getting by. They may not be dropout factories, but they’re not preparing anywhere near enough of their pupils to revive our economy.” As if public schools CAUSED the stagnant economy (and piled up deficits and debt) in the first place.
He whines inaccurately that “our schools haven’t improved,” and he says that “Republican governors like Chris Christie, Mitch Daniels, John Kasich, and Scott Walker are demonstrating real reform.” Say what? Is Petrilli living on the same planet as the rest of us.
Like his pals – Scott Walker, Mitch Daniels, etc. – Petrilli attacks teachers and their “unions” for “their gold-plated health care benefits or retirement pensions or lifetime job protections.”
Petrilli wants more “rigorous academic standards and tests,” and he has stated that we “should rate schools on an easy-to-understand scale, ideally from A to F, as Florida started doing under Governor Jeb Bush.” Jeb Bush? Seriously?
And Petrillli thinks the world of vouchers, saying that “one of the best ways to get more bang for the education buck is to strap it to the backs of individual kids and let parents decide which schools deliver the best value for money.”
Petrilli is a conservative charlatan. A huckster.
He might be an “analyst,” but he’s a very poor one, and most assuredly cannot be “trusted” to present an accurate picture of public education in the U.S.
So why does ANYone cite him? Ever? For anything?
Date: Sat, 24 Aug 2013 11:04:11 +0000 To: nlt_tn@hotmail.com