William Mathis is managing director of the National Education Policy Center and a member of the Vermont Board of Education. He says that you can take the model below and apply it to any state; the result will be the same. The high schools in affluent communities are the “best,” and the high schools enrolling students in low-income communities don’t make the cut. That is about the way both NCLB and Race to the Top determined which schools needed to be closed: the schools attended by poor kids. It was knowing and heartless malpractice.
He writes:
Evaluating High Schools: Born on Third Base or hit a Triple?
If you were lucky, you missed it. But U.S. News and World Report recently committed their annual statistical malfeasance by offering up a rank order of what it proclaims as the nation’s “best” high schools. They amalgamated state tests, participation in AP/IB classes (Advanced Placement and International Baccalaureate), and graduation rates. For the tests, they weighted scores “on what matters most,” which they defined as reading and math scores. Don’t search for anything about rich, engaging curriculum; committed teachers, community involvement or piano concertos. These don’t count. And don’t search for any recognition of the reality that their “best” schools serve students who have enormous opportunities to learn outside of school while schools you won’t find on the list serve students on the opposite end of our society’s appalling opportunity gaps.
Statisticians have long protested this improper use of ordinal scales because they purport to measure something they don’t – the purpose of high schools. Few parents and fewer students would say the highlight and the most useful part of their high school experience was tests.
But again, the real problem with these – and many other – rankings is that test scores do a good job of measuring kids’ socio-economic status, but they are a pathetic measure of school quality. Yet U. S. Newsranks the nation’s schools, making the amusing distinction that the 6701stschool is better than the 6702nd.
This year, the magazine went a step further and rated schools within states. To illustrate, here’s Vermont’s top ten, in order, along with information about area wealth:
Vermont’s Best Schools: Community Income and Economic Deprivation
School Town per Percent
Capita Income Economically Challenged
|
Note that the per capita income in the “top ten” is more than $4,000 above the Vermont state average. For a family of four, this means an extra $16,000 per household. That is a sizable piece of change and likely indicates a highly educated community as much or more than it does a successful school. To avoid the senseless public shaming of rank orders, I will not list the bottom ten or the “worst schools.” But their per capita income is $27,245 – a full $10,000 per person per year less.
Looking at the low income side, 41% of the state’s children are living in economic deprivation (defined here as eligible for free and reduced lunch) but our top ten must have been born on third base, as their poverty rate is a staggering one-third lower.
The low population schools of Lakes Region, Proctor and Craftsbury made the cut even with high poverty. Missing from the list is Leland and Gray, which won a national honor for educating all the children regardless of their circumstances. They actually hit the triple. Yet they don’t get the honor.
This is not to deny the fine and diverse work of many of the state’s more affluent schools. But nationally we know that, on the whole, schools serving affluent communities almost always have strong outcomes. When children are given valuable learning resources in their communities and their homes, this shows up in their outcomes just as it does when those learning resources are in the school. Look at the concentration in Chittenden County, for example. When poverty and discrimination deny children those resources, then the load carried by their schools is much heavier. The policy lessons that adults must learn are equally weighty: we must revisit our educational programs as well as our method of evaluating (and ranking!) schools.
At the turn of the century, it became law that the secretary annually determine if all schools provide substantially equal educational opportunities to all students. The language is punitive in that it singly holds the school responsible. But doesn’t the state have an ethical obligation to make sure that all children have the right to be born on third base?
William J. Mathis is Managing Director of the National Education Policy Center and serves on the Vermont State Board of Education. The views expressed are strictly those of the author.
Diane Ravitch's blog 
Why doesn’t someone do a better ranking? Value other things and use those as a measure? No one should be relying on this one pro-reform media company as the single arbiter of “quality” anyway. It’s an echo chamber.
LikeLike
Because the problem itself is with using a ranking, a single indicator, to attempt to compare something that is inherently a complex human endeavor. It doesn’t matter whether one or the other “thing” is used. It’s a false assessment.
LikeLike
I wish I could convey in words how damaging these rankings are. Schools are considered “great” if their students come from educated, affluent families, and are set up to be closed if they enroll the neediest children. We have been engaged in one of the worst, most destructive activities of our history during these past two years of ranking schools by test scores and AP participation. I have not been able to determine how many hundreds or thousands of schools have been needlessly closed simply because their students were poor.
LikeLiked by 1 person
It’s been amusing to watch ed reformers move the goalposts with the lock-step adoption of vouchers within the “movement”. Scores are no longer valid as a measure of a school, because many of the private schools they’re funding are not “better” using a score measure than the public schools ed reformers hope to replace with private schools.
If one has been reading them for a while it’s a 180. They sound exactly like public school supporters 20 years ago- “schools are more than a score!”
It must be disconcerting to public school leaders, however. The scores ed reformers have punished them into chasing are suddenly no longer the gold standard, because that measure doesn’t fit with the ideological goals. Twenty years of scolding, threatening, and punishing public schools with test scores goes out the window when they they’re lobbying for vouchers.
It’s a bait and switch. They tricked those people. Everyone who took them at their word is made a fool. Scores, no scores, doesn’t matter. They’re privatizing anyway.
LikeLike
Now it’s all about “freedom to choose,” scores be damned. How many good public schools were forced to close because of scores?
LikeLike
Standardized test scores correlate to socioeconomic levels of families. Privatizers have used test scores to determine which schools get targeted for privatization. That is why the poorest scores with minority majority enrollments have been taken over and handed over to privatizers. This system is unfair and flawed. It has disrupted the lives of mostly minority students. Moreover, these same schools are often the schools that receive less funding from real estate taxes. The worst part of this scenario is that these same students targeted for privatization are denied the opportunity to attend a well resourced public school. Instead, this system places them in a separate and unequal school with fewer rights, and the schools are more likely to be segregated. In my opinion, the system is designed to exploit mostly poor, minority students. This segregative system has failed to deliver on its promises as these students are treated differently from other students in legitimate public schools.
Affluent schools tend to be rated higher because the students are wealthier and better prepared than those in poor schools. My children attended highly rated schools in New Jersey. These schools did not have better teachers than other school districts. They simply had many students that arrived at third base. This district continues to be considered one of the best in the state because of affluence. What is surprising to me in looking over the US New and World Report rankings of New Jersey high schools is how out of sync the ‘college and career readiness’ scores are with the number of students that attend and complete college. Some of the magnet schools score high because they are highly selective. Some of the top community schools in the state have ‘college readiness’ scores of about 65. Yet, most of these top schools send over 95% of the students to college.https://www.usnews.com/education/best-high-schools/new-jersey/rankings
I taught in an excellent diverse school district. My school won a “Blue Ribbon” at one point because our students performed better on standardized tests than other districts with comparable students. In my school we did not emphasize “test prep.” We emphasized real reading, writing and reasoning. We also were very vigilant about helping students we saw falling through the cracks. The school provided lots of supports for students as well as well trained teachers. This is an excellent school district, but district wide the scores tend to fall in the middle of the county because it is a diverse school district, Scores alone do not tell the story of a school district, and they may be misleading.
LikeLike
Hello from Hollywood, California! Ranking schools is nonsensical, but ranking teachers and schools by standardized test scores is beyond absurd. It’s a fool’s errand. Trying to capture the quality of a school or a teacher with a test is like trying to capture Jaws with a little fishing pole. You’re gonna need a bigger boat.
Trying to convince people who have been tricked into believing in test scores is like trying to convince Tom Cruise that Scientology aliens from outer space are not living in his bloodstream, controlling him. We have all the facts and logic on our side, but it just doesn’t matter. It’s a corporate-monied cult. They don’t want to be shown facts and logic; they only repeat, “Show me the money!”
People want to believe they can make it to third base on effort alone, so they believe it. Policy makers want to believe they made it to third base on their merits alone, so they believe it. Many policy makers weren’t born on third base, so they can delude themselves, not realizing they made it not with effort, but the Third Way (with help from hedge fund and tech billionaires).
I was going to request a toolkit of research or an elevator speech that would help teachers like me convince test-centric people at the local level that the students’ scores do not correlate with the quality of teachers or schools, but would it even help? I’m not so sure. After decades of pervasive pro-testing propaganda from every media outlet, especially Hollywood, I’m not so sure the test-centric zombie cult can be brought back from the living dead.
LikeLike
The so-called rankings can cause real harm to communities. The whole “Great Schools” algorithm is a perfect example. The bogus “scores” of a community aid and abet illegal red lining quietly done by real estate agents anyway.
LikeLiked by 1 person
school grade red lining: exactly
LikeLike
Very important point.
LikeLike
I just checked the GreatSchools website. They have removed from public view the “reaoning and metrics” that they have used to rank schools. Few people realize that the data on file is in the public domain, most of it gathered from state and federal longitudinal databases funded and standardized since about 2007, complemented by the B&M Gates Foundation’s funding of a massive Data Quality Campaign. This non-profit will push ads for a fee and claims to have 54 million visitors.
And this website is paid for by guess who?
https://www.greatschools.org/gk/supporters/
LikeLike
This is the big moneymaker. Take public data and sell it for a fee. Or repackage it and get advertisers.
LikeLike
Do you know what would stop this rankings in their tracks?
If every organization that put out a public school ranking was forced to do an entirely separate private school ranking and also a separate charter school ranking but all of those rankings MUST be base on exactly the same criteria used to rank public schools. Each group of schools is ranked separately, much as US News ranks research universities and liberal arts colleges separately.
I can just imagine the outcry by the wealthy parents who send their kids to the expensive and fancy private school that used to have a good reputation but now is officially noted as middling ranked because so few kids are well-educated enough to take AP exams and pass them. What if colleges believe these rankings that their child’s private school is 2nd or 3rd rate? In the charter world, I can just imagine the outcry when BASIS Charter gets top rankings and KIPP and Achievement First are shown up to be utterly mediocre at best in the separate charter school rankings list.
Then you would hear the rich billionaires whose kids are at the newly anointed 2nd and 3rd rate private school complaining that the rankings are useless. Then you would hear the rich billionaires who sit on the boards of the most mediocre charters complaining about the data used.
The only way to stop rankings is to force them on the people with power and give them a taste of their own medicine.
There is already so much privilege among the schools these billionaires send their own kids to. When their private schools are docked in the rankings because too few students take AP Exams, let those schools offer their laughable excuse that their kids are far too special to take the same AP Exams that top performing public school students take. And let their whining be ignored as the public and media insist that those parents admit that their child’s private school is 3rd rate.
LikeLike
“But doesn’t the state have an ethical obligation to make sure that all children have the right to be born on third base?”
Yes, depending on what this metaphor implies. To my way of thinking, a better image would be that all get to come to bat with plenty of hitting practice and advice. It would also help to be like The Domenican Republic, where baseball is a recreation for the culture.
we actually can learn from the number of great Domenican players. Those guys grew up like Joe Garigiola, playing in the sand lots in child-organized games. My uncle played that way until he was forty. What we need to create in education is a play culture so that people enjoy what Bob Shepard calls narrative. I would add the necessity of a culture of playing logic games and building stuff for fun.
Testing and Dicken’s Gradgrind philosophy rarely creates scholarship. What does produce the willingness to dig into a job is a clear understanding of its ramifications.
LikeLike
“But again, the real problem with these – and many other – rankings is that test scores do a good job of measuring kids’ socio-economic status, but they are a pathetic measure of school quality.”
As my mom used to say to me after repeatedly requesting me to do something far too many times: “For the umpteenth time” voice rising in frustration with my intransigence. . .
For the umpteenth time here, test scores do not measure anything, no, not even SES status. Those scores correlate with SES status, but a correlation is not a measurement.
The most misleading concept/term in education is “measuring student achievement” or “measuring socio-economic status”. The concept has been misleading educators into deluding themselves that the teaching and learning process can be analyzed/assessed using “scientific” methods which are actually pseudo-scientific at best and at worst a complete bastardization of rationo-logical thinking and language usage.
There never has been and never will be any “measuring” of the teaching and learning process and what each individual student learns in their schooling. There is and always has been assessing, evaluating, judging of what students learn but never a true “measuring” of it.
But, but, but, you’re trying to tell me that the supposedly august and venerable APA, AERA and/or the NCME have been wrong for more than the last 50 years, disseminating falsehoods and chimeras??
Who are you to question the authorities in testing???
Yes, they have been wrong and I (and many others, Wilson, Hoffman etc. . . ) question those authorities and challenge them (or any of you other advocates of the malpractices that are standards and testing) to answer to the following onto-epistemological analysis:
The TESTS MEASURE NOTHING, quite literally when you realize what is actually happening with them. Richard Phelps, a staunch standardized test proponent (he has written at least two books defending the standardized testing malpractices) in the introduction to “Correcting Fallacies About Educational and Psychological Testing” unwittingly lets the cat out of the bag with this statement:
“Physical tests, such as those conducted by engineers, can be standardized, of course [why of course of course], but in this volume , we focus on the measurement of latent (i.e., nonobservable) mental, and not physical, traits.” [my addition]
Notice how he is trying to assert by proximity that educational standardized testing and the testing done by engineers are basically the same, in other words a “truly scientific endeavor”. The same by proximity is not a good rhetorical/debating technique.
Since there is no agreement on a standard unit of learning, there is no exemplar of that standard unit and there is no measuring device calibrated against said non-existent standard unit, how is it possible to “measure the nonobservable”?
THE TESTS MEASURE NOTHING for how is it possible to “measure” the nonobservable with a non-existing measuring device that is not calibrated against a non-existing standard unit of learning?????
PURE LOGICAL INSANITY!
The basic fallacy of this is the confusing and conflating metrological (metrology is the scientific study of measurement) measuring and measuring that connotes assessing, evaluating and judging. The two meanings are not the same and confusing and conflating them is a very easy way to make it appear that standards and standardized testing are “scientific endeavors”-objective and not subjective like assessing, evaluating and judging.
Those supposedly objective results are used to justify discrimination against many students for their life circumstances and inherent intellectual traits.
LikeLike
One cannot but think of the Olympic sports with judges scores to legitimize saying that one incredible athlete is better than another. What is the point?
It reminds me of a time I was talking to Charles Wolfe (a wonderful folklorist and English professor where I went to college, and originally from Missouri). I was complaining that the fiddlers conventions should not be judged since it was so difficult to say one style was better than another. He responded by wondering how else we would get to hear the music in public. It seemed a too-easy surrender to the American tendency to make everything a competition.
Time to go read Jumping Frog.
LikeLiked by 1 person
We do in fact make competitions out of too many things, but worse, when I am judged by my students’ scores, I feel less like a performer of musical or athletic feats being rated by judges, and more like a member of the audience being rated by judges based on the performances the of musicians or athletes I’m listening to or watching. I don’t really have much control over the performances, other than cheering them on.
LikeLike
I feel the same way about awards for performances and music. It is all very subjective, and there is no one “best.” Yet, one person walks off with the Oscar, Tony or Grammy.
LikeLiked by 1 person
Vermont is typically not the go-to state to highlight disparity—but it is evident still. Have the great fortune of graduating from Mt. Mansfield Union High School, the #2 school behind Stowe. My parents made VT their home after my dad’s Air Force career. In 1980, when I graduated, we had NO testing. What we had were 8 or so towns which completely embraced their schools—and paid dearly via property taxes. Teachers often returned to teach for 30 or 40 years after college. I took 4 years of Latin and am a Latin teacher today—and still keep in touch w/ Mr. Slayton. We put on a Broadway musical each year and enjoyed band, chorus, orchestra & coding/programming. We had Varsity & JV field hockey teams, gymnastics, hockey, wrestling & soccer, basketball, baseball, track, softball & skiing—lacrosse & football teams came later. One brother took woodworking—which he does professionally—while another brother helped build a house with the Voc. Ed. program. When I went to UVM, I knew I could compete w/ the kids who went to Choate, Exeter & Deerfield.
With or without testing, schools like MMU will always be at the top. Money always matters. That remains our challenge: how to give ALL kids the opportunities I had.
LikeLike
All this comparative ranking is childish, and it’s based in numerology. One sees this kind of thing ALL THE TIME in popular culture–beauty pageants for toddlers, Quora posts about “the greatest guitarist of all time”–utter rot like that. People can only take such nonsense seriously if they lack discernment, if they are crude and stupid. Let us compare apples and oranges and shoelaces and football teams to see which is best. Crude and stupid.
LikeLike
These comparisons are the logical equivalent of “Are Rome apples better than the New England Patriots.”
LikeLike