The National Education Policy Center published a review of a recent report about school finance, written by Rutgers Professor Bruce Baker, an expert in school finance. In the upside-down report, the states that spend the least and have the most charter schools get high rankings.
BOULDER, CO (November 27, 2018) –The Reason Foundation recently published a policy brief that offers an alternative ranking of states’ education systems. The brief, which was based on a working paper from the Department of Finance and Managerial Economics at the University of Texas at Dallas, purports to offer needed adjustments and nuance, but makes its own serious mistakes, according to a new review.
Rutgers professor Bruce D. Baker reviewed Everything You Know About State Education Rankings Is Wrong and the underlying working paper, Fixing the Currently Biased State K-12 Education Rankings. He found the analyses provided did little or nothing to advance the conversation about the effectiveness of state education systems.
The twin reports begin with the presumption that high average test scores combined with lower school spending should be the basis for state rankings, which are reasonable premises, depending upon how the analyses are approached. But the reports then head off the rails, Professor Baker explains.
Offering a ‘corrected’ representation of student outcomes and a crude analysis asserting that spending has no relation to those outcomes, the reports declare states such as New Jersey and Vermont to be poor-performing, highly inefficient systems by comparison to many states. The reports then estimate a regression model and assert that the higher performing states are those with (a) weaker teachers’ unions and (b) more children in charter schools.
However, Baker’s review details how the reports’ so-called corrections involved unreasonable and illogical assumptions and adjustments. For example, the reports re-weight racial and ethnic subgroups so that they inappropriately place equal weight in states like Vermont or Wyoming on students comprising 1 to 2% of the population as the other 98 to 99%. Other problems concern a decision to ignore economic status entirely and a poorly executed adjustment for cost of living.
Regressing multiple, highly related, interdependent measures against a specious outcome measure leads to even more suspect findings and, Baker concludes, would only mislead policymakers.
Find the review, by Bruce D. Baker, at:
http://nepc.colorado.edu/thinktank/review-rankings
Find Everything You Know About State Education Rankings Is Wrong, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Reason Foundation, at:
https://reason.com/archives/2018/10/07/everything-you-know-about-stat
Find Fixing the Currently Biased State K-12 Education Rankings, written by Stan J. Liebowitz and Matthew L. Kelly and published by the Department of Finance and Managerial Economics at the University of Texas at Dallas at:
https://ssrn.com/abstract=3185152
Diane Ravitch's blog 
I NEVER liked RANKINGS in the first place. Rankings are misleading. Thanks for this article, Diane.
LikeLike
Any study is only as valid as are the assumptions on which the study is based. False assumptions lead to false results. We have seen this type of blind allegiance to data from the “Great Schools” rankings where charters and and homogeneously white districts are considered top ranked because test scores are the primary determinant of the rankings. This misleading information is then used to red line communities.
It should not be surprising that so-called reformers that live and breathe data and rankings would come up with a system to diminish the accomplishments of states that invest in their public school systems. Like most of what “reformers” present it is based on false assumption and an erroneous conclusion. Here’s an inspiring articles that shows us why we should look at what deformers tell us with a critical lens. https://medium.com/@bardmatla/reform-resentment-983ae3271e95?fbclid=IwAR37K3yHuToEWx5xkKwjVIpgha4NnUY9lBATp_NuEc5Ge6-75YuI6XflrNA
LikeLike
The Abstract ( edited by LHC) shows the intent in this study is to make education look great in states without unions, especially “Southern states.” This study is just about bang for the buck–equated with “efficiency” in education
“State education rankings published by U.S. News and World Report, Education Week, and others, play a prominent role in legislative debate and public discourse concerning education.
….
Many states with right-to-work laws in the South and Southwest score much higher. Furthermore, we create another ranking of states based on the efficiency of education spending. In this efficiency ranking, achieving successful outcomes while economizing on education expenditures is considered better than doing so through lavish spending.
Again, Southern states that are ranked low in conventional rankings experience a reversal of fortune. Finally, our regression results indicate that unionization has a powerful negative influence on educational outcomes.”
LikeLike
I have been in the South since I retired. Good public systems in the northeast are better than many perceived good systems in south, IMHO. I have met many current and former teachers here in north Florida. While there is a core group of dedicated professionals, there are also those that take a teaching job and intend to stay for a shorter period of time due to the low pay, micromanaging and poor benefits. Low pay and a hostile work culture do not attract the best people. Yesterday, I saw a news report on technology in schools. The teacher’s comment was, “Computers is our future.” I rest my case.
States with strong unions attract a better teaching staff.
LikeLike
So, once again, a purported “research” paper will come up with a predetermined outcome, massage and manipulate the data to fit the outcome, and use inappropriate assumptions from which to make the conclusions that it wants, namely that teachers’ unions suck, charters are awesome, and it’s perfectly OK to continue to starve public education. The usual cast of ed reform charlatans will breathlessly gloam on to these “conclusions”, putting out this information into the ether, where it will remain fodder for DPEers everywhere. It boggles the mind that these so-called researchers have this much time, money, and support that they can spend coming up with this nonsense. Mr. Baker is a statistics hero; time and time again, he is able to come up with a logical analysis and reasoning as to why these “studies” are garbage and just another tool in the DPE arsenal.
LikeLike
It is meaningless to score schools based on expenditures. Overhead costs and administrative costs vary wildly from New York City to Yazoo City, Miss.
Of course, quality costs more, but comparisons of outcomes based on expenditures lead nowhere.
LikeLike
It can be done. However, as Mr. Baker points out, in this case, the researchers chose to make flawed assumptions that generate useless conclusions about spending (and a lot of other things):
“The spending adjustment compounds the reports’ problems. Ignoring or simply being unaware of the literature on regional cost variation in education, the authors choose to adjust their spending measure using a cost of living measure: “The statewide cost of living adjustments are taken from the Missouri Economic Research and Information Center’s “Cost of Living Data Series 2017 Annual Average” (p. 13).21 The vast body of research on this topic explains that reasonable models attempting to evaluate the efficiency of education dollars spent must consider three factors: a) regional variation in competitive wages, b) regional variation in population density and economies of scale, and c) variations in student needs including child poverty concentration, language proficiency and the distribution of children with mild, moderate, and severe disabilities. Most analyses fall short on the precision and completeness of these measures. These new analyses don’t even come close.”
“First, even when it comes to adjusting for regional cost variation, cost of living is not the appropriate or generally accepted measure. Rather, the National Center for Education Statistics has adopted a Comparable Wage Index approach developed by Professor Lori Taylor at Texas A&M, which remains publicly available at the district or state level.4 o Second, a plethora of additional factors, including student needs (poverty in particular), economies of scale, and population sparsity also affect the value of the education dollar (for example, significantly raising the per-pupil cost of providing schooling in states like Wyoming).”
LikeLike
Diane, “school” rankings are inherently invalid.
They’re based on the assumption that school-wide (as in school-building wide) or even individual grade level-wide statistics can sum up in an educationally meaningful and actionable manner effective and/or ineffective practices. They leave unanswered the questions: why are some children learning at an expected pace? why are some children not learning at an expected pace?
It also presumes that the principal as “educational leader” is key. That’s why in DCPS principals are required to provide 5 statistical improvement goals prior to the start of the school year. They constitute 50% of the principal’s evaluation.
It’s clear that this “school” ranking concept is taken from hedge-fund driven corporate takeovers:
– Gain control of a corporation
– Using some broad statistical measures, e.g. profitability of individual departments and/or categories of employees, “restructure”: keep what’s profitable; sell off what isn’t.
LikeLike
I agree. I have always opposed letter grades for schools, which Jeb Bush invented.
LikeLike
And I have almost always opposed letter grades for students, which almost everyone thinks are fine and dandy. Take the next step, Diane. It’s right in front of you.
LikeLike
The reports then estimate a regression model and assert that the higher performing states are those with (a) weaker teachers’ unions and (b) more children in charter schools.
What a coincidence that they found precisely what they set out to prove.
“Möbius Proof”
Möbius proof is all the rage
Make a loop from cutup page
Bend the proof back o’er with glue
That will surely prove it’s true
LikeLike
Wonderful image and perfectly matched to this and so many other reports.
LikeLike