In a rational world, we accept that knowledge, honestly applied, will set things right. But the nation’s political world is not so rational. In our current controversies, “fake news” and “alternative facts” are used to establish an ideology or to dismiss inconvenient truths. This is not a recent phenomenon. Throughout history fake news has been used to justify wars, unify people, and assault opponents. Its purpose is to invent or establish some scientific appearing veneer to justify activities which otherwise would be on shaky grounds. Unfortunately, education does not escape such politicized prevarications.

As a case in point, Donald Trump and his education secretary Betsy DeVos tout charter schools and privatization reforms.

Supplying the veneer, the Center on Education Reform (CER) headlined a recent press release, “Florida Charters Outperform Traditional Public Schools.” Citing the Florida Department of Education’s “Annual statewide analysis of student achievement” they claim that charter schools show higher achievement than traditional public schools (TPS) in 65 of 77 comparisons – or 84% of the cases.

This would be an impressive statistic if it were a true reflection of charter school performance!

But it’s not.

As a researcher and former superintendent, my first red flag is that such spectacular results are incompatible with the body of research on charter schools in general and with Florida-based research in particular. For example, researcher Matthew DiCarlo in his study, “The Evidence on the “Florida Formula” for Education Reform” noted that Florida charter schools had no impact on math scores and a negative effect on reading scores in 2013. In 2015, the prestigious CREDO study was expanded to examine seven Florida urban regions which showed “decidedly mixed” results — some scored higher and some lower. Whether high or low, the magnitude of differences was small or moderate.

These are dramatic discrepancies – from no meaningful differences in previous studies to 84% positive. A closer look indicates three glaring problems: selection effects, demographic differences, and faulty analysis.

Selection Effects – It is commonly known that parents who choose to send their children to a charter school place greater value on education and provide greater support than those who don’t participate. The accepted research adjusts for this fact. In short, selection effects could account for all the reported differences in performance. Yet, the Florida Department of Education produced a report that apparently ignores this basic truth. In their press release, the Center for Educational Reform also ignored selection effects.

Demographic Effects – Both the Florida Department of Education and CER tout the advantages of charter schools for minority children. This is misleading. As is well known, affluent children and white children score higher and do so in Florida, as well. The department’s report prominently places a demographic chart but then does not discuss the importance or relevance of these facts.

Let me explain:

• Poverty – Traditional Public Schools (TPSs) enroll a greater proportion of economically needy children. For TPSs, 61.5% of the children are on free and reduced lunch (FRL) while charter school students are more affluent (49.1% FRL).

• Students with special needs represent 14% of TPS students but only 9.4% of charter school students.

In other words, traditional public schools in Florida are serving more students with greater educational needs. Research tells us that socio-economic circumstances are the strongest predictor of test score performance. Thus the report may be measuring student demographics more than comparative school effectiveness. Like selection effects, demographics may account for all the differences.

Using the wrong numbers – Proficiency levels – Typically, the scores of students in one group would be averaged and compared to another group’s average score. But in the Florida and CER reports, they compare the percentage of children who passed a cut-off score. This hides valuable information. For instance, if a group made a great deal of progress and the average student score went up but still did not cross the cut-off threshold, this is scored as not proficient. This method has been critiqued for some time because it introduces perverse incentives like schools concentrating on students just under the cut-off. Thus, when the wrong numbers are used, great gains can be hidden and very small gains can be manipulated to look like very large gains.

So did charter schools outperform traditional public schools in Florida as the state department of education’s report says? Did charter schools achieve “remarkable results?” From the data they presented, we really don’t know. Yet, based on a strong body of knowledge, we can be certain that the comparisons touted by CER and the state are exaggerated. The weight of the evidence favors the CREDO studies — Florida’s charter schools do about the same as public schools.

This raises an ethical and government problem. The three major problems addressed here are commonly known to researchers and people who work with charter schools. Yet, the state report fails to adjust, compensate or even mention these concerns. They clearly had the data to do a proper analysis. So the real question is, are the people in the Florida Education Agency’s, Office of Independent Education and Parental Choice, partisans or just simply unaware of the research in their field?

It is hard to imagine that The Center for Educational Reform, a pro-charter advocacy group, does not know and understand these issues. Yet, they do not raise these questions in their press release. They simply repeat the state’s results.

Florida Department of Education (2017).”Student Achievement in Florida’s Charter Schools: A Comparison of the Performance of Charter School Students with Traditional Public School Students. Retrieved April 23, 2017 from http://www.fldoe.org/core/fileparse.php/7778/urlt/Charter_Student_Achievement_Report_1516.pdf