A group of scholars recently published a peer-reviewed study of online schooling and virtual charter schools. The authors are Brian Fitzpatrick, Mark Berends, Joseph J.Ferrare, and R. Joseph Waddington.
The University of Kentucky College of Education, where Waddington is a professor, summed up the findings:
Online Schooling’s Impact on Student Achievement
Online schooling quickly became the new normal for U.S. students when school buildings shuttered to help curb the spread of COVID-19. Although the full impact this will have on student performance will not be understood for quite some time, a study published in the April issue of the journal Educational Researcher may offer a glimpse.
Dr. Joseph Waddington, an assistant professor in the University of Kentucky College of Education Department of Educational Policy Studies and Evaluation, is part of a research team that analyzes student performance in charter schools. Virtual charter schools are offered in 21 states across the U.S., including in Indiana, where the majority of Waddington’s research data is collected.
The results of the study published in April do not bode well for virtual learning. The research team found that students who switched from traditional public schools to virtual charter schools saw test scores in mathematics and English/language arts drop substantially, and the lower scores persisted over time.
When students across the U.S. abruptly shifted to online learning, Waddington and his colleagues considered whether their performance would mirror that of students in virtual charters.
“Researchers, policymakers, teachers, school administrators, and parents alike have all been concerned about the negative consequences for student learning resulting from the dramatic shift to online instruction during COVID-19, amongst other health, safety, and socioemotional outcomes,” Waddington said. “We knew we could not directly compare virtual charter schools and the online learning taking place during COVID-19. However, we thought it would be beneficial to provide the community with a research-informed discussion of the two online learning environments, since many individuals have been eager to catch a glimpse of the potential impacts on student achievement.”
The discussion was published by Brookings, a non-profit public policy institute based in Washington D.C. It, along with the study published in Educational Researcher, was authored by Brian R. Fitzpatrick and Mark Berends at the University of Notre Dame, Joseph J. Ferrare at the University of Washington-Bothell, and Waddington at UK.
A bill allowing charter schools in Kentucky, HB 520, was signed into law in 2017. Kentucky’s charter school legislation does not allow for virtual charter schools.
In the authors’ Brookings blog Post, they explain their peer-reviewed work. The major conclusion is:
We find the impact of attending a virtual charter on student achievement is uniformly and profoundly negative, equating to a third of a standard deviation in English/language arts (ELA) and a half of a standard deviation in math. This equates to a loss of roughly 11 percentile points in ELA and 16 percentile points in math for an average virtual charter student at baseline as compared to their public school peers (see Figure 1 above). There is no evidence that virtual charter students improve in subsequent years. We could not “explain away” these findings by looking at various teacher or classroom characteristics. We also use the same methodology to analyze the impact of attending brick-and-mortar charter schools. In contrast, we find that students who attended brick-and-mortar charters have achievement no different from their traditional public school peers (see Figure 2 below). Our confidence in these results is further buoyed by other studies of virtual charter schools in Ohio and nationwide having similar findings.
If we want to dumb down a generation of students, we will make distance learning a permanent part of the landscape.
Diane Ravitch's blog 
Copies of the research should be sent to various parent groups in New York, the NY Board of Regents and Gov. Cuomo. Before he allows Gates to “reinvent” public education, Cuomo should know that the evidence is not promising for online instruction.
They all knew it was failing in Ohio, locally. Our school superintendent said her MOST difficult issue was students dropping into online charters and then back into public school and then back to online charters, over and over, where they had to be brought up to speed over and over and over. We had parents where any time there was any kind of conflict or difficulty- a poor grade, a teacher they didn’t like, not making a sports team -they would pull their child out of the public school and then the child would be back the following year.
Jeb Bush and national ed reformers were all promoting it and they continued to promote it long after most people who live here had recognized it as a disaster. No one was fired over the Ohio disaster, no one lost an election, and all the same people who promoted it are all still running education policy in this state. No accountability at all for any of them.
i do not believe Cuomo will be able to convince New York to accept an all cyber program. My gut feeling is Cuomo will try to push for blended learning, and use it as a way to extend the teaching force. I know they are going to try to eliminate costly jobs, but it remains to be seen how awful it will be. Cuomo wouldn’t be talking to Gates unless he intends to incorporate more online products into schools.
Yes, online school stinks so bad, people will not put up with it for long. Gates will try to blend the smelly online garbage with the perfume of in-person school so as to make it less offensive. There will be part time brick and mortar school. They will experiment on us with blended instruction to see how much distance we’ll put up with. Experimenting on people is Gates’ modus operandi.
I am afraid we are heading in the direction of more technology regardless of what the evidence shows. Parents are the best hope to push back against this as there are legitimate concerns about student health and privacy rights.
In Los Angeles, the superintendent (friend of billionaire Eli Broad) is planning staggered schedules with students staying home, online about half the time. I think parents are going to be far from thrilled about it. My worry is that his actions will be like the NCLB and Common Core, that no amount of upheaval after the mandate will change the mandate. That’s why it’s so important to fight to make sure all emergency measures have review and renewal dates required.
The schedule proposed in Los Angeles would be a very hard one for working parents, especially for young students. I doubt a lot of parents would want to leave middle school students on their own for half of the day, and the same can be true for some high school students as well.
LCT & rt, Can do only if a parent is working all day from home. That’s a pyramid of people w/small end middle-class & big end higher-income. For the rest [probably majority in LAUSD]: those w/young children, impossible. Middle-schoolers, maybe OK, but often not. I had NYC friends w/”latchkey children” in the days before after-school programs, & they spent a worrisome 2hrs daily on the phone & periodically rushing home early – not doable in many jobs, much less for the whole day. High-schoolers likely to leave home & gather, defeating the purpose.
I note that those Euro countries phasing into school re-opening are doing it swiftly, i.e., the whole ramp-up is planned for a 4-6wk period. Not, planning a vague, perhaps semester-long period of staggered school (as I keep hearing about here). I think this reflects (a)those countries’ far lower incidence/ infection rates to start with, (b)6 wks of having cut that low rate to near-zero & keeping it there, & (c)most important, highly-available, organized testing & contact tracing – which means they’re teaching tested, covidd-free kids, & are right on top of stats so they can strategically re-close where indicated.
The online charters Brookings studied average 100
students per class. Class size matters.
To be blunt. on-line or remote learning is asking the impossible of 99.5% of children and young adolescents. And it is asking far too much of parents. This unexpected covid experiment in on-line instruction was only for a few months and I’m sure that the compliance fell over time. Most kids found traditional classroom experiences rather repetitive and somewhat boring but most like the social interactions with friends and even teachers. The on-line experience eliminates the social and debases the instructional. Is it any wonder kids absolutely hate on-line “learning”. As Joni Mitchell once said, “Don’t it always seem to go, that you don’t know what you’ve got ’til it’s gone.
All good teachers know that vurtual teaching
is not real. Our students need real education, not a substitute for it. As an English teacher for 31 years, I believe students learn & grow from real teachers. There is no substitute.
The larger question is whether evidence matters at all. The only evidence that counts in these studies appears to be test scores in ELA and math, a common and pervasive problem in educational research. In this case, there is no evident interest in science, social studies or whether these few test scores have any bearing on the quality of the programs and very different contexts in which they operate.
There are other problems with this study, some of them mentioned in the Brookings report. For example, the Brookings report points out the scandals from profit-seeking online schools.
Readers should note that the Institute of Education Sciences has a “What Works Clearing House.” In theory that unit is devoted to studies of online instruction and charter school. All studies bearing on charter schools, a total of nine, date back to 2012 or 2015, and since 2018 there are no peer reviewers on board to review any studies of charter schools. Prior studies rated programs from KIPP and Green Dot charter schools. This is to say that IES has no current interest in studies of charter schools.
Meanwhile, the journal Educational Research (May 2020), has three articles with nine-co-authors demolishing the longstanding idea that random controlled trials (CRTs) widely promoted as a gold standard for education and earning the highest rating from the What Works Clearing House are valid…at all. The authors converge in their judgment that contexts make a huge difference in outcomes and those complications are best addressed through case studies.
The hoopla about evidence-based practice, required for federal grants under ESSA, is being blown up by these criticisms of the BEST kind of educational research…ever, with the CRT model appropriated from medical practice where, for example, CRTs are used to determine if a vaccine for COVID-19 is effective and has a very low risk of doing harm.
If you have heard a lot about evidence-based practice, that is because the Education Sciences Reform Act (ESRA) a law concurrent with NCLB in 2002 stipulated that federal officials must seek scientific proofs of effective, low-cost, user-friendly, and replicable “best practices” in education.
“Best practices” in turn were defined as: (a) aligned with national and state standards for achievement, (b) “scientifically proven” to be least costly and with best outcomes, and (c) “able to be applied, duplicated, and scaled-up” for wide use.
Scientific proof meant that evidence for best practices had to come from experimental research, with random assignments of students to “interventions,” not qualitative research alone (ESRA Title I).
The goal back then, and today, was to ensure that teachers would only use a limited set of teaching methods and other interventions, guaranteed to work and available from a USDE affiliated Web site, “What Works,” at http://www.w-w-c.org/.
The hoopla over evidence-based practice with a gold standard of “randomized control trials” (CRTs) for research, was the result of a growing dispute between researchers who were advocates for qualitative research—sharply focused on contexts for education, case studies, and “thick” nuanced descriptions of teaching and learning—versus researchers who thought the medical profession’s model of CRT’s provided a better guide to practice, and practice “at scale.”
Readers may be familiar with the “paradigm wars” that surfaced in the 1980s with two main camps–supporters of qualitative research and supporters of experimental research especially CRTs. What remains from those disputes is a lot of posturing about “the science of education,” including a quest for continuous improvement. Some researchers also hoped for prestige associated with work in “a science of education” These researcher won the day in 2002 when the new Institute for Education Sciences eliminated the older Office of Educational Research and Improvement.
A recent revision of the Procedures Handbook for the What Works Clearing House classifies “evidence-based” practice as required by ESSA with four levels evidence for “an intervention.”
–Strong evidence means there is at least one well-designed and well-implemented experimental study including CRTs.
–Moderate evidence means there must be at least one well-designed and well-implemented quasi-experimental study.
–Promising evidence means there must be at least one well-designed and well-implemented correlational study.
–Evidence that demonstrates a rationale includes a logic model informed by research that suggests improvements in outcomes are likely.
If you are teaching it is likely that all of this history, these disputes, and these nuances many not matter a whole lot unless you are required, and many teachers are, to follow evidence-based practices. The likelihood of finding much guidance at the What Works Clearing House outside of math and ELA, is very slim. Perhaps that is why so many other rating schemes have proliferated in the last two decades, including for example, those from charter school authorizers, those from EdReports for Common Core compliant resources, and those school rating schemes from the Education Trust and GreatSchools.org. You will also find a lot of teaching practices that conflate vintage 1950s training with education, as seen in this example. https://www.evidencebasedteaching.org.au/evidence-based-teaching-strategies/
That was intriguing because all I have ever known in this century is being pushed to follow “evidence-based best practices”. It’s correct to note there are other ways to highlight the detractors of online school beside test scores. Politicians seem to only understand the pseudoscience of high stakes testing, though. That is to say they seem only to understand $$$ from industry lobbyists.
Generally, having evidence that demonstrates the value of a particular approach or content is generally better than having no evidence. I would question the revision of the definition of evidence. I think they need more examples of the different ways to demonstrate evidence.
Evidence seems to be in short order when school districts buy programs or materials from vendors. Tech products are full of marketing claims, but no evidence. What we do know is that most of the online charter schools have produced the worst results of all.
Enlightening & depressing, Laura. My reaction to the posted article is probably related. I was wondering if we can ever expect to see bona fide methods for evaluating educational progress. Comparing brick& mortar to online ed via percentile points on stdzd tests is so useless. If the goal were scoring better on stdzd tests– & we’re forced to settle for a major helping of online ed until successful covid vaccine– where do such stats lead? The binary thinkers who led us down this road will come up w/boneheaded remedies like, let’s supplement w/SEL training & assessments. Or maybe, x# log-ins/ assnt-completions opens portal to fun vidgame.
I gather from your post that what I think of as “bona fide” is called qualitative research & already long ago thrown out in the bathwater in favor of CRT’s et al irrelevant methods borrowed from lab science. What strikes me is who’s calling the shots. Legislation limiting & microdefining ed research methods (ESRA – passed, no surprise, concurrently with NCLB)? I see I am very naive about the long arm of public policy. Seems a real overreach in the context of the niggardly contribution of fed to costs of public ed.
Amazing that something called the “What Works Clearinghouse” is run by bean-counters as opposed to an association of professional teachers.
Searching for valid and reliable methods for evaluating “educational progress” is a fool’s errand. There are far too many variables beyond the control of educators. However, schools can be evaluated on what they can and should provide students: educational, vocational, and extra- curricular opportunities, high quality, experienced teachers, support services and counseling, and motivation and encouragement. On-line schooling falls short in every on of these categories.
Schools should also provide routine, structure, and discipline.
“Searching for valid and reliable methods for evaluating “educational progress” is a fool’s errand. There are far too many variables beyond the control of educators. ” No actually, even your second sentence is a premise proven repeatedly in studies– educational research evaluating ed progress– whose results support experimentation with community schools, & I gather there are some studies which are revealing positive results with community schooling. And your examples of what schools should be evaluated on are all supported by research. So there is research out there that supports what teachers see daily with their own eyes. The point is as Laura says “whether evidence matters at all.” The bogus lab-borrowed CRT’s etc dictated by that ESRA law & posing as “ed research” is the fool’s errand. Kill the whole stds/ annual aligned assessment movement, & it dies a swift & natural death.
Thank you, Laura. Great analysis.
Observations with schools these last two months and what is ahead:
(After the teacher paragraph, feel free to skip right down to what we shouldn’t do).
Teachers are amazing, absolutely amazing.
“Seasoned” and new teachers alike jumped into this in varying degrees – they shared, self-taught, and figured out technology as schools and district worked similarly tracking down every student, handing out food and technology, and more. Overnight – they re-invented – even if temporarily – education far from optimum – but wow did they work at it for kids.
Blended Learning is a thing – and (hold on before you complain) not a completely bad thing.
Simply blended means in-person and virtual or live-via-online and virtual. There numerous approaches under that umbrella.
Blended learning done well (and I saw many, many examples of “done well” because of those teachers noted above) is not electronic worksheets and all drill/kill.
We can learn from this. Small groups with a teacher (guided reading) that is almost impossible with middle school schedules and size; volunteers and college student reading 1-1 and tutoring live online; sneak in project based learning 🙂 with online research projects, inventions, daily logs after planting seeds and virtulal field trips to the ocean floor (thank you National Geographic and PBS)
What we should NOT DO:
Do NOT not assess kids and start with the traditional pacing guide at grade level; rather assess AUTHENTICALLY* and meet kids where they are.
Do NOT test the kids the first day back (in person or virtual).
Do NOT test the kids the first weeks back.
Do NOT make school and teaching all about scores
Do NOT go quietly back to weeks of statewide testing.
Do NOT open emails from corporations claiming they can assess and monitor your Covid-return growth.
*Authentically Assess with authentic practices – listening to reading, reviewing problems, reading essays, good old fashioned homework and science labs, asking questions…
“Online Schooling’s Impact on Student Achievement”
Don’t give a damn about schooling’s “impact on student achievement”.
I do care about each individual’s learning and how we can advance the teaching and learning process so that each individual student can learn to his/her fullest extent and desires.
The most problematic aspect of this “new normal” is that instead of being a temporary strategy while the Covid pandemic persists, some “Education Deformers” are seeking to make the online/remote learning permanent in our public schools systems, thus destroying the traditional public school system. In Miami Dade County, our Superintendent Alberto Carvalho, is one of these individuals who is pushing to make sure that the future of public schooling is changed forever. Please watch his interview with the founder of Khan Academy, Sal Khan, on May 19th (see link below). Carvalho talks about how he saved mdcps over the past 12 years and how he will take advantage of this “opportunity” to recreate education in Miami-Dade and institute a permanent “Choice-driven model” of “blended learning” that will individualize learning and give parents the choice to put their kids permanently in remote schooling, this will reinvent our schools. This narrative is both dangerous and disappointing as it will destroy our public schools and lead to the privatization of schools as tech firms take over in-classroom teacher jobs through virtual platforms.
Home room Live with Sal and Alberto Carvalho, May 19th
de Blasio wanted to bring Carvalho to NYC. Carvalho used NYC offer as a bargaining chip to get more money in Miami. Glad NYC dodged this bullet. He should stay in Miami.
The district in which I work is smaller, but the superintendent is in the same vein. He wants us to break everyone into “small groups” in our classes, where a few are working “one on one” with the teacher, while the rest are online nearby. Which is stupid, since no kid is going to work online with no supervision, and what if something goes wrong or they get into something bad? Am I responsible for that because I couldn’t be monitoring kids because I was “working with a small group?”
hitting the nail on the head: kids NEED teachers, and the ‘experts’ endlessly arguing that children can do everything alone or in small groups while the teacher works elsewhere repeatedly show their lack of understanding
Frauds, cons, and liars that think like Donald Trump, and there are too many of them involved in online education, do not care if they are dumbing down the kids as long as they get the money. It’s all about money and has nothing to do with educating children to help them get ready for the rest of their lives.
Short version of what I thought I sent earlier.
Some observations after two months of this.
1. Teachers are amazing and if anyone had any doubts abs have kids at home they dont doubt now. They reinvented teaching overnight. Varying degrees of how well it worked but ask any teacher how much she or he had to learn, learned, and implemented in days to contact and be there for kids and try to teach beyond digital worksheets. Incredible.
Some….. some… of the tech use was good and could be adapted to beneficial blended learning models, projects, small group and one to one with a teacher, virtual field trips and more.
Equity and Access
Millions of kids WILL be home in August and September. Equity and access are non-negotiable principles so this has to be made to work with quality.. not drill/kill and in isolation.
DONT START TESTING.
(That’s the double edge sword of these studies).
Everyone is going to want to research the effects and measure anything that breathes and flashes on a screen and diagnose.
LEAVE THEM KIDS ALONE!
Start of year moratorium on testing.
Yes we need to assess where they are to meet them there … but not with a batteries of standardized tests the first day or first weeks they are back and home. Listen to them read. Ask questions. Essays. Projects. Problem sets.
No rearview mirror thinking.
Fight like crazy to not let states return to weeks of testing. Spend the year proving kids are progressing with authentic work.
I’ve been asked by a few administrators and fellow teachers what we can do to make online instruction more worthwhile. It’s understandable, searching for good ideas and truly wanting to help students during the shutdown. Unfortunately, the answer to the question, “How can we make this work better?” is: We can’t. Virtual school is not school.
Yesterday I had an interesting conversation with a recently graduated college student from a liberal arts college. She reported great frustration with attempts at on-line learning at this college, which is generally filled with motivated students above the average college student. The professors just “threw up their hands” aftr a time. The smallest glitch worked its way into the system as a problem that would sometimes take days to iron out. General ed courses were a little better, but all reported frustration with zoom and chat learning.
We have one choice for the fall. School as usual with investment in testing and tracing like nothing anyone has ever seen. Would it not be interesting to see if a really clean building with air that was tested for all sorts of allergens and contaminants might lead to a lower incidence of common cold and general absenteeism?
There’s probably something to that. My kids’ middleschool nurse had had a separate intake/ exh/ filtered air-recirc system installed in her office. During a yr when one son had chronic bronchitis, she had him spend 1 period/day in there. It cut his absences in half.
This issue of less than stellar results from online charters and now distance learning has been of great interest to me for some time. I have been a classroom teacher since 1964 and continue in this delightful activity today and this latest information comes at a time when I am completing my first distance learning experience which finished with less than stellar results for most of my 3rd and 4th graders.
As I pondered my virtual failures and read the information you have shared, I experienced a bonafide “ah-ha” moment. I know that I have always been a less than brilliant lesson designer and not a stimulating presenter (I am rather average at most things.) And, perhaps I’m rather boring to boot. That may be why I have so few of these moments. It occurred to me that I have always let the kids do the work. I set up the task, provide some support materials, and explain how one might begin and then turn it over to the kids. Then they start talking to each other and I find that at least one out of every three or four turns out to be pretty good at seeing through the muddling and shortly almost everyone is figuring the whole bumbling mess out. Today’s, or this trimester’s “ah-ha” rose suddenly out of my general confusion and made me seriously wonder if the problem with distance learning and online study achievement is simply that there are often no peers to talk to about how to do the less-than-perfect lessons I provide. I think my next distance learning attempt will require that students only work collaboratively in groups of 2-4 using “MEET” or “Zoom” which means I will have to find some clever kids in the class to help the rest figure it out.
This issue of less than stellar results from online charters and now distance learning has been of great interest to me for some time. I have been a classroom teacher since 1964 and continue in this delightful activity today and this latest information comes at a time when I am completing my first distance learning experience which finished with less than stellar results for most of my 3rd and 4th graders.
As I pondered my virtual failures and read the information you have shared, I experienced a bonafide “ah-ha” moment. I know that I have always been a less than brilliant lesson designer and not a stimulating presenter (I am rather average at most things.) And, perhaps I’m rather boring to boot. That may be why I have so few of these moments. It occurred to me that I have always let the kids do the work. I set up the task, provide some support materials, and explain how one might begin and then turn it over to the kids. Then they start talking to each other and I find that at least one out of every three or four turns out to be pretty good at seeing through the muddling and shortly almost everyone is figuring the whole bumbling mess out. Today’s, or this trimester’s “ah-ha” rose suddenly out of my general confusion and made me seriously wonder if the problem with distance learning and online study achievement is simply that there are often no peers to talk to about how to do the less-than-perfect lessons I provide. I think my next distance learning attempt will require that students only work collaboratively in groups of 2-4 using “MEET” or “Zoom” which means I will have to find some clever kids in the class to help the rest figure it out.
“the full impact this will have on student performance will not be understood for quite some time” needs to be rewritten to read
“the full impact this will have on student performance will not be understood.”
“The research team found that students who switched from traditional public schools to virtual charter schools saw test scores in mathematics and English/language arts drop substantially, and the lower scores persisted over time.”
Who cares?! The reform movement wants people to care deeply about test scores, but I think that this focus has deleterious consequences on education as a whole.
Success Academy charters have high test scores. I would not say that makes them good schools. And if other schools have lower test scores, I would not say that necessarily makes them worse schools.
Don’t use standardized test scores as the dependent variable.
I agree, Nicholas, that test scores are a misleading measure of success. But the “reformers” treat them as the Gold Standard. So they are a reasonable way to respond to Reformer claims.
Thank you for the response, Diane. Hope you are well.
Attention span and concentration levels decrease considerably when students learn online. It was is also difficult for the teacher to monitor over the students resulting in decreased outcomes.
Things will hopefully be better with long-term adjustments.