A new evaluation published by Mathematica Policy Research concluded that the School Improvement Grant strategies promoted by former Secretary of Education Arne Duncan were ineffective.
Schools that received School Improvement Grants (SIG) to implement school intervention models used more of the practices promoted by these models than schools that did not receive grants. However, the SIG-funded models had no effect on student achievement, according to a new report released by the U.S. Department of Education. Through $3.5 billion dollars in grants in 2010, the SIG program aimed to improve student achievement in the nation’s lowest-performing schools. This is the final report from the multiyear SIG evaluation led and conducted by Mathematica Policy Research, with partners American Institutes for Research and Social Policy Research Associates, for the Department of Education’s Institute of Education Sciences.
These schools implemented one of four strategies recommended by the U.S. Department of Education:
- Transformation: replace the principal, use student test scores to evaluate teachers, use data to inform instruction, and lengthen the school day or year;
- Turnaround: replace the principal, replace at least 50% of the staff, use data to inform instruction, lengthen the school day or year;
- Restart: convert to charter school;
- Closure: close the school and send students to higher-achieving schools.
“There are several possible reasons why the SIG program had no impact on student achievement,” says Lisa Dragoset, a senior researcher at Mathematica and director of the evaluation. “One possible reason is that the program did not lead to a large increase in the number of SIG-promoted practices that schools used. It is also possible that the practices were ineffective or not well implemented.”
Wow! $3.5 billion down the drain. $3.5 billion that might have been used to reduce class sizes for struggling students, that might have been used to create health clinics for needy students, that might have been used to fund orchestras and teachers of the arts.
While we are all shaking our heads over Betsy DeVos and her evangelical agenda, we have to save a few shakes of the head for the disastrous education legacy of the Obama administration, which spent billions on testing, privatization, closing schools, invalid teacher evaluations, Common Core, and other ineffective strategies.
Diane Ravitch's blog 
I find it interesting that the study concludes that the flat results may be due to not using enough intervention strategies. What they fail to mention is the impact of poverty on low performing students. Poverty is like a dirty little secret that is almost never considered on even mentioned by “reform” groups. $3.5 billion would have been better spent on reducing class size or implementing community based initiatives. As Sahlberg has said,”America does not have a failing school problem; it has a poverty problem.”
Duh! As we said as kids when another made some amazingly silly mundane and obvious statement:
No Shit Sherlock!
I wonder how many comments there are in the archives of this site alone, much less everywhere else, wherein folks “predicted” that nothing good would come from that wasteful spending? My guess thousands upon thousands upon thousands.
But, hey, why ask the practitioners and professionals what might work best in their local environment? Especially when you, Duncan and crowd believe (erroneously) that you’re the elite of the elite thinkers in this country. What do the effin peeons know? My despise of these bastards knows no bounds.
$3.5 billion is small potatoes in comparison to the $20 billion DeVos and Trump are about to squander on more poor ideas with no evidence to support their merit.
I’m agreeing with you retired teacher… I waited all day to post a comment because I figured people would be angry at what I said… But what you have said here makes sense…. If they come down the pike (from Washington) and offer your state funds (a) for SIG or (b) for Devos voucher’s …. choose SIG….. it is like a weeding where you will have chicken or fish. In 50 years I have never said hat I would prefer block grants to states but this is the first time I will say “give us block grants”…. I don’t want any of the federal funds…. (If we look back , the former commissioner of TX said that about Race to the Top… TX does not want your funding for racing)….. But our repub governor is out at the ball tonight looking for funds again (this is what happened when so many commissioners and governors signed off on Race applications and lied on the applications to get money)…. Give me block grants!!!!
The so_called study is actually junk because it gauges “achievement” at least partly on test scores.
I know that is the gold standard for the deformers, so it is right to throw that back in their face when test scores do not change, but as you have pointed out many times, there is a danger in basing a claim of “no effect” on test scores because it implicitly legitimizes an illegitimate process.
Live by scores, die by scores
Reformers hoist on their own petard
They know no other means of evaluation
A deeply essential point: so long as public school activists validate action upon test scores there can be no real change at the foundation of the bigger and much more dangerous game — being able to base educational decisions upon testing is the glue which holds the profiteering game together.
At lest the study said that one reason could have been that the interventions were not effective. That is the closest we may see to an admission that chaos is not an effective intervention in high poverty schools. Poor school communities are fragile.
Au contraire
It had several effects: enriching testing and “educational” software companies and charter operators.
And, of course, landing Arne Duncan and other deformers lucrative gigs in the private sector.
Mission accomplished!!
YEP!
You are right, SomeDAM Poet. It’s sick. Remember the DFERs!
I truly do not see a great distinction between Betsy DeVos and the last billionaire to run the U.S. Department of Education, Bill Gates. (Arne Duncan and John King were his puppets.) Both billionaires have little to no understanding of public education. Both operate with the whimsical hubris of inherited wealth. Both sway politics, independently of voters’ wishes, by bribing public officials. They call it donating. They call it philanthropy. Both seek to divert monetary support for public education and outsource all of its functions to cheaper labor sources. Both blindly worship without considering other views, one worshipping her particular religion, the other worshipping the inanimate objects he sells.
Out with the old boss…
The only difference is DeVos like the Blues Brothers is on a mission from God. https://www.youtube.com/watch?v=-4YrCFz0Kfc
Like!
“Party Animals”
The Party has changed
But party remains
And deckchairs arranged
For DavyJones chains
Good to see you in print again poet! Thanks for the eloquent reminder that the more things change, the more they are the same (with regard to bad ideas in new packages).
Has anyone ever considered that perhaps the students really can’t developmentally do any more than they are? That perhaps in our rush to implement the common core (modules if in New York) that students were not taught fundamental skills and therefore have no background from which to work? That maybe we are asking for skills that are still developmentally inappropriate? Every so called “intervention” has been to change they system that was not even broken in the first place! The students are not older or more mature learners as a result of these changes, they’re still kids. Disrupting their learning by constantly trying new systems does not promote any continuity or give any foundation for learning. None of the strategies has anything to do with the students and they are in fact detrimental to many. I wish people could wake up….
“. . . they’re still kids.”
And that right there is one of the most important facts in all of this nonsense in which it appears that few take heed of.
Muy bien dicho.
Business re-organizations– on which these SIG practices are obviously based– do not do very well either. From a Forbes 2010 article:
“Changing an organization’s structure can seem an effective way of shaking up the entire operation and thereby unlocking better performance.
“But corporate re-organizations are costly investments of time, energy and resources, and many do little to improve the business. Chrysler restructured its organization three times in the three years preceding its bankruptcy and combination with Fiat. None of those reorgs had much effect. A recent Bain & Co study of 57 major reorganizations found that fewer than one-third produced any meaningful improvement in performance. Some actually destroyed value.”
Like the debunked MBO methods informing ed-reformed teacher & school evaluation, the so-called ‘SIG practices” are just more of the warmed-over obsolete square-peg business methods imposed on round public education by the giant bus-consulting industry seeking an aftermarket for their failed products. Maybe after DeVos converts public education to a privatized free market, we can look forward to replicating the tepid “fewer than one-third meaningful improvement” business model… [>sarc<]
bethree5.
Good and relevant observations. The summary of the report says: “It is also possible that the practices were ineffective or not well implemented.”
Not mentioned: The premises were wrong and they were wrong precisely because they came from buisness models.
“Prince of Piece”
A Prince of Piece
For guns in schools
To keep the peace
When grizzly rules
*Prince is DeVos maiden name. Her brother Erik, head of Blackwater/XE (or whatever it is now called to avoid it’s past) is The Prince of Piece (“piece” being slang for gun, of course)
The SIG required certain interventions and did not give any autonomy or decision making power to the people who already worked in the school.
So for example in the school where I work, SIG required that an outside organization provide social emotional support to students- rather than supplementing the counseling and social work staff who are highly qualified and already know the students. Whenever new people come into a situation there is a long learning curve. Also people from an outside organization do not have a long term commitment to the school.
Another example, staff came in for the grant who merely measured and “coached” – what the school really needed was smaller class size, so for example another math teacher instead of a “coach.” Experienced teachers for the most part know what to do, they are just overwhelmed by the large number of students who have special issues – and they do not have support.
Hundreds of thousands of dollars were spent on technology – but the librarian and IT person were let go.
The presumption on the part of the administrators (not in the school) of the grant was that the problems in the school lay with the teachers – not with poverty, an insufficient number of qualified staff, and an unstable district.
Sara this is essentially what they are doing in Lawrence MA and they call it a “hybrid”…. in one of their “innovations” the teacher has ear buds and someone at a control panel dictates into her ears what to say to each student…. they define the problem as teacher failure, teacher unions, lazy teachers, unmotivated students and feckless parents. . That is why we get these so-called “innovations”….
Still waiting for the IES contract of similar scope and rigor to evaluate the educational outcomes of the federal Charters School Program. Or at least an IES contract to determine which schools got money & didn’t open, which schools opened and then closed or those with outcomes that can be measured because they stayed open.
dwelker
I doubt if you will ever find an evaluation report for our charter school investments, from USDE or IES.
USDE shoved money out the door for anything charter–startup, replication (franchising), and facilities and facilities financing.
Negative reviews of the grant applications were ignored. I read some of these reviews. Even the questions for the reviewers were rigged to minimize “accountability.”
We paid federal dollars for absurdities, including advertising for charter students; cross-country junkets to recruit teachers and leaders; uniforms for the students including backpacks with logos, various goodies for “awards.”
I have yet to find any reports from states back to USDE on what happened to the money channeled to states. No federal reports on schools opened, closed, etc. No credible peer-reviewed independent studies on student outcomes.
USDE let the charter authorizers and franchisers call the shots. Some of the grant applications had redacted information–unreadable chunks of text blacked out because this information was “proprietary” and might “leak,” offering a competitive advantage to other charters. USDE rolled over on that request from the grant applicants. What was redacted? Test scores and enrollments were redacted.
I just checked the 2016 active contracts of USDE bearing on charters. Only two are there, and both have been granted extensions from the original contracts.
WESTAT, INC. The purpose of this procurement is to obtain technical services for the U.S. Department of Education Charter Schools Program to support the Credit Enhancement Program with grantee monitoring. Monitoring grant projects means examining policies, systems, and procedures to ensure compliance with Federal statutes, regulations, Guidance, grant applications, and performance agreements.
Dates: 9/25/15 to 9/24/17 extended to 9/24/19
Amount: $514,554. This is a very small contract for WESTAT: It is a major subcontractor for many federal agencies.
SAFAL PARTNERS, INC. The purpose of this contract is to obtain technical assistance for the U.S. Department of Education Charter Schools Program for a range of activities, including online assistance, meetings, reports, studies, and assistance in a variety of focus areas, that COULD include human capital resources, facilities, authorizing, accountability, students with disabilities, English learners, military-connected children, and others.
Dates: from 9/27/13 to 9/26/17 extended to 9/26/18-
Amount: $12,872,533.
SAFAL Partners leads the National Charter School Resource Center, which hosts other USDE subcontractors. SAFAL Partners appears to be the go-to outfit for charter-friendly research. Clients include the Bill and Melinda Gates Foundation, Laura and John Arnold Foundation, Education Pioneers, George W. Bush Institute, Teach for America Houston among others.
A 2016 report from SAFAL Partners titled “Student Achievement in Charter Schools: What the Research Shows” is a very limited and dubious “comparison” of five studies, with caveats dismissed (e.g., fewer ELL and special education students in charter schools, only math and reading scores, test scores not from the same tests, and more). This report is more PR more than credible research.
dwelker states: “federal Charters School Program. Or at least an IES contract to determine …” there was an IES study of charters. It will take me a minute to ind it and post the results here. I came arose it when we were doing our recent League of Women Voters consensus study on charters in MA….
we have a ten page report with two pages of notes and references for our League of Women Voters consensus study. This is the quote from the IES study (they awarded the funds to Mathematic to do the research).
“IES (Institute for Educational Services, U.S.D.O.E.) statement of evaluation in Gleason, P., Clark, M., Tuttle, C. C., and Dwyer, E. (2010).
NEPC CITES MATHEMATICA STUDY The evaluation of charter school impacts: Final report (NCEE 2010-4029). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. “On average, study charter schools did not have a statistically significant impact on student achievement. However, the averages mask wide variation across the charter schools in how well their lottery winners performed compared to the lottery losers, who typically went back to their neighborhood schools. Based on the comparable scale, math achievement impacts ranged from a minus .78 for one charter to a plus .53 for another. (statistically significant variation in reading impacts was demonstrated although not as wide).”
We know that there is selection bias in the fact that students accepted into the charter are a total different population from our traditional public school population. So it is their way of controlling for “bias” when they say “lottery winners” as opposed to “lottery losers”… but this does not correct/control for all of the factors that David Berliner has told us are important.
The NEPC review is the best reading or go to the IES study (performed by Mathematica).
Thanks for the reference. I note that it is dated 2010. I also agree that NEPC reviews of research, and with clear distinctions between statistical significance and educational significance.
this came out in a Feb 9th radio interview… “It is also the case that there are a number of schools who have been put in turnaround status and been relieved from various kinds of union regulations who still have not improved. So it’s not always the unions. We have not done very well in school turnarounds.” statement is important because it is Paul Reveille who said it….
He should be held accountable. People, especially minorities are sentenced to many years in prison for much less. How is it he can get by with this?
Which schools got the money? Charters? Public?
Researching this should show which black hole the money disappeared into.
So now we now that Arne is a poorer Secretary of Education than DeVos would be. When she spent her $200 million, she bought something for it – elected officials.
So now we know
I could have saved our country all of that money wasted, before it was even spent! But no one asked the teachers at my school.
We are in our last year of the 3-year SIG money as a turnaround school. From the moment I knew what was going to happen under the grant, I told my colleagues that the whole thing is not going to make a difference, because it wasn’t created under any valid research or by teachers. But on the slim chance it made a difference, it would be because of the sole efforts of expert teachers at my school. We know how to take nothing and turn it into something brilliant.
For years before we started our grant process, we had been asking, even begging, for help with our students. We have had an increasing population year to year, of immigrant and refugee students enrolled at our school. The affects of the violence they’ve been exposed to since they were born and the obstacles of poverty, has been our nemesis. We have over-crowded classrooms, pennies for a supply budget, and no resources to provide to our students who are in desperate need of interventions. The culture of our school is violent, very low English proficiency rates, and high behavior problems due to PTSD and gang influenced families. But teachers at my school persevered as our pleas fell on deaf ears and blind eyes. As the building representative for my district association (Union), I focused on advocating for our students and teachers. People can’t believe you when you share a snippet of how a normal day goes. The absenteeism rate surpasses what is considered as “chronic”, along with a 50% mobility rate. Over 40 different languages are spoken among our students, while the culture of poverty has control over everything about a student. But yes, the rewards could be great! And teachers were dedicated, stable, cohesive, and always collaborating.
Year one of the grant timeline, we had a new principal, and about half of the faculty was new; mostly first year teachers. We all know the idea of new teachers coming into classrooms with minimal education and practical experience, would fail. Absolutely! Some of those newbies taught one year, then left the profession completely. The second year, even more of the veterans at my school decided to transfer, and another half of the teachers left as well. Now, in our last year, there are only 4 teachers left, who we consider the veterans of our school. The running joke for us is if you can teach here, you can teach anywhere! Assessment data that shows levels of mastery and benchmarks, shows that about 75% of our students rank in lower levels across the spectrum; we refer to this as our “many shades of red”, because low performing students are color-coded in red, on data spreadsheets.
But the most difficult pill to swallow in this situation, is that the majority of money is spent on the consultant groups. Really? Some expert with a Ph.D. can’t give us ideas or strategies to use with our very unique, and sometimes very volatile students and their disruptive behavior. We have an electronic program to use for documenting behavior, and it shows how much instruction time is lost due to disruptions. It’s shocking to see that the amount of time, in hours and days, is in the double digits. This is outrageous and unacceptable, but still…deaf ears and blind eyes. Despite our efforts inviting administration staff and consultants to come observe our students and see what we deal with, no individual has actually taken up our offer. I think that after they hear about it, they don’t want to see it in real time.
So as the school year is getting closer, we all know what could happen to our school if there wasn’t a high level of proficiency demonstrated among students – state takeover and turned into a charter, or simply closed down all together. Naturally, teachers are worried about what will happen, and at the point, even administration doesn’t really know what is going to happen. I also predicted that in this situation, nothing will happen. We’ll continue doing what we are doing, wondering every year if it’s the last year for our school, before being taken over. No way…no one can honestly say what will happen, but I can surely say that nothing will happen, and our school will stay open as a public K-6 school, for years to come. The building would end up being condemned before becoming a charter school. Whatever….
One last thing…teaching social studies is not always acceptable in this situation, because only writing, math, and science are tested subjects. I had to convince my principal to allow me to teach social studies. I see what our newer generation lacks in understanding and skill levels. Haven’t we seen those late shows moments when the host asks random civic questions to people on the street, and they do not know a damn thing! That is scary for me!
Wish you’d gone after these folks with the same energy you’re devoting to DeVos…
As we ready for the next ‘intervention initiatives and improvement’ ideas, we must use this information to be careful consumers of PD, strategies, ‘best-practice’, research and evidence based ideas. Retooling, renaming, and rebranding these ideas would continue to waste precious energy, time and money to the detriment of students.
MARKETING BAD IDEAS is the name of the game these days.
why is it we have precious R&D funds that are squandered. I am remembering the Right to Read grants and how then ABT came out with reports on “failing”.. and now we have a “Failing” label assigned to the SIG grants through the next administration. Of course, the most R&D squandered right now is continuing with the tests developed in monopolies to enrich the Pearson et al … This has continued through various administrations. I have also been dismayed at the direction of most of the Labs & Centers because that was our R&D funding. Formerly it would go to universities that had some truly innovative programs and had the research capacity to test out the innovations to see if they were truly cost-effective and efficient ; but it seems we have lost that total capacity (Perhaps some of the individual labs & centers are doing current work that is credible and I am just not in tune with recent happenings. I try to follow up on what IES is publishing from these programs) .
Laura, this came through my email today but I wanted to put it here because we were talking about this…
Limitations in Methodology Mar Report on School Turnarounds
Key Review Takeaway: Report provides a profile of some state-initiated school turnarounds but falls short in providing substantive guidance for policymakers.
Facebook Twitter Email
Find Documents:
Press Release: http://nepc.info/node/8490
NEPC Review: http://nepc.colorado.edu/thinktank/review-turnarounds
Report Reviewed: https://crpe.org/sites/default/files/crpe-measures-last-resort.pdf
Contact:
William J. Mathis: (802) 383-0058, wmathis@sover.net
Betty Malen: (301) 405-3587, malen@umd.edu