Archives for category: Research

Gene Glass has written one of the most brilliant, most perceptive commentaries on the billionaires’ reform movement that I have ever read.

He gives a witty, well-sourced analysis of the familiar corporate reform narrative and punches giant holes in it.

Here is the opening sentence:

“A democratically run public education system in America is under siege. It is being attacked by greedy, union-hating corporations and billionaire boys whose success in business has proven to them that their circle of competence knows no bounds.”

Glass is one of our nation’s most celebrated and honored researchers. He called VAM “stupid” back in 1998. Unlike many ivory-tower academics, he is taking sides: he is on the side of public education, democracy, and truth.

If you don’t read this, shame on you.

Please tweet it, post it on Facebook, share it with your friends and your elected officials.

John Thompson, historians and teachers, assesses a discussion about the role of scholars in the current era of tumult in education.

He writes:

Education Week published essays by four scholars, Jeffrey Henig, Jay Greene, Jeannie Oakes, and Rick Hess, on the role of academic researchers in school improvement. While I respect all four contributors, and with the key points of the four commentaries, I found a part of Henig’s message http://www.edweek.org/ew/articles/2016/01/13/the-responsibility-of-edu-scholars-in-the-public.html to be unsettling, so I will get my concerns out of the way before embracing the thrust of their arguments.

Being an academic turned inner city teacher, I know the joy that can come from bringing advanced scholarship into public education. I’m not surprised by Henig’s explanation why academics would be leery of edu-politics, however, especially during this era of bitter reform wars. He writes, “Younger scholars worried that those with opposing views would wreak revenge on them.” Moreover, Henig reports:
Seasoned and secure scholars worried about being drawn into making more simplistic and extreme statements than they felt comfortable with, believing that necessary to be heard above the noisy background of claim and counterclaim. As one researcher put it to me, “Once somebody else brings a knife to the fight, you have to bring a knife to the fight, too.”

Henig correctly complains that public discourse about education has become partisan and ideological. But, I wonder what exactly does he mean when charging that the debate has become “simplistic” and “simple-minded.” And, I was downright offended by his call for “at least some reasonable voices to be heard—voices that distill and accurately reflect what research has to say.” (emphasis mine) Speaking only for our side of the reform wars, teachers and unions are not just (belatedly) bringing a metaphorical knife to the fight that was imposed on us. Our spokespersons include some of the nation’s greatest education experts and social scientists.

Although I object to the ideology of the contemporary reform movement, scholars who embrace it are very skilled in their fields (such as economic theory and data modeling) and reasonable. The ones who I have communicated with merely don’t know what they don’t know about actual schools and systems. Had they seriously contemplated the social science of the Johns Hopkins Everyone Graduates Center and the Consortium for Chicago School Research, the historical wisdom of Diane Ravitch and Larry Cuban, and the practical implementation insights of Jack Jennings and John Merrow, I can’t believe that many would have gone down the test, sort, reward, and punish path to school improvement.

In the 25 years since leaving academics for the inner city, I have repeatedly seen situations in schools and policy-making that are downright surrealistic, as well as tragic. To be blunt, scholars who do not visit with teachers and students may not have the background to determine whether an argument is simplistic or simple-minded, or whether it is an accurate identification of policies, imposed by non-educators, that are “simplistic and extreme.”
In my experience conversing with pro-reform academics dismayed by the pushback against their policies by practitioners and patrons, the issue of Common Core usually comes up. Even after we teachers had seen students denied high school diplomas because they could not pass college readiness exit exams, I would hear the claims by some who still believed that Common Core only applied to math and English. Later, policy people protested that very few 3rd graders have been denied promotion due to Common Core tests. In doing so, they ignore the obvious reality that it was the Opt Out movement and the grassroots anti-“reform” counter-attack that prevented the full implementation of Common Core high stakes tests that would have been disastrous.

So, I’d add a concrete point to Henig’s commentary. An academic who wants to help improve schools should at least see how well he fares on a Common Core GED high school equivalency math test before assuming that our positions are simplistic.

Next, Jay Greene http://www.edweek.org/ew/articles/2016/01/13/truth-telling-is-academias-privilege-and-obligation.html warns against engaging in “delicate ‘messaging’ [that] will produce a desired outcome or please a powerful patron.”

He bluntly but accurately writes:

Researchers involved in the Gates Foundation’s “Measures of Effective Teaching” study from 2009 claimed the study found that teachers are best evaluated using a formula that combines multiple measures when the research actually found no such thing.
Greene links to specific misstatements issued by the Gates Foundation, but I would make a more general point. The MET methodology would have been beneficial if the Gates Foundation had acknowledged what it was actually conducting – theoretical research. It was hopelessly inappropriate for policy research.

I still find it hard to believe that academics would bring no more than regression models to a real-world fight against the legacies of poverty and discrimination. Why would they assume that statistical models could capture the complexities of urban education?

Then, Jeannie Oakes http://www.edweek.org/ew/articles/2016/01/13/public-engagement-is-essential-to-scholarship.html and Rick Hess http://www.edweek.org/ew/articles/2016/01/13/the-responsibility-of-edu-scholars-in-the-public.html offer solid advice to scholars. Oakes cites John Dewey in urging academics to embrace “the ‘hurly burly’ of social policymaking.” She explains that, “Education policymaking must negotiate strongly held public perceptions and contested political terrain—factors usually far more influential than research findings.” Oakes then encourages public scholars to “nurture trusting and respectful relationships with policymakers and public actors. These are not one-way relationships, but reflexive.”
Rick Hess adds that there are multiple “right way(s) to think about education.” Hess affirms that, “Parents, students, community leaders, journalists, and more all have their own legitimate, valuable perspectives.” He notes, “This robust pluralism is the very foundation of the American project.”
Hess is correct that “scholars have an important role to play in that democratic cacophony, though far too few play it enthusiastically or well.” Moreover, “public debates and decisions benefit when all of our talents are brought to the table.” Academics must “connect with and learn from their fellow citizens.”

I would add that academics need to learn from each other when they engage in policy research. For the life of me, I can’t understand why so much faith was placed in regression models, and how scholars seemed to believe they could advance policy studies without thrashing out old-fashioned falsifiable hypotheses. Had quantitative and qualitative researchers joined the same table to draft hypotheses, and ask what results would be necessary to support their assumptions and put their findings into a sound narrative, we all would have benefitted. Such conversations would have identified the nuances of education issues and prompted academics to talk with other stakeholders in the ways that are proposed by the four scholars.

How many times have you read in a report or in the newspaper that X method or Y school was able to produce an extra 40 days or extra weeks of learning in reading or math?

 

How do gains in test scores get converted into days or weeks or months?

 

The answer, according to Gary Rubinstein, is that they don’t. Or they shouldn’t. It is nonsense.

 

I recently read a Mathematica Policy Research report on the Teacher Incentive Fund (merit pay), which claimed that a 1% increase in test scores was equivalent to an additional three weeks of learning. See here (study snapshot) and here (executive summary) and here (full report).

 
Performance Bonuses for Educators Led to Small Improvements in
Student Achievement

 

Educators’ understanding of bonus program improved, but challenges remain

 

New findings from Mathematica Policy Research show that a federal program providing bonuses to educators based on their performance had a small, positive impact on student achievement. In the first report to describe the effects of pay-for-performance bonuses within the Teacher Incentive Fund (TIF) program on student achievement, researchers found that student scores on standardized reading tests rose by 1 percentile point—the equivalent of about three weeks of additional learning. The study also showed similarly positive, but statistically insignificant, improvements in math.

 

I asked Gary if it made sense to translate a one-point gain into three weeks of learning, and he replied:

 

Mathematica should stop using that ‘weeks of learning’ metric. They use a calculation that says that average teachers don’t teach very much so that they maybe get the kids to increase their scores from 24 percent passing (if they did not teach anything) to 34 percent in the entire year. So each ‘point’, by that logic, amounts to about a sixth of the year. A teacher with merit pay, then who gets that extra ‘point’ would be teaching 10% more in that year which is an extra three weeks. I wish they would just give the raw score which people could relate to, like there were 50 questions on the test and students of people without merit pay got 25 correct and students of people with merit pay got 26 correct. Then people would be able to put these numbers into perspective and realize that they are not a big deal.

 

 

 

 

Blogger Chaz’s School Daze explains why the NYU study on the “success” of closing large high schools and replacing them with small high schools is bogus.

He writes:

“This week, NYU released a study showing that students fared better with the closing of the many large comprehensive high schools and replaced by the Bloomberg small schools. The basis for the study’s conclusion was the increased graduation rate from the small schools when compared to the closed schools. However, the study is fatally flawed since the graduation rate is a bogus parameter and easily manipulated by the school Principal to allow students to graduate academically unprepared for college and career. Let’s look at how schools manipulate the graduation rate.”

I posted Leonie Haimson’s critique of this Gates-funded study, which relied on the views and insights of those in charge of designing and implementing the policy in the NYC Department of Education.

Chaz points out that the study ignored the pressure on teachers in the new small schools to pass students; the pressure on principals to raise graduation rates; and the widespread use of fraudulent “credit recovery” to hand diplomas to low-performing students.

The data on graduation rates are made meaningless by these corrupt practices. The researchers did not see fit to examine nefarious ways of graduating students who were unprepared for college or careers.

Chaz points out that educators in Atlanta went to jail and lost their licenses for changing grades. Why was there no investigation or prosecution of equally serious actions in Néw York City?

 

Leonie Haimson is a fearless advocate for students, parents, and public schools. She runs a small but mighty organization called Class Size Matters (I am one of its six board members), she led the fight for student privacy that killed inBloom (the Gates’ data mining agency), and she is a board member of the Network for Public Education. None of these are paid positions. Passion beats profits.

 

In this post on the New York City parent blog, she takes a close look at a new report that lauds the Bloomberg policy of closing public schools as a “reform” strategy. The report was prepared by the Research Alliance at New York University, which was launched with the full cooperation of the by the New York City Department of Education during the Bloomberg years (Joel Klein was a member of its board when it started).

 

Haimson takes strong exception to the report’s central finding–that closing schools is good for students–and she cites a study conducted by the New School for Social Research that reached a different conclusion. (All links are in the post.)

 

Furthermore, she follows the money–who paid for the study: Gates and Ford, then Carnegie. Gates, of course, put many millions into the small schools strategy, and Carnegie employs the leader of the small schools strategy.

 

Haimson writes:

 

“The Research Alliance was founded with $3 million in Gates Foundation funds and is maintained with Carnegie Corporation funding, which help pay for this report. These two foundations promoted and helped subsidize the closing of large schools and their replacement with small schools; although the Gates Foundation has now renounced the efficacy of this policy. Michele Cahill, for many years the Vice President of the Carnegie Corporation, led this effort when she worked at DOE.

 

“The Research Alliance has also been staffed with an abundance of former DOE employees from the Bloomberg era. In the acknowledgements, the author of this new study, Jim Kemple, effusively thanks one such individual, Saskia Levy Thompson:

 

[He wrote:] ‘The author is especially grateful for the innumerable discussions with Saskia Levy Thompson about the broader context of high school reform in New York City over the past decade. Saskia’s extraordinary insights were drawn from her more than 15 years of work with the City’s schools as a practitioner at the Urban Assembly, a Research Fellow at MDRC, a Deputy Chancellor at the Department of Education and Deputy Director for the Research Alliance.’

 

Levy Thompson was Executive Director of the Urban Assembly, which supplied many of the small schools that replaced the large schools, leading to better outcomes according to this report — though one of these schools, the Urban Assembly for Civic Engagement, is now on the Renewal list.

 

After she left Urban Assembly, Levy Thompson joined MDRC as a “Research Fellow,” despite the fact that her LinkedIn profile indicates no relevant academic background or research skills. At MRDC, she “helped lead a study on the effectiveness of NYC’s small high schools,” confirming the efficacy of some of the very schools that she helped start. Here is the first of the controversial MRDC studies she co-authored in 2010, funded by the Gates Foundation, that unsurprisingly found improved outcomes at the small schools. Here is my critique of the follow-up MRDC report.

 

“In 2010, Levy Thompson left MRDC to head the DOE Portfolio Planning office, tasked with creating more small schools and finding space for them within existing buildings, which required that the large schools contract or better yet, close.

 

“And where is she now? Starting Oct. 5, Saskia Levy Thompson now runs the Carnegie Corporation’s Program for “New Designs for Schools and Systems,” under LaVerne Evans Srinivasan, another former DOE Deputy Chancellor from the Bloomberg era Here is the press release from Carnegie’s President, Vartan Gregorian:

 

“‘We are delighted that Saskia, who has played an important role in reforming America’s largest school system, is now joining the outstanding leader of Carnegie Corporation’s Education Program, LaVerne Evans Srinivasan, in overseeing our many investments in U.S. urban education.'”

 

Concludes Haimson:

 

“How cozy! In this way, a revolving door ensures that the very same DOE officials who helped close these schools continue to control the narrative, enabling them to fund — and even staff — the organizations that produce the reports that retroactively justify and help them perpetuate their policies.”

 

 

The American Educational Research Association issued a warning against the use of value added measures for high-stakes decisions regarding educators and teacher preparation programs. The cardinal rule of assessment is that tests should be used only for the purpose for which they were created. A measure of fourth grade reading measures the student, not the teacher, the principal, or the school.

 

 

AERA Issues Statement on the Use of Value-Added Models in Evaluation of Educators and Educator Preparation Programs
WASHINGTON, D.C., November 11—In a statement released today, the American Educational Research Association (AERA) advises those using or considering use of value-added models (VAM) about the scientific and technical limitations of these measures for evaluating educators and programs that prepare teachers. The statement, approved by AERA Council, cautions against the use of VAM for high-stakes decisions regarding educators.

 

In recent years, many states and districts have attempted to use VAM to determine the contributions of educators, or the programs in which they were trained, to student learning outcomes, as captured by standardized student tests. The AERA statement speaks to the formidable statistical and methodological issues involved in isolating either the effects of educators or teacher preparation programs from a complex set of factors that shape student performance.

 

“This statement draws on the leading testing, statistical, and methodological expertise in the field of education research and related sciences, and on the highest standards that guide education research and its applications in policy and practice,” said AERA Executive Director Felice J. Levine.

 

The statement addresses the challenges facing the validity of inferences from VAM, as well as specifies eight technical requirements that must be met for the use of VAM to be accurate, reliable, and valid. It cautions that these requirements cannot be met in most evaluative contexts.

 

The statement notes that, while VAM may be superior to some other models of measuring teacher impacts on student learning outcomes, “it does not mean that they are ready for use in educator or program evaluation. There are potentially serious negative consequences in the context of evaluation that can result from the use of VAM based on incomplete or flawed data, as well as from the misinterpretation or misuse of the VAM results.”

 

The statement also notes that there are promising alternatives to VAM currently in use in the United States that merit attention, including the use of teacher observation data and peer assistance and review models that provide formative and summative assessments of teaching and honor teachers’ due process rights.

 

The statement concludes: “The value of high-quality, research-based evidence cannot be over-emphasized. Ultimately, only rigorously supported inferences about the quality and effectiveness of teachers, educational leaders, and preparation programs can contribute to improved student learning.” Thus, the statement also calls for substantial investment in research on VAM and on alternative methods and models of educator and educator preparation program evaluation.

 

Related AERA Resource:
Special Issue of Educational Researcher (March 2015)—
Value Added Meets the Schools: The Effects of Using Test-Based Teacher Evaluation on the Work of Teachers and Leaders

Kenneth Zeichner and Hilary G. Conklin complain that vendors of alternative pathways into teaching have been misusing research to slam university-based teacher education. In an excerpt from a longer study, they document how organizations like Teach for America, the National Council on Teacher Quality, and the Relay “Graduate School of Education” have selectively quoted research to support their own self-interest. They seek not to improve university-based teacher education, but to replace it with entrepreneurial programs.

Zeichner is a professor of teacher education at the University of Washington, Seattle, and professor emeritus in the School of Education at the University of Wisconsin-Madison. A member of the National Academy of Education, he has done extensive research and teaching and teacher education. Conklin is a program leader and associate professor of secondary social studies at DePaul University whose research interests include teacher learning and the pedagogy of teacher education.

They write:

Critics of college and university-based teacher preparation have made many damaging claims about the programs that prepare most U.S. teachers–branding these programs as an “industry of mediocrity”–while touting the new privately-financed and- run entrepreneurial programs that are designed to replace them. These critics have constructed a narrative of failure about college and university Ed schools and a narrative of success about the entrepreneurial programs, in many cases using research evidence to support their claims.

Yet in a recent independently peer-reviewed study that will be published in Teachers College Record, we show how research has been misused in debates about the future of teacher education in the United States. Critics have labeled university teacher education programs failures and decreed their replacements successes by selectively citing research to support a particular point of view (knowledge ventriloquism), and by repeating claims based on non-existent or unvetted research, or repeatedly citing a small or unrepresentative sample of research (echo chambers).

After citing specific examples of the misuse of research, they make the following recommendations:

In order to hold all programs — public and private — to common standards of quality and evidence, we believe that several things need to be done to minimize the misuse of educational research.

First, all researchers who conduct studies that purport to offer information on the efficacy of different program models, and those who produce syntheses of studies done by others, should reveal their sources of funding, their direct and indirect links to the programs, and they should subject their work to independent and blind peer review.

Second, given that much academic research on education is inaccessible to policymakers, practitioners, and the general public, researchers should take more responsibility for communicating their findings in clear ways to various stakeholders.

Third, the media should cover claims about issues in teacher education in proportion to the strength of the evidence that stands behind them and whether or not they are supported by research that has been independently vetted.

Fourth, we should assess the quality of programs based on an analysis of a variety of costs and benefits associated with particular programs, and not just look at whose graduates can raise test scores the most. Research suggests that an emphasis only on raising test scores deepens educational inequities and continues to create a second-class system of schooling for students living in poverty.

Terry Marselle has written a research-based review of the Common Core standards. The title is: “Why the Common Core is Psychologically and Cognitively Unsound.” His book is available on amazon for FREE on Kindle from June 1-5.

Linda McNeil of Rice University has started a new blog that you should read regularly. Linda is a respected scholar of high-stakes testing and its inequitable effects. Read this paper, for example, “Avoidable Losses,” which she co-authored.

She writes:

“I want to personally invite you to visit my new blog, Educating All Our Children. This project is my response to the need I perceive for a place to bring people together around the issues of public schooling, equity and high-stakes testing. I am reaching out to you, a group that feels as passionate about equality in our schools as I do. Many of you have already shown me much support in conceptualizing this blog, for which I am very grateful.

“I have a tag line for my blog: Bringing together research, analysis, advocacy and community on behalf of the public’s schools. I’m writing in part to ask you to help me with this goal by sending me important links, leads, new stories and information that I can use to advance the movement. I would also ask that you forward, share and circulate my posts with people who share our interests. I’m not on Facebook (yet), but you can add me on Blogger and Google+ and subscribe by email to my blog.

“Thanks again for all your support over the years, and I look forward to your participation in my new venture. I hope you will visit regularly and comment often.”

Here is a sample of one of her recent posts.

Grover “Russ” Whitehurst was ousted from his job as head of the Brown Center on Education Policy at the Brookings Institution. Whitehurst had previously been research director of the Institute of Education Sciences in the George W. Bush administration. He was, of course, a big supporter of testing and choice. He even created an annual ranking of the districts with the most school choice. He gave Brookings, once known as a liberal think tank, a rightwing gloss.

In 2012, Whitehurst fired me as a Senior Fellow at Brookings, an unpaid position. I had been at Brookings since 1993. From 1993-95, I was in residence and wrote a book there on national standards. I had originally been offered the Brown Chair at Brookings but turned it down because I wanted to return to Néw York City.

Whitehurst said he was removing me from my unpaid position because I was “inactive.” At the time, my book “Death and Life of the Great American School System” was ranked as the #1 social policy book on amazon. Inactive? Hardly.

The day I was terminated, I posted on the Néw York Review of Books blog a piece that was highly critical of Mitt Romney. Whitehurst was a Romney adviser. A few hours later, I received an email from Whitehurst saying I was no longer a Senior Fellow due to my inactivity. He later said there was no connection between my anti-Romney post and his decision to drop me from a post I had held for many years, without any notice or conversation.

It will be interesting to see who replaces Whitehurst, whether liberal, conservative, neoliberal, neoconservative.