Archives for category: Research

This teacher thought she was doing a swell job. But then
the
ratings came out and she discovered she is the worst
tea
cher in the state! In the past, she has won many
awards, and she loves teaching. In addition: I initiated
and continue to run the chess and drama clubs with no
remuneration. I do get a small stipend for being the
academic games coordinator, running the Mathletes team and spelling
bee for the school, along with keeping the staff and students
informed of enrichment opportunities like academic
competitions. I organize the field trips for my grade
level and a trip for 4th and
5th graders to spend three days at an
oceanographic institute in the Florida Keys.

My own 5th grade
gifted students will end this year with a full understanding of
three Shakespearean plays, as class sets of these and other texts
were secured through my Donors Choose
requests. Saturday, I’ll be the designated
representative picking up free materials for my
school. I write the full year’s lesson plans over the
summer (then tweaking as I go).
She is the victim of the ceiling
effect. Her students got such high scores last year that they can’t
get higher scores this year.
She explains:
Last year, many of my students had had the
highest scores on the state tests possible the year prior—a 5 out
of 5. That’s how they get in to my class of gifted and
high achieving students. Except, last year, they
raised the bar so that the same
5th graders who scored 5s in
4th grade were much less likely to earn
5s in math and reading in
5th grade. Some still DID
score 5s in math AND reading, yet were still deemed not to have
made sufficient progress because they did not score as high within
the 5 category as they had the year before.

It’s like expecting the members of an Olympic
pole vaulting team to all individually earn gold medals every time
the Olympics come around, regardless of any other factors affecting
their lives, with the bar raised another five inches each go
around. In a state where 40% of students pass the
5th grade science test, 100% of my
students passed; but no one (at the state level) cares about
science scores.
Therefore, I suck.
How nutty is this? Why does the
U.S. Department of Education insist that states must adopt flawed
measures? Does anyone at the U.S. Department of Education consider
the consequences of their policies? Do they know anything about
research or evidence? Do they care how many people lives or
reputations they carelessly ruin with their dumb ideas?
Just wondering.

If the answer is yes, please come to one or both of the two
sessions where I am speaking on April 3. I will give the
John Dewey Society lecture at the
Convention Center, 100 Level, Room 114, from 4-7 pm. (Lots of time
for discussion). My topic: “Does Evidence
Matter?”
Fair warning: The room holds only 600
people. Before the Dewey lecture, I will join Philadelphia parent
activist Helen Gym and Carl Grant of the University of Wisconsin
(chair) in a special Presidential session from 2:15 to 3:45,
on the same level in Room 121B The
title of the session is:
Rising to the
Challenges of Quality and Equality:

The Promise of a Public
Pedagogy
If you join me at the early session,
you will have to race with me to the lecture, and the room may be
full.

One of our Marion’s leading experts on teacher evaluation, Audrey Amrein Beardsley, here evaluates Michelle Rhee’s efforts to promote her failed ideas in South Carolina.

Rhee trots out her familiar rhetoric about bad teachers and failing schools in one of the nation’s poorest states, urging them to buy her snake oil. Will they buy? Or will they do some research?

A respected researcher recently pointed out to me that there is a vast divide between most economists of education–who devoutly believe (it seems) that whatever matters can be measured, and if it can’t be measured, it doesn’t matter–and education researchers, who tend to think about the real world of students and teachers.

Here is an excellent example of the divide.

Bruce Baker takes issue with the currently fashionable idea that education can be dramatically improved by identifying the “best” teachers, giving them larger classes, and getting rid of the loser teachers.

Or, as he puts it:

“The solution to all of our woes is simple and elegant. Just follow these steps.

Step 1: Identify “really great” teachers (using your best VAM or SGP) who happen to be currently teaching inefficiently small classes of 14 to 17 students.

Step 2: Re-assign to those “really great” teachers another 12 or so students, because whatever losses might occur in relation to increased class size, the benefits of the “really great” teacher will far outweigh those losses.

Step 3: Enter underpants Gnomes.

Step 4. Test Score Awesomeness!”

He has a suggestion: Why not try the same at the fancy private and public schools?.

“One might assert that affluent suburban Westchester and Long Island districts with much smaller average class sizes should give more serious consideration to this proposal, that is, if they are a) willing to accept the assertion that they have both “bad” and “good” teachers and b) that parents in their districts are really willing to permit such experimentation with their children? I remain unconvinced.

“As for leading private independent schools which continue to use small class size as a major selling point (& differentiator from public districts), I’m currently pondering the construction of the double-decker Harkness table, to accommodate 12 students sitting on the backs of 12 others. This will be a disruptive innovation like no other!”

University of Washington scholars Wayne Au and Joseph J. Ferrare have written an excellent analysis of the big money that flooded the state of Washington to pass charter legislation in 2012. Although defeated three times before by voters, this time the proposal passed by a tiny margin. Its major funders were Bill Gates, who has no children in public school, and Walmart heiress Alice Walton,who lives in Arkansas. Substantial help was provided by other members of the Billionaire Boys Club and their claque (such as Stand for Children).

The more than $10 million they amassed was sufficient to buy what they wanted.

The moral of the story: a small number of very wealthy individuals and organizations bought a policy of their choosing. This subverts democracy. It subverts the principle of one man, one vote.

These are not reformers. They are plutocrats who use their vast wealth to buy what they want.

Here are a few choice quotes:

“Conclusions/Recommendations: This study concludes that, compared to the average voter in Washington, an elite group of wealthy individuals, either directly through individual donations or indirectly through their affiliated philanthropic organizations, wielded disproportionate influence over the outcome of the charter school initiative in the state, thereby raising serious concerns about the democratic underpinnings of an education policy that impacts all of the children in Washington State. This study also concludes that elite individuals make use of local nonprofit organizations as a mechanism to advance their education policy agenda by funding those nonprofits through the philanthropic organizations affiliated with those same wealthy elites. In light of these conclusions, the authors recommend that a mechanism for more democratic accountability be developed relative to education policy campaigns, initiatives, and legislation.

“INTRODUCTION

“To paraphrase F. Scott Fitzgerald, today’s plutocrats are not like you and I; nor do they resemble the politicians we elect. Even when they assume the authority to set public policies, they are, I fear, not sackable. (Bosworth, 2011, p. 386)

“With the backing of both major political parties, billionaire philanthropists, venture capitalists, business leaders, and a growing network of nonprofit organizations and research centers, charter school policy has evolved into a major component of the current education reform movement in the United States (Fabricant & Fine, 2012; Rawls, 2013). As of 2012, all but nine U.S. states allowed charter schools (National Alliance for Public Charter Schools, 2013), and in one of those nine, Washington State, charter school legislation was passed by popular vote in November 2012 (Reed, 2012)…..”

And more:

In this section we present the findings of our network analysis in two phases. First, through two tables, we present data on cash and in-kind contributions to the Yes On 1240 campaign and funding relationships between campaign donors, affiliated philanthropies, and organizational campaign supporters (Tables 1 and 2). Second, we visualize these relationships through a simple directed graph that traces the flows of sponsorship (material and symbolic) among policy actors (Figure 1).

YES ON 1240 CAMPAIGN CONTRIBUTIONS

Several important findings arise when we analyze the contributions to the Yes On 1240 campaign.

Table 1: Yes On I-1240 Campaign Cash and In-kind Contributions of $50k and More

Yes On 1240 Donor
Donation Amount
1.
Bill Gates Jr. – Microsoft cofounder and current chairman
$3,053,000.00
2.
Alice Walton – heiress; daughter of Walmart founder, Sam Walton
$1,700,000.00
3.
Vulcan Inc. – founded by Paul Allen, Microsoft cofounder
$1,600,000.00
4.
Nicolas Hanauer – venture capitalist
$1,000,000.00
5.
Mike Bezos – father of Amazon.com founder Jeff Bezos
$500,000.00
6.
Jackie Bezos – mother of Amazon.com founder Jeff Bezos
$500,000.00
7.
Connie Ballmer – wife of Microsoft CEO Steve Ballmer
$500,000.00
8.
Anne Dinning – managing director D.E. Shaw Investments
$250,000.00
9.
Michael Wolf – Yahoo! Inc. board of directors
$250,000.00
10.
Katherine Binder – EMFCO Holdings chairwoman
$250,000.00
11.
Eli Broad – real estate mogul
$200,000.00
12.
Benjamin Slivka – formerly Microsoft; DreamBox Learning cofounder
$124,200.00
13.
Reed Hastings – Netflix cofounder and CEO
$100,000.00
14.
Microsoft Corporation
$100,000.00
15.
Gabe Newell – formerly Microsoft; Valve Corporation cofounder
$100,000.00
16.
Doris Fisher – Gap Inc. cofounder
$100,000.00
17.
Kemper Holdings LLC – local Puget Sound developer
$110,000.00
18.
CSG Channels
$60,000.00
19.
Education Reform Now
$50,000.00
20.
Bruce McCaw –McCaw Cellular founder
$50,000.00
21.
Jolene McCaw – spouse of Bruce McCaw
$50,000.00
Source: Washington State Public Disclosure Commission (2012a)

Table 1 highlights that $10.65 million in total, or almost 98% of the $10.9 million raised for the Yes On 1240 campaign, was funded by 21 individuals and organizations who each donated more than $50,000 to the campaign (Washington State Public Disclosure Commission, 2012a).

Notably, Bill Gates Jr. is the biggest contributor ($3M) to the campaign, nearly doubling the next biggest contributions coming from Walmart heiress Alice Walton ($1.7M) and Vulcan Inc. ($1.6M),2 Microsoft cofounder Paul Allen’s company. As a more general finding, these amounts indicate that a number of select wealthy individuals with no immediate connection to Washington State (e.g., Eli Broad and Alice Walton) demonstrated a vested interest in charter school policy in the state. Another finding that emerges from the data is that wealthy individuals who are connected to the technology sector also demonstrated a vested interest in promoting charter school policy in Washington State (12 of the top 21 contributors to Yes On 1240 are strongly connected to the technology sector). Additionally, as might be expected given the interconnectedness of any sector of industry, several of these individuals have historical and industry-related connections to Microsoft Inc. and Microsoft Inc. cofounder and chairman, Bill Gates Jr.

It is also of value to highlight the $50,000.00 donation to the Yes On 1240 campaign from Education Reform Now Advocacy Committee because it illustrates the tightly woven interconnectedness of organizations and funding structures associated with education policy reform advocacy. New York State tax records from 2006 explicitly indicate that Education Reform Now, Inc., Education Reform Now Advocacy Committee, and DFER all share officers, personnel, office space, and paymasters (Libby, 2012). Tax records from 2007 further indicate that Education Reform Now Inc. and Education Reform Now Advocacy Committee share these same resources (New York State Office of the Attorney General, 2013). Thus, it is difficult to determine where DFER, Education Reform Now Inc., and Education Reform Now Advocacy Committee begin and end individually because, in essence, they represent a financially intertwined cluster of three organizations that seem to operate as a single organization with overlapping staff and resources. Consequently, even though tax records do not allow us to fully understand the exact relationship, the $50,000.00 donation to the Yes On 1240 campaign from Education Reform Now Advocacy Committee is functionally also a donation from Education Reform Now Inc. and DFER.

YES ON 1240 CONNECTED ORGANIZATIONS

As discussed above, four organizations, LEV, DFER, Stand for Children, and Partnership for Learning, publicly claimed credit for leading and coordinating the Yes On 1240 WA Coalition for Public Charter Schools (Yes On 1240, 2012a). An analysis of the in-kind donations to the Yes On 1240 campaign (that is, donations of labor or other services that are given cash value and added to the campaign donation total) supports this claim: Those four organizations predominate the in-kind donations database and are the only organizations listing “staff time” as donated in kind to the campaign (Washington State Public Disclosure Commission, 2012c). Further, as a university-based research center, they cannot be listed as having provided in-kind donations (or any donations) directly to a political campaign in the state. Because of their active role in providing direct, nonmonetary support for the Yes On 1240 campaign vis-à-vis being highlighted prominently in a campaign video (Yes On 1240, 2012b) and authoring a research report explicitly in support of I-1240 (Lake et al., 2012), we have included the CRPE here as a “connected organization” for their symbolic contribution to the campaign through the lending of their expertise.

PHILANTHROPIC CONNECTIONS TO THE YES ON 1240 CAMPAIGN

Cross referencing information gathered from the Google search engine, philanthropy websites, and available tax records (Foundation Center, 2013) produced the following 11 foundations directly connected to major donors to the Yes On 1240 campaign (in alphabetical order): Apex Foundation (formerly the Bruce & Jolene McCaw Foundation), Bezos Family Foundation, Bill & Melinda Gates Foundation, Corabelle Lumps Foundation (formerly the Anne Dinning and Michael Wolf Foundation), the Doris & Donald Fisher Fund, The Eli and Edythe Broad Foundation, the Goldman Sachs Philanthropy Fund (connected through the Connie and Steve Ballmer advised Biel Fund),3 Lochland Foundation (Katherine Binder, cofounder, officer, and contributor), The Walton Family Foundation, and Wissner-Slivka Foundation. Using foundation databases, foundation reports, available tax records, organizational websites, and institutional reports, we then looked for whether or not these foundations provided funding to the Yes On 1240 campaign-related organizations.

Table 2: Philanthropic Support for Yes On 1240 Connected Organizations

Organization

Amount

Foundation

Center on Reinventing Public Education
$8,578,000
Bill & Melinda Gates Foundation
$701,000
The Walton Family Foundation
$512,813
The Eli and Edythe Broad Foundation
Education Reform Now (Democrats for Education Reform)
$2,925,000
The Walton Family Foundation
$2,481,716
The Eli and Edythe Broad Foundation
$600,000
Doris & Donald Fisher Fund
$500,000
Corabelle Lumps Foundation
$15,000
Bezos Family Foundation
League of Education Voters
$4,790,000
Bill & Melinda Gates Foundation
$257,000
Lochland Foundation
$160,139
Bezos Family Foundation
$1,000
Apex Foundation
Partnership for Learning
$4,700,000
Bill & Melinda Gates Foundation
Stand for Children™
$9,000,000
Bill & Melinda Gates Foundation
$2,857,945
The Walton Family Foundation
$350,000
Goldman Sachs Philanthropy Fund
$120,304
Bezos Family Foundation
$55,000
Wissner-Slivka Foundation
$25,000
Lochland Foundation
$1,000
Apex Foundation
(Sources: Bill & Melinda Gates Foundation, 2013; Foundation Center, 2013; Libby, 2012; New York State Office of the Attorney General, 2013; Stand for Children, 2013; University of Washington Bothell Office of Research, 2013; University of Washington Bothell Office of Sponsored Programs, 2013)

“As Table 2 indicates, the philanthropic foundations connected to major contributors to the Yes On 1240 campaign provided a range of support directly to three of the four campaign-coordinating organizations and the CRPE: the Apex Foundation’s $1,000.00 contributions to each LEV and Stand for Children were the smallest, and the Bill & Melinda Gates Foundation’s total contribution of $9,000,000.00 to Stand for Children was the largest. Further, while DFER received no direct philanthropic support, its sister organization Education Reform Now received ample support from campaign-connected philanthropies, and, as detailed above, the overlap of resources between the cluster of Education Reform Now Inc., Education Reform Now Advocacy, and DFER, is very fluid. The Bill & Melinda Gates Foundation is the most prominent here, haven given over $27 million total to Yes On 1240 campaign-connected organizations across multiple years, grants, and contracts. The Walton Foundation is second-most prominent, having contributed $6.48 million to campaign-connected organizations, followed by the Broad Foundation at $2.99 million in support for campaign-connected organizations. There is a precipitous drop in total support after these three, potentially indicating smaller amounts of financial support originating from smaller foundations (e.g., Lochland Foundation or the Bezos Family Foundation). Regardless of the amount, foundation support of the organizations directly involved in the Yes On 1240 campaign is indicative of ideological alignment around specific education reforms (in this case, charter schools) between funders and grantees/contractors.”

The centerpiece–and the most destructive element–of Race to the Top is the insistence that teachers must be evaluated to a significant degree by the test scores of their students, whether they go up or down.

It is destructive because it makes standardized tests the purpose of education.

The tests cease to be a measure and become the aim.

That is wrong.

It leads to a narrowed curriculum, teaching to the test, and cheating.

And the measure itself is fraught with error. The teachers with high ratings one year may get low ratings the next year. Some with low ratings may get high ratings the next year. They did exactly the same things but their ratings shifted. One gets a bonus, the other gets fired. It is wrong to make the tests so consequential.

Here, if you have not read it, is an excellent summary of the VAM research, explaining why VAM is misused, by the distinguished psychometrician Edward Haertel, presented in a lecture to ETS.

You should also follow VAMboozled, which is testing expert Audrey Amrein-Beardsley’s blog. She will publish a book this spring, showing the invalidity of VAM. She points out that more than 90% of researchers in the related field agree that VAM is misused by federal policymakers.

Children are not data points; teachers do more than tests measure. Education is more than standardized tests can measure.

Neither children nor teachers nor education itself can be reduced to a metric or an algorithm.

VAM is Junk Science.

In a rational world, NCLB and the Race to the Top would be consigned–quickly–to the ash heap of history.

Hang on, friends. That day is coming.

A group of expert researchers have published a new collection of articles about teacher evaluation and high-stakes testing and their consequences.

The collection appears online in the Teachers College Record. It is called “High-Stakes Teacher Evaluation: High Cost, Big Losses.”

You will recognize the names of many of the contributors. (I wrote the foreword, the least significant part of the volume.) This collection provides solid research evidence demonstrating that the teacher evaluation practices promoted by Secretary of Education Arne Duncan are not only worthless but harmful. Sad to say, his advocacy for these misguided policies have been adopted in many states and may well damage American education for many years to come.

A while back, we saluted Bruce Baker of Rutgers University as an extraordinary truth teller and demolisher of tall tales disguised as “research”

In this post, he reveals his pet peeve:

“Perhaps more than anything else, I hate it when pundits – who often have little clue what they are talking about to begin with, toss around big numbers with lots of zeros… or “illions” attached in order to make their ideological point.”

See the tweet he refers to.

In mid-December, Matt di Carlo of the Shanker Institute reviewed the year’s production of research about charters, teacher incentives, and other aspects of the market-based approach to schooling, that is, the use of incentives and sanctions to produce higher test scores.

Schools Matter has published critiques by John Thompson of di Carlo’s review. Di Carlo is known for his scrupulous nonpartisanship; although part of the AFT, he does not take the union’s side. He writes as a researcher with no skin in the game.

In part 1, Thompson asks

“Where is the Matt DiCarlo of corporate reform? Where is a reformer who is will break ranks, for instance, on Washington D.C.’s IMPACT? There are plenty of individual reformers who have open minds. Are there any who are allowed to be like DiCarlo and acknowledge the strengths of evidence on the teachers’ side?”

Thompson is hard on di Carlo for standing back and above the fray:

Matt DiCarlo voices agreement with Gates that teachers and students should continue to be lab rats for Gates’ theories before rejecting them. If DiCarlo doesn’t believe that the jury is in and that twenty years of “reform” has failed, that’s fine. Decent people can agree to disagree.

But, I’m growing more frustrated with DiCarlo’s timidity in the face of corporate reformers singing from the same hymnal. He doesn’t challenge their assumptions and he keeps letting them define the issues. Moreover, I’m getting upset at the way he equates the arguments of test-driven reformers and teachers who resist them. It was the Billionaires Boys Club that came into school reform, denied that they had the burden of proof to show that their hypotheses would do more good than harm, and employed scorched earth politics against teachers and unions. Now, educators are criticized when we raise our voices in protest.

Above all, I am perplexed by DiCarlo’s refusal to push back on policy papers that employ sophisticated quantitative methods, but make no effort to ground their models in reality. 

Thompson adds:

The Billionaires Boys Club has tried to replace the traditional peer review process, social science and education history with Big Data. In doing so, they are trying to drive the clash of ideas from public schools.

The accountability hawks’ utilitarianism, bordering on anti-intellectualism, helps explain why high-stakes testing has failed. They imposed a radical and risky experiment without bothering to study the evidence on teaching and learning. Old Blood and Guts Bill Gates, however, says we need to endure another decade of his bubble-in experiments to see if they work. Yeah. His guts; Our blood.

In part 2 of Thompson’s critique of market-based reforms, he reviews many of the research studies on value-added measurement, showing that many of them contain warnings about the inaccuracy of VAM (one says that 68% of the ratings produce false positives and false negatives, a heck of a way to fire a teacher).

He concludes:

I would add another point which applies to nearly every aspect of market-based reforms. In schooling, the feces rolls downhill. The venom dumped on adults flows down onto children. When principals are subject to not-ready-for-prime-time accountability schemes, the #1 priority often will be to make sure the patient doesn’t die on their operating table. The search for scapegoats takes off. That is why the blame game is the prime legacy of market-driven reforms.

Part 3 of Thompson’s critique analyzes “the pitfalls that were discovered after value-added systems were implemented.” Value-added studies consistently find that teachers in high-poverty schools get less VAM than those in low-poverty schools. Is it the teachers at fault or the model or the assumptions behind the model?

Thompson concludes:

The idea that value-added evaluations could address the teacher quality equity issues in high-challenge schools is based on three assumptions. Firstly, that overworked inner city principals (who in my experience always worked over 80 hours a week, but who went weeks or months at a time without being able to allow classroom instruction to enter their consciousness, much less their “to do” list) are so craven as to allow this situation to persist, and that principals could have confidence that qualified replacements for bad teachers would apply for jobs. The answer, it is assumed, is to dump far more work on those overwhelmed principals.The second assumption is that even though it is not hard to document bad performance and fire bad teachers, that collective bargaining agreements play a significant role in protecting those teachers. In the case of the problem documented by Xu, Ozek, Hansen, and Sass, one must assume that, when a termination approaches, that mild-mannered union officials (even in Right to Work states) step into a phone book, shed their moderate, collaborative demeanor, and emerge as supermen and save the jobs of the “lemons.”

The third assumption is that the federal government should treat teachers differently from any other employees, coerce states into repudiating good faith contracts, turn the evaluation process, which traditionally evaluates what employees do or don’t do, on its head and incorporate the opinions of a few billionaires into law. It assumes that the dismissal of bad teachers would not correlate with the dismissal of “ineffective” teachers. The corollary assumption is that stripping teachers of their rights is the way to make the profession more attractive to new talent.

Even if such bizarre assumptions proved to be true, I question whether such a policy, which is collective punishment of teachers who committed to schools where it is harder to meet test score growth targets, should be considered appropriate in a constitutional democracy. 

Another installment is forthcoming. Stand by for more analysis of the perils of VAM by John Thompson!

Please read this article about an important new book by Christopher and Sarah Lubienski, scholars at the University of Illinois.

Their book is The Public School Advantage: Why Public Schools Outperform Private Schools.

The article contains an interview with Christopher Lubienski, in which he explains their “counterintuitive” findings.

Here is a sampling:

IDEAS: The thought of a “public school advantage” seems counterintuitive, and as you mention in the book it was initially a surprise to you, too. What did the data show?

LUBIENSKI: We know that private school students tend to score higher than students in public schools. But we also know that these are different populations, and they have different selection criteria. So we looked at the demographics of the different students in these nationally representative data sets, and we found those demographics more than explain the student achievement patterns….We focused specifically on mathematics, because math achievement is a better reflection of the school effects rather than the other subjects, like reading, which are often reflective of what the students are learning at home….Once we actually delved into those achievement statistics, public schools turned out to be more effective. Public school students are outscoring their demographic counterparts in private schools…at a level that is comparable to a few weeks to several months.

IDEAS: So public school students might be months ahead of their peers. And what about charter schools?

LUBIENSKI: They were already scoring beneath public schools before you control for demographics….But even once you control for those demographics, charter schools were still performing at a level lower than public schools, by as much as several months.

This is a book that Arne Duncan and every state and local superintendent should read.