Archives for category: VAM (value-added modeling)
« Older Entries
Newer Entries »

Politico: Will the ED Department’s Office of Civil Rights Survive?

By dianeravitch
December 8, 2016 //
11

Politico speculates that the Trump administration will get rid of the Office of Civil Rights in the U.S. Department of Education. This would satisfy the hard-right, which has always objected to federal enforcement of civil rights laws. If it is not abolished outright, it might be handed over to someone who is opposed to civil rights enforcement, which seems to be an emerging pattern in Trump’s hires. The Office then might exist to cancel out existing federal enforcement activities.

 

Politico reports:

 

THE OFFICE FOR CIVIL RIGHTS’ LAST HURRAH? The Education Department’s Office for Civil Rights, which could be on the chopping block once Donald Trump takes office, is celebrating its work over the last eight years – a period in which it became significantly more aggressive than ever before. The office has cracked down on colleges that mishandle sexual assault allegations and used Title IX, a federal law that prohibits discrimination based on sex, to protect the right of transgender students to use the bathrooms and locker rooms of their choice – an issue now headed to the Supreme Court. The department this morning is releasing two new reports highlighting its work under the Obama administration at a celebration in D.C.

 

– The highlights: The office has been flooded with complaints during the Obama administration – more than 76,000 in all, with each year seeing more than the last. It has settled 66,000 of them. That work has been done with a near record-low staff of 563 full-time employees. The office had about 1,100 staff in 1981, according to the report. “Much progress has been made in the past eight years, but much work remains to ensure all children enjoy equitable access to excellence in American education,” U.S. Education Secretary John B. King Jr. said in a statement. “These two reports highlight the ongoing vital necessity of OCR’s work to eliminate discriminatory barriers to educational opportunity so our nation’s students may realize their full potential.”

 

– But the office faces an uncertain future. Civil rights groups say they’re “deeply concerned” that the extension of civil rights protections to gay and transgender students by the Obama administration will be dismantled by Betsy DeVos, who Trump has tapped to lead the Education Department. DeVos’ family has a long history of supporting anti-gay causes, POLITICO previously reported. Trump’s surrogates, meanwhile, have said there’s no need to have an Office for Civil Rights, period.

 

– Schools remain hostile environments for LGBT students, according to a new report by Human Rights Watch, a group that advocates for LGBT rights. The group conducted in-depth interviews with students, parents, teachers and administrators in Alabama, Pennsylvania, South Dakota, Texas and Utah and found that in many schools “discriminatory policies and practices exacerbate the sense of exclusion students face.” Teachers still fear for their jobs if they identify as gay or support LGBT students, according to the report. Students in same-sex couples said they were discouraged – or even prohibited – from attending events as a couple. Many schools censor discussions about LGBT topics, and eight states restrict discussions of LGBT topics in schools, according to the report.

 

– The Office for Civil Rights has also become a watchdog over colleges that mishandle investigations of sexual assault on campus. This week alone, the office opened four new investigations, bringing the list of schools currently under investigation to 219. OCR is also currently investigating some high-profile cases, such as the sexual assault cover-up by coaches and administrators at Baylor University that led the Texas school to demote its president and fire its star football coach.

 

Politico also reports on the latest from two rightwing groups that have established themselves as gatekeepers of the teaching profession, although they themselves have no credentials or authority, other than wealth:

 

REPORT: TEACHER PREP PROGRAMS MAKE PROGRESS: Nearly 900 programs preparing elementary school teachers are showing “significant progress,” particularly when it comes to how reading instruction is taught. That’s according to a new National Council on Teacher Quality review. But programs aren’t selective – a little more than a quarter of programs draw aspiring teachers from the top half of college-goers based on GPA or SAT/ACT scores, the report says. Still, programs have improved their selectivity over the years, and programs that are selective have also shown they’re diverse. More.

 

– Speaking of teachers, the Fordham Institute finds that it’s still really difficult to remove an ineffective teacher from the classroom after a decade of teacher evaluation reform. In 17 out of 25 districts studied, “state law still allows teachers to earn tenure and keep it regardless of performance.” And in most districts, an ineffective teacher’s dismissal is “extremely vulnerable” to appeal, the report says.

 

Comment: NCTQ’s standards of quality for teacher education programs is whether they are faithfully teaching the Common Core standards. Wonder if they will stick to that criterion in the age of Trump? Their definition of good reading instruction is phonics. Their judgments are not based on campus visits, but on reading catalogs and websites.

 

TBF, of course, judges teacher “effectiveness” by test scores, or value-added measurement, a method that has been debunked by scholarly associations like the American Statistical Association.

 

 

Categories Bigotry, Civil Rights, National Council on Teacher Quality (NCTQ), Standardized Testing, Teacher Evaluations, VAM (value-added modeling)

Rachel Klein: Don’t Blame Teachers for the Achievement Gap

By dianeravitch
November 12, 2016 //
24

Rachel Klein, the education editor of Huffington Post, reports on a recent study by Mathematica Policy Research that found that teachers and low-income and in upper-income schools are no different in effectiveness.

This study is very important because a central tenet of the corporate reform movement is that “bad teachers” cause low test scores, and that low-scoring schools are overrun by “bad teachers.” We have heard this claim from the mouths of Bill Gates, Arne Duncan, Michelle Rhee, Joel Klein, and every other corporate reformer. This is why Race to the Top required every state applying for a share of $4.35 billion in federal funds had to agree to evaluate teachers based in large part on the test scores of their students. Think tanks, states, and the U.S. Department of Education have poured time and energy into the pursuit of the best way to find and fire those “bad teachers.” Another of Arne Duncan’s bad ideas was the “turnaround” model, which involved firing half, or most, or all of the school staff, since in his mind the staff caused low scores.

The study used the “reformers'” favorite methodology–value-added measurement–to look for differences in teacher effectiveness.

Klein writes:

Teachers shouldn’t be held responsible for the big gap in the achievement levels of rich and poor students, new data suggests.

By looking at the effectiveness ratings of teachers who work with students from varying socioeconomic classes, Mathematica Policy Research determined that rich and poor children generally have access to equally impressive educators. The research, which was sponsored by the U.S. Department of Education, stands in the face of arguments that a more equitable distribution of teachers could substantially move metrics of educational attainment.

Affluent students outperform their low-income peers on meaningful educational benchmarks. They have higher high school graduation rates and higher standardized test scores. Policymakers have said in the past that teachers might influence this gap. Indeed, previous data shows that low-income students tend to have less access to experienced teachers.

“We know from past research that there is a very large gap in achievement between high- and low-income kids, and we also know some teachers are quite a bit more effective than others,” said Eric Isenberg, senior researcher for Mathematica. “So we were interested in exploring whether there’s a link between those two things ― if achievement gaps could be explained by low-income kids having less effective teacher than high income kids.” .

The study looked at effectiveness ratings for English language arts and math teachers in 26 districts over the course of five years. These teachers worked with students in the fourth through eighth grades.

Researchers used a value-added model to measure the effectiveness of a teacher. This statistical model is controversial in the education world ― Isenberg called it an “imperfect measure,” but he said it’s the best available option. This statistical technique is used to isolate how students’ test scores change from year to year, and how much a teacher is contributing to these changes.

Although researchers did not work with a nationally representative sample of school districts, “the study districts were chosen to be geographically diverse, with at least three districts from each of the four U.S. Census regions,” the report says. About 63 percent of the students in the studied districts qualify for free or reduced-price lunch, and the districts’ achievement gaps tend to reflect those at the national level.

Overall, researchers only found small differences in the average effectiveness ratings given to teachers working with low-income and affluent students. The average teacher of a low-income student rates around the 50th percentile, while the average teacher of a more wealthy student rates around the 51st percentile.

As Linda Darling-Hammond once wrote: “You can’t fire your way to Finland.”

In Finland, teaching is a highly prestigious profession. Entry into teachers’ colleges is very competitive. Teachers must have five years of education before they can teach. Once they are professionals, they have broad autonomy, and they collaborate with their peers to make school wide decisions. There is no standardized testing until the end of high school. There is an emphasis on the arts, play, technology, and collaborate learning. Children have a recess after every class. Teachers are professionals. They are not judged by test scores because there are no test scores.

The policies of the Bush-Obama era have failed and failed and failed. It is time to think anew.

Categories Accountability, Corporate Reformers, Education Industry, Education Reform, Teacher Education, Teacher Evaluations, Teachers and Teaching, Testing, VAM (value-added modeling)

Dilbert Explains How VAM Really Works

By dianeravitch
October 9, 2016 //
11

It takes a comedian or a cartoonist to explain the nutty world of education reform.

Check out this great cartoon by Dilbert, giving a fast explanation of the idiocy of VAM.

Enjoy!

PS: Thanks for KrazyTA for sending me the cartoon and also giving the correct link!

Categories Humor, Teacher Evaluations, VAM (value-added modeling)

Cathy O’Neil: The Fallacy of VAM as “One of Multiple Measures”

By dianeravitch
October 8, 2016 //
28

Cathy O’Neil has written s new book called “Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy.” I haven’t read it yet, but I will.

In this article, she explains that VAM is a failure and a fraud. The VAM fanatics in the federal Department of Education and state officials could not admit they were wrong, could not admit that Bill Gates had suckered the nation’s education leaders into buying his goofy data-based evaluation mania, and could not abandon the stupidity they inflicted on the nation’s teachers and schools. So they say now that VAM will be one of many measures. But why include an invalid measure at all?

As she is out on book tour, people ask questions and the most common is that VAM is only one of multiple measures.

She writes:

“Here’s an example of an argument I’ve seen consistently when it comes to the defense of the teacher value-added model (VAM) scores, and sometimes the recidivism risk scores as well. Namely, that the teacher’s VAM scores were “one of many considerations” taken to establish an overall teacher’s score. The use of something that is unfair is less unfair, in other words, if you also use other things which balance it out and are fair.

“If you don’t know what a VAM is, or what my critique about it is, take a look at this post, or read my book. The very short version is that it’s little better than a random number generator.

“The obvious irony of the “one of many” argument is, besides the mathematical one I will make below, that the VAM was supposed to actually have a real effect on teachers assessments, and that effect was meant to be valuable and objective. So any argument about it which basically implies that it’s okay to use it because it has very little power seems odd and self-defeating.

“Sometimes it’s true that a single inconsistent or badly conceived ingredient in an overall score is diluted by the other stronger and fairer assessment constituents. But I’d argue that this is not the case for how teachers’ VAM scores work in their overall teacher evaluations.

“Here’s what I learned by researching and talking to people who build teacher scores. That most of the other things they use – primarily scores derived from categorical evaluations by principals, teachers, and outsider observers – have very little variance. Almost all teachers are considered “acceptable” or “excellent” by those measurements, so they all turn into the same number or numbers when scored. That’s not a lot to work with, if the bottom 60% of teachers have essentially the same score, and you’re trying to locate the worst 2% of teachers.

“The VAM was brought in precisely to introduce variance to the overall mix. You introduce numeric VAM scores so that there’s more “spread” between teachers, so you can rank them and you’ll be sure to get teachers at the bottom.

“But if those VAM scores are actually meaningless, or at least extremely noisy, then what you have is “spread” without accuracy. And it doesn’t help to mix in the other scores.”

This is a book I want to read. Bill Gates should read it too. Send it to him and John King too. Would they read it? Not likely.

Categories Accountability, Corporate Reform, Data, Education Industry, Failure, Fraud, Teacher Evaluations, VAM (value-added modeling)

This is a Classic: The American Statistical Association Statement on Value-Added Measurement

By dianeravitch
October 5, 2016 //
21

When this statement first appeared in 2014, I said at the time that it should be on the bulletin board of every public school.

The American Statistical Association explains here why the evaluations of individual teachers should not be based on their students’ test scores.

Here is an excerpt. Read the whole statement, which is only 8 pages long:

It is unknown how full implementation of an accountability system incorporating test-based indicators, such as those derived from VAMs, will affect the actions and dispositions of teachers, principals and other educators. Perceptions of transparency, fairness and credibility will be crucial in determining the degree of success of the system as a whole in achieving its goals of improving the quality of teaching. Given the unpredictability of such complex interacting forces, it is difficult to anticipate how the education system as a whole will be affected and how the educator labor market will respond. We know from experience with other quality improvement undertakings that changes in evaluation strategy have unintended consequences. A decision to use VAMs for teacher evaluations might change the way the tests are viewed and lead to changes in the school environment. For example, more classroom time might be spent on test preparation and on specific content from the test at the exclusion of content that may lead to better long-term learning gains or motivation for students. Certain schools may be hard to staff if there is a perception that it is harder for teachers to achieve good VAM scores when working in them. Overreliance on VAM scores may foster a competitive environment, discouraging collaboration and efforts to improve the educational system as a whole.

Research on VAMs has been fairly consistent that aspects of educational effectiveness that are measurable and within teacher control represent a small part of the total variation in student test scores or growth; most estimates in the literature attribute between 1% and 14% of the total variability to teachers. This is not saying that teachers have little effect on students, but that variation among teachers accounts for a small part of the variation in scores. The majority of the variation in test scores is attributable to factors outside of the teacher’s control such as student and family background, poverty, curriculum, and unmeasured influences.

The VAM scores themselves have large standard errors, even when calculated using several years of data. These large standard errors make rankings unstable, even under the best scenarios for modeling. Combining VAMs across multiple years decreases the standard error of VAM scores. Multiple years of data, however, do not help problems caused when a model systematically undervalues teachers who work in specific contexts or with specific types of students, since that systematic undervaluation would be present in every year of data.

Despite the warning from ASA, which has no special interest and does not represent teachers or public school administrators, many states continue to use this method (called VAM, or value-added measurement or value-added modeling).

States were coerced into adopting this unproven method by the U.S. Department of Education, which said that states had to adopt it if they wanted to be eligible to compete for nearly $5 billion in federal funds in 2009, as every state was undergoing a budget crisis caused by the economic meltdown of fall 2008.

Many states adopted it, and it has not had positive effects in any state.

In Colorado and New York, among others, VAM scores count for as much as 50% of teachers’ evaluation.

A state court in New York ruled this method “arbitrary and capricious” when challenged by fourth grade teacher Sheri Lederman and her lawyer-husband Bruce Lederman.

Some states assign VAM scores to teachers based on students they never taught in subjects they don’t teach.

This is an example of federal and state policy that has no basis in evidence and that has harmed the lives of many teachers. It very likely has caused teachers to leave the profession and contributed to teacher shortages.

Categories Race to the Top, Teacher Evaluations, VAM (value-added modeling)

Should Teachers Be Evaluated by Test Scores? Does One Teacher Affect Your Lifetime Earnings?

By dianeravitch
September 23, 2016 //
42

This post, with an anonymous author, reviews the research on value-added measurement, with frequent references to those who claim that the rise or fall of test scores is the best way to judge teacher quality.

 

The basic question he or she addresses is whether the actions of your kindergarten teacher or your third-grade teacher can affect your lifetime earnings, as Raj Chetty and his team asserted in a study a few years back.

 

The author goes into a lengthy back-and-forth about whether such claims make sense.

 

But the one essential fact that his post is missing is that 70% of teachers do not teach tested subjects. A district or school can evaluate teachers with VAM only when there are enough years of test scores to document the effects of the teacher over several years. Teachers of subjects other than reading and mathematics in grades 3-8 will never get VAM ratings.

 

But many states have solved this problem by assigning VAM ratings to the 70%. Their ratings are based on the scores of students they never met in subjects they never taught. This is called an “attributed rating.”

 

That makes sense, said no one ever.

 

That may be why Hawaii and Oklahoma have dropped VAM. It is expensive and gives false positives and false negatives. Expect more states to join these two states.

Categories Teacher Evaluations, VAM (value-added modeling)

Andy Jones: Hawaii’s Education Future Is Looking Brighter

By dianeravitch
September 10, 2016 //
1

Andy Jones is a high school language arts teacher in Hawaii. The public school teachers in Hawaii have been energetic in stopping the worst extremes of the Race to the Top grant that the state received. This past spring, Hawaii dropped the test-based teacher evaluation that Race to the Top had forced on the state as a condition of winning RTTT funds. The money was all gone, and so was this bad idea.

Andy knows the history of American education, the cycles of criticism that follow one another, and he believes that the new federal law will give the state the ability to chart a better course than the one imposed by the Bush-Obama agenda of test and punish. I regret to say that his article is behind a pay wall, but I will share with you what he shared with me. In this time of doom-and-gloom, it is great to hear an optimistic forecast!

Andy writes:


Alarm, overhaul, stagnation, denial, recognition. Repeat ad infinitum.

Students of educational history will recognize in these five words the cycle that has defined American school culture for decades.

Thirty-three years ago, the “A Nation at Risk” report rang the alarm of educational decline. Its clarion call for improved public education resounded for the next generation and led eventually to initiatives such as No Child Left Behind and Race to the Top – programs that inaugurated sweeping, possibly irreversible changes to schools and school communities across the country.

A consensus has now emerged that these changes have led to dismal failure — a consensus signaled by the Every Student Succeeds Act (ESSA), which emphatically seeks to reverse the damage done, in part by giving states back the freedom to define and enact their own vision of 21st-century education.

On the heels of ESSA and the widespread discussion it has initiated, the National Conference of State Legislatures (NCSL) has released a report that may prove as decisive as “A Nation at Risk.”

The title — “No Time to Lose: How to Build a World-Class Education System State by State” — is misleading in that it seems to announce a mere repeat of the alarmist tone of the “Nation” report, perhaps to be followed by a new round of dubious policy suggestions from non-educators.

However, in what must come as a welcome shock to educators accustomed to routine governmental denial of policy failure, “No Time to Lose” fully acknowledges the mistakes of the past 15 years and seconds the sustained criticisms of prominent researchers such as Diane Ravitch and Pasi Sahlberg. These and many others have analyzed the extensive Organisation for Economic Co-operation and Development reports on international education and have concluded that the misguided “reforms” of the past years have had an overwhelmingly negative impact on American schools, leading to ever further decline internationally.

They have also highlighted an additional, sinister aspect of these “reforms,” which have involved the gradual removal of educational decisions from the purview of teachers and educators and the corresponding enrichment of educa- tional corporations profiting from the proliferation of mediocre materials and programs that schools are forced to use.

We are fortunate to be living in a state led by a governor who recognizes what is at stake and who has created a robust task force that is working to establish grassroots consensus as to what is best for Hawaii schools and the students they serve.

We are also fortunate to have an increasingly dynamic teachers union that has sponsored a teacher- written report, “Schools Our Keiki Deserve,” which echoes the advice of our top educational researchers as well as the urgent tone of “No Time to Lose.”

Hawaii Department of Education (DOE) officials have shown signs recently that they are beginning to veer away from the pattern of denial that for years has characterized state and district education departments across the country. They have, for instance, conceded the unhealthy aspects of standardized testing, and they have also begun to embrace the idea of whole-child education as practiced in the world’s top-performing school systems.

As the DOE continues revising the Strategic Plan which will guide Hawaii education over the course of the coming years, teachers and citizens should encourage DOE officials to fully embrace the sobering findings of “No Time to Lose,” the tremendous energy and wealth of ideas emerging from Gov. David Ige’s task force, and the “Schools Our Keiki Deserve” report.

The report outlines a plan that is fully in accordance with the best educational research — one that can and should be integrated into the blueprint of the document that will determine much of what happens in our schools.

Categories Hawaii, Teacher Evaluations, VAM (value-added modeling)

New Jersey Triples Weight of Test Scores in Teacher Evaluations

By dianeravitch
September 3, 2016 //
15

New Jersey has decided that teachers are now fully familiar with the Common Core and PARCC testing, even though most kids “fail” it, and henceforward the rise or fall of test scores on PARCC will count for 30% of teacher evaluations. Previously they had counted for only 10%.

This method has been debunked by the American Statistical Association and the American Educational Research Association. It has been in use in Colorado and in many states for five years without producing any results.

This is faith-based policy.

http://www.nj.com/education/2016/08/nj_parcc_tests_teacher_evaluations.html

The only sensible aspect of this change is that it counts only for teachers who teach the tested subjects in the tested grades. In neighboring New York and in other states, this discredited method applies to all teachers, and they are judged by the scores of students they didn’t teach in subjects they don’t teach.

In New York, an outstanding fourth grade teacher, Sheri Lederman, sued the state after receiving a low rating. The judge ruled that the rating system was “arbitrary and capricious.” For now, the rating system is in abeyance. At some point the Regrnts and Legislature will have to clarify how this ruling affects state law.

Categories New Jersey, Teacher Evaluations, VAM (value-added modeling)

John Ewing: How to Attract and Retain the Teachers We Need

By dianeravitch
August 14, 2016 //
10

John Ewing, mathematician, is CEO of Math for America, an organization that supports teachers of mathematics.

In this post, he reviews some of the recent ill-considered efforts to “respect” teachers and offers advice about the minimum conditions necessary to assure that teachers have the respect, autonomy, and trust that professionals deserve.

You will also enjoy reading John Ewing’s brilliant takedown of teacher evaluation by test scores, which he called “Mathematical Intimidation.”

Categories Education Reform, Teacher Evaluations, Teachers and Teaching, VAM (value-added modeling)

Melinda Gates: We Won’t Back Down in Face of Opposition to Our Education Reforms

By dianeravitch
August 12, 2016 //
81

Melinda Gates told the National Conference of State Legislatures that the Gates Foundation has no intention of backing away from their agenda of Common Core, teacher evaluations that include test scores, charter schools, and digital learning.

No matter how controversial, no matter how much public pushback, they are determined to stay the course. For some reason, she thinks that the foundation is a “neutral broker,” when in fact it is an advocate for policies that many teachers and parents reject. She also assumes that the Gates Foundation has “the real facts,” when in fact it has a strong point of view reflecting the will of Bill & Melinda. There was no reference to evidence or research in this account of her position. Her point was that, no matter what the public or teachers may say, no matter how they damage the profession and public education, the multi-billion dollar foundation will not back down from its priorities. The only things that can stop them are informed voters and courts, such as the vote against charter schools in Nashville and the court decision in Washington State declaring that charter schools are not public schools.

The question that will be resolved over the next decade is whether the public will fight for democratic control of public schools or whether the world’s richest man can buy public education.

Melinda Gates said she and her husband, Microsoft co-founder Bill Gates, learned an important lesson from the fierce pushback against the Common Core State Standards in recent years. Not that they made the wrong bet when they poured hundreds of millions of dollars into supporting the education standards, but that such a massive initiative will not be successful unless teachers and parents believe in it.

“Community buy-in is huge,” Melinda Gates said in an interview here on Wednesday, adding that cultivating such support for big cultural shifts in education takes time. “It means that in some ways, you have to go more slowly.”

That does not mean the foundation has any plans to back off the Common Core or its other priorities, including its long-held belief that improving teacher quality is the key to transforming public education. “I would say stay the course. We’re not even close to finished,” Gates said.

The Bill & Melinda Gates Foundation has helped shape the nation’s education policies during the past decade with philanthropic donations that have supported digital learning and charter schools and helped accelerate shifts not only to the new, common academic standards, but to new teacher evaluations that incorporate student test scores.

The Obama administration shared and promoted many of the foundation’s priorities, arguing that they were necessary to push the nation’s schools forward and close yawning achievement gaps. Now that a new federal education law has returned authority over public education to the states, the foundation is following suit, seeking to become involved in the debates about the direction of public schools that are heating up in state capitals across the country.

Speaking here at a meeting of the National Conference of State Legislatures, Melinda Gates told lawmakers on Wednesday that the new federal education law, the Every Student Succeeds Act, gives them a chance to grapple with whether “we are doing everything in our power to ensure that students are truly graduating ready to go on to meaningful work or to college.”

“I want the foundation to be the neutral broker that’s able to bring up the real data of what is working and what’s not working,” Gates said in an interview afterward.

She went on to say that the foundation would continue to pursue its priorities.

“I think we know what the big elements are in education reform. It’s how do you support the things that you know work and how do you get the whole system aligned behind it,” Gates said. “I’m not telling you it’s going to be easy. There are now 50 states that have to do it, and there isn’t this federal carrot or the stick, the push or pull, to help them along.”

The agenda she described is not one that everyone considers neutral. It includes supporting the Common Core standards and developing lesson-planning materials to help teachers teach to those standards; promoting personalized learning, or digital programs meant to target students’ individual needs; and, above all, improving the quality of teachers in the nation’s classrooms, from boosting teacher preparation to rethinking on-the-job professional development.

Categories Accountability, Charter Schools, Common Core, Corporate Reformers, Education Industry, Education Reform, Gates Foundation, Bill Gates, Privatization, Resistance, Teacher Evaluations, Testing, U.S. Department of Education, VAM (value-added modeling)
« Older Entries
Newer Entries »