Archives for category: Testing

The Working Families Party briefly flirted with the possibility of running an independent slate. Its candidate for governor was going to be Zephyr Teachout, a Fordham Law professor who specializes in investigating public corruption.

However, at the party’s annual convention, WFP endorsed Cuomo after he promised to govern like a Democrat instead of a conservative Republican.

Teachout has not given up. She may challenge Cuomo in the Democratic primary.

Here is her statement on education.

Two sixth-grade classes in Ipswich, Massachusetts, lost a week of instruction while taking field tests, and they want to be paid for their time.

“But for now the test is still in its trial period and Laroche’s 37 students are among the 81,000 that spent two 75-minute periods in March and two 90-minute periods this past week completing the test.

“This time would have otherwise been spent writing and solving and graphing inequalities from real-life situations.

“During class last Monday, May 19, a teacher jokingly mentioned that the students should get paid for taking the test since their participation helps the PARCC and at the end of class the students pressed Laroche further on the idea.

“The kids proceeded to tell me that PARCC is going to be making money from the test, so they should get paid as guinea pigs for helping them out in creating this test,” said Laroche. “So I said, ‘OK, if that’s the case and you guys feel strongly then there are venues and things you can do to voice your opinion, and one would be to write a letter and have some support behind that letter with petition.”

“At 8 p.m. that night Laroche received a shared Google document with an attached letter from A-period student Brett Beaulieu, who asked that he and his peers be compensated for their assistance.

“I thought it was unfair that we weren’t paid for anything and we didn’t volunteer for anything,” said Beaulieu. “It was as if we said, ‘Oh we can do it for free.’”

“Beaulieu used his math skills in the letter, determining that the two classes would collectively earn $1,628 at minimum wage for their 330 minutes of work. He then went on to figure out how many school supplies that amount could buy: 22 new Big Ideas MATH Common Core Student Edition Green textbooks or 8,689 Dixon Ticonderoga #2 pencils.

“Even better, this could buy our school 175,000 sheets of 8 ½” by 11″ paper, and 270 TI-108 calculators,” Beaulieu wrote.

“On Tuesday, May 20 he gathered over 50 signatures from students, as well as from assistant principal Kathy McMahon, principal David Fabrizio and Laroche.”

The students wrote to PARCC, Arne Duncan, and Massachusetts Secretary of Education Matthew Malone.

Audrey Amrein Beardsley invited an economist to review Raj Chetty & Co’s effort to take down the statement of the American Statistical Assosociation.

Chetty and friends are the leading advocates for using test scores to rank teachers and fire those whose students have the lowest scores. The ASA report was inconvenient for their thesis, as it pointed out that teachers account for a small percentage of the variance in test scores, that what is observed is likely to be correlation all, not causal, and that putting so much weight on test scores was likely to cause bad effects.

There is also the inconvenient fact that VAM is no longer a neat theory, but has moved into the realm of reality. Some districts have used it, often with unfortunate results. One would think that academics would feel some obligation to see how their theory is working in practice rather than continue to market it on grounds of its theoretical elegance.

Last night, the school board in Cambridge, Massachusetts, voted to delay the implementation of PARCC, one of the two federally-funded online Common Core assessments. This was something of a problem for Mitchell Chester, Commissioner of the State Department of Elementary and Secondary Education, because he is chair of the governing board of PARCC.

The Cambridge School Committee acted in response to a parent petition. Parents are concerned about teaching to the test, narrowing the curriculum, and relying on online testing.

Take Action and Click Here to Demand the Release of the 3-8 Grade NYS Tests, Only 30 Seconds of Your Time

Dear Allies,

New York parents and teachers are outraged that they are not allowed to see the NYS Standardized 3-8 grades tests.

Before 2011, all tests were posted on the New York State Education Department (NYSED) website. That changed after the State Education Department signed a contract with Pearson in 2011 that doubled annual spending on NY State tests. Now, despite the already substantial increase in spending on testing, Commissioner King and Chancellor Tisch say an additional $8 million is required to restore full transparency. According to the contract, Pearson must produce unique test questions each year. This further raises legitimate questions about the need for additional funds.

Parents and educators are denied access to the information they need to determine what their child got wrong and what is needed to improve learning. Just as important, any flaws in the tests and scoring are not subject to public scrutiny and correction.

Beneath it all remains research that refutes the validity of these tests not just in their accuracy and legitimacy for measuring student growth, but in measuring teacher performance as well. Failing to be fully transparent only escalates those concerns.

As was the case with NYSED’s refusal to act within its own authority to address parent data privacy concerns with inBloom, we call on the legislature to act now!

We are asking all parents, educators and community members to join us in demanding that elected and appointed officials in Albany RELEASE THE TESTS!

Again, Please Take Action and Click Here to Demand the Release of the 3-8 Grade NYS Tests.

Thank you,
NYS Allies for Public Education
http://www.nysape.org

.

__,_._,___

On May 3, I received an email from Professor Raj Chetty of Harvard University, informing me that his famous paper on value-added assessment of teachers was being published by the American Economic Review. The paper has three authors: in addition to Chetty, the other authors include John Friedman and Jonah Rockoff, also at Harvard. When the paper was first released, it was reported on the front page of the New York Times, one of the authors discussed it on the PBS Newshour, and President Obama referred to it in his 2012 State of the Union address.

The New York Times story appeared on January 6, 2012. it began thus:

“WASHINGTON — Elementary- and middle-school teachers who help raise their students’ standardized-test scores seem to have a wide-ranging, lasting positive effect on those students’ lives beyond academics, including lower teenage-pregnancy rates and greater college matriculation and adult earnings, according to a new study that tracked 2.5 million students over 20 years.”

The reporter noted that the effect of a single “high-value” teacher was actually quite modest: “The average effect of one teacher on a single student is modest. All else equal, a student with one excellent teacher for one year between fourth and eighth grade would gain $4,600 in lifetime income, compared to a student of similar demographics who has an average teacher. The student with the excellent teacher would also be 0.5 percent more likely to attend college.” But think of the aggregate effect on an entire classroom: “Replacing a poor teacher with an average one would raise a single classroom’s lifetime earnings by about $266,000, the economists estimate. Multiply that by a career’s worth of classrooms.” President Obama cited the aggregate income gain for a classroom in his State of the Union address 18 days later.

This was the takeaway from the authors, as reported in the New York Times:

“The authors argue that school districts should use value-added measures in evaluations, and to remove the lowest performers, despite the disruption and uncertainty involved.

“The message is to fire people sooner rather than later,” Professor Friedman said.

“Professor Chetty acknowledged, “Of course there are going to be mistakes — teachers who get fired who do not deserve to get fired.” But he said that using value-added scores would lead to fewer mistakes, not more.

“Still, translating value-added scores into policy is fraught with problems. Judging teachers by their students’ test scores might encourage cheating, teaching to the test or lobbying to have certain students in class, for instance.”

The Chetty, et al, study supported VAM, which was the central feature of Race to the Top. Fire teachers sooner rather than later. One great teacher can produce lifetime gains.

Over the past few years, as more districts have implemented VAM, it has turned out to be far more complicated than the economists predicted to determine which teachers would produce great scores year after year, and which would not. Teachers were rated effective one year, ineffective the next year. Those who taught English learners, the gifted, and students with disabilities were less likely to get big gains. It turned out that VAM is affected by the composition of the classroom, since students are not randomly assigned.

But their paper continues to be the lodestar of VAM research.

Whereas it had originally appeared as a single paper published by the National Bureau of Economic Research, the editors suggested the paper was so important that it should be split into two papers and published separately. The last time this had happened was in 1971, for papers on taxation that had won two Nobel Prizes.

Here are the papers.

Click to access w19423.pdf

Click to access w19424.pdf

Professor Chetty’s email was addressed to me and Audrey Amrein-Beardsley, who has written extensively and critically about value-added assessment. In addition to her recently published book on VAM—Rethinking Value-Added Models in Education: Critical Perspectives on Tests and Assessment-Based Accountability—she writes a blog called VAMboozled that I often cite.

For the record, I have never met Raj Chetty, and I have met Beardsley once, when she interviewed me for an oral history archive.

I asked Beardsley if she would be willing to review the latest iteration of this now famous study of VAM, and she did, here on her blog.

Beardsley notes that there is a divide between econometricians, like Chetty, Friedman, and Rockoff, and educational researchers, who often feel some obligation to visit classrooms and see the effects of policies, not just analyze data from a great distance, without reference to context or something like reality.

Professor Chetty and I exchanged several emails. I asked for his permission to post our exchange. He said that he preferred that I not post his comments, which were invariably polite, but of course I was free to post my comments to him.

So here goes. This was my first response:

Dear Professor Chetty,

I certainly agree that teachers are valuable. I had some wonderful teachers
as I was growing up, also some mediocre ones, and a few really bad ones. I
went to an ordinary public school system in Houston, not an elite private
school.

I wish that this sentiment about the value of teachers was all that came
from your vast publicity machine.

Instead, we get more high-stakes testing, more test prep, more phony claims
that the work of my fourth grade or fifth-grade teacher was responsible for
my not getting pregnant when I was 15. Maybe my lifetime income was
increased by my sixth-grade teacher, though I doubt it. Funny, I was one of
eight children. We all had the same teachers, and we all turned out
differently. Some of us did well in school, others nearly flunked out. Was
it the fault of our teachers?

I know you love your celebrity–and hobnobbing with Obama and Duncan and
supporting their emphasis on testing and firing teachers sooner rather than
later—but think of the harm that you do to millions of children and their
teachers by the way you publicize your work. Do you feel good every time you
read about a teacher who is graded based on the work of children she never
taught? Or the “highly effective” teacher who was rated ineffective the next
year based on test scores? Or the precipitous decline in the number of
people who want to be teachers because of the non-stop attacks on teachers?
I don’t think your positive message is getting through. All people hear is
that you want those lousy teachers whose kids get low scores to be fired.
Now.

Diane Ravitch

On May 5, I wrote to both Raj and Audrey (we had reached a first-name basis):

Raj and Audrey,

I don’t know whether my thoughts advance or retard this informed discussion.

I look at the Chetty, etc. study as comparable to a pilot in a bomber
dropping a bomb on a city 30,000 feet below. He didn’t construct the bomb,
he doesn’t know how it hurts the people below, he can’t be held responsible
if his good intentions went wrong.

I invite you to read this blog by a teacher in Oklahoma:
http://bluecerealeducation.blogspot.com/2014/05/ms-bullens-data-rich-year.html

The odds are that he never heard of Raj Chetty. But look what Raj Chetty has
done to the quality of education, the students, and the teachers in
Oklahoma. Is this something to be proud of?

Your work–not yours alone, of course–has encouraged a technocratic
approach to education that would never be tolerated in our nation’s elite
private schools.

The pursuit of higher test scores on stupid multiple-choice standardized
tests does not improve education: it corrupts it.

Those who care deeply about humanistic education, about the life of the
mind, about deep learning, find your work–no matter how technically
perfect–utterly appalling. It drains education of joy and discovery and
makes everyone a slave to Pearson.

I would love to discuss this further with you over a glass of wine. I can’t
believe you do not understand the pernicious effects of your famous study,
featured on the first page of the New York Times, on the PBS Newshour, and
in President Obama’s State of the Union Address.

It seems to be my life work to insist that education is far, far more than a
score on a standardized test. Somehow, I suspect you agree. You are far too
intelligent not to.

Diane

Later on the same day, May 5, Raj responded, and I wrote:

Thanks, Raj,

A question and a comment.

My question: Could I publish our exchange on my blog? I get about 25,000-40,000 readers daily. But I would publish nothing without your permission.

My comment: Race to the Top has incentivized the use of VAM in most states. Your study has been cited by Obama and Duncan as evidence that they are on the right track, that it is “bad teachers,” not poverty, that cause low test scores.

Based on the real-world effects of VAM on real children and real teachers, I conclude that VAM has limited use, perhaps informative in looking at the effects of policies and programs (faithfully enacted, which they seldom are) in a school or a district, but of zero value in assessing individual teacher quality. As you must know by now, the ratings for individual teachers are unstable, and may change if a different test is used or unstable for no apparent reason at all. Teachers intuitively know that their ratings reflect the composition of the class, not their “quality” or efficacy as teachers. Even if VAM did work–and it does not–it would keep every teacher singularly focused on standardized tests, which narrow the curriculum, encourage schools and teachers to avoid the neediest students, promote test prep and cheating, and have other perverse effects.

At the end of the day, I as a mother and grandmother would not want my offspring to be enrolled in a school where standardized tests dominate teaching and learning. And that is precisely what VAM is doing to our nation’s public schools.

My third grandson enters third grade in a New York City public school next September. I hope by then that the opt out movement has grown so strong that teachers cannot be subjected to unfair and inaccurate VAMs. I will do whatever I can to encourage parents in every school district in the U.S. to keep their children home on testing day. That seems to be the only way that the giant standardized testing machine can be stopped.

Your work has been crucial in promoting standardized testing as the measure of teacher quality, even though major scholarly organizations disagree (the American Educational Research Association, the National Academy of Education, the American Statistical Association).

If you have modified your views (message: “fire teachers sooner rather than later”); if you have learned anything new since you first introduced your findings, I would love to know about it.

I repeat that I do not have the technical ability to argue algorithms with you. Your study may be technically brilliant. But its consequences for the quality of education and the lives of children and teachers have been disastrous. In its current application, it is Junk Science. Since I feel certain you don’t want to be remembered in history as the economist who sponsored Junk Science and treated children as data points, I hope you will give me reason to believe that you have rethought the conclusions of your study and provided clear warnings about the limitations and misuses of VAM.

Diane

We ended with the understanding that I would not quote his words or paraphrase them. I think I was true to that understanding.

Now, as I told him, I am not an economist, and I lack the technical proficiency to critique his paper. Maybe it will win two or three Nobel prizes. If all it says is that teachers are valuable, I agree. If it says that teacher affect eternity, I agree.

But if he really expects me to believe that my fifth-grade teacher (or was it my fourth-grade teacher) caused me to get higher test scores, and that because of her and my higher test scores, I did not get pregnant when I was 15, I think this is just plain silly.

This strikes me as the kind of study that brings huzzahs from economists for its technical precision, but is unrelated to the messiness of real life. The numbers may all add up, but there are no living, breathing students or teachers here, just data.

It is so incredibly frustrating to me to see economists and policymakers playing with the lives of children and teachers as if they were ants seen from a far distance or merely data points. I recommend to my new friend Raj a book by Yale Professor James C. Scott titled “Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed.” It changed my life. Maybe it will change his too.

Robert Berkman, a veteran math teacher, writes a blog called “Better Living Through Mathematics, where he regularly skewers nonsense.

In this article, he looks closely at a chart that purportedly demonstrates how pathetic is the performance of U.S. adults, compared to many other nations.

Berkman says this may be the “stupidest article about Common Core math program” that he has ever read.

To begin with, the graph does not identify the highest possible score, making it impossible to draw conclusions or comparisons. So one conclusion from the graph, Berkman says, is: “whatever sample of US adults took this test did 88% as well as the adults in the top scoring nation, Japan. I think that’s pretty damned good, considering the United States is second to the world in poverty, leaving Japan in the dust by over 10 percentage points (and I’m sure Japan uses a much higher economic benchmark for poverty than we do here in the US.) Of course, we all know that poverty is the single greatest predictor of poor school performance.”

[Note to Robert Berkman: that “second in the world in poverty” is nonsense, despite the authoritative source. It is a comparison not of all nations, but of the most economically developed nations, and the U.S. is supposedly second to Romania. This is an absurd comparison because Romania doesn’t belong in this group of nations. Romania is an Eastern European nation whose economy was mismanaged and impiverished by central planning for decades. Oh, well, I may never get this error corrected, but I keep trying. The fact is that we have the highest level of child poverty of any advanced nation in the world.]

After pointing out other errors, Berkman writes:

“Finally, this article is yet another example of the “waking up on third base” phenomena, which posits that everything that you see in a Common Core math curriculum is the direct result of the implementation of the Standards. Nothing could be further from the truth: all of the items described on in the article have been documented, published and taught since the NCTM published its curriculum standards a quarter of a century ago. If you’ve been teaching math using a textbook that was published in the last 20 years, you’ve probably seen all this stuff before including, with all deference to Mr. Colbert, the infamous description of a “number sentence.” Telegram for Mr. Colbert: 1989 is writing to tell you to “LOL!”

He notes with dismay that “NCTM actually tweeted the link to this worthless piece of codswallum, and smelling something rotten, I just had to follow the scent.”

Crazy Crawfish, aka Jason France, used to work in the assessment division of the Louisiana State Department of Education. He has sources there, and he knows the games that officials can play to make data look better or worse.

In this post, he asserts that State Commissioner of Education John White, with two years in TFA and a few years in Joel Klein’s NYC Department of Education, came to Louisiana with a strong view of what to do and how to lead the change that he thought was needed.

France says that White did not agree with Governor Jindal’s voucher plan, but went along.

He says he knew that the VAM plan was problematic.

But he also knew he had to produce higher test scores to claim victory.

What he had going for him is that the media opin Louisiana is passive, and the public can’t or won’t follow stories that involve math. That provides plenty of leeway for officials looking for a win.

Under current federal and state laws, test scores are supposed to go higher every year. Every year, the students are a different cohort, but their scores must be higher than those who preceded them. High expectations–no matter how unrealistic–are supposed to produce high achievement. Think of it this way, if students are running track and can barely jump over a 3′ bar, raise the bar to 4′ and see what happens. The assumption (usually by politicians) is that raising the bar will cause students to jump higher. But many will fail because the theory doesn’t work.

A seventh -grade teacher in a Title I (high-poverty) school in Texas writes:

“I can say from personal experience that the “scores” are ripping my department apart. Last year two-thirds of our group were shuffled between grades because of low test scores. Result, even worse scores this year. All I can say is that failure begins at the top. Ugly comments have been made, morale could not be lower.. The students are beginning to check out.

“Is it worth it all?”

The New York State United Teachers issued a statement supporting th districts that are opting out of state field tests.

“ALBANY, N.Y. May 30, 2014 – As tens of thousands of students statewide prepare to begin vfield testing” questions for future state exams, New York State United Teachers President Karen E. Magee said today more time should be devoted to teaching – not testing – and called for an end to student participation in field tests.

The State Education Department is administering field tests from June 2-11 to try out prospective standardized test questions developed by the giant testing company, Pearson PLC. These tests do not count for students, teachers or schools and are solely used to “test the test.” Stand-alone field tests in English language arts and math will be administered in most schools to students who took the state’s 2014 Grades 3-8 Common Core ELA and math tests.

“New York’s over-reliance on standardized tests continues to have a negative effect on our students,” Magee said. “Teachers and parents are united in our concern that excessive state standardized testing takes important time away from instruction.”

Magee called on the state to grant the SED’s request for sufficient resources so it can develop a bank with enough age- and grade-appropriate, reliable and valid questions to entirely eliminate the need for field testing.

Magee noted a number of school districts – Comsewogue and Ossining, for example – are reportedly boycotting this year’s field tests.

“NYSUT applauds those school districts that have recently opted out of this year’s field testing. Just as we support parents’ rights to opt their child out of the state tests, we support those districts that are opting out of the field tests,” Magee said. “Why should New York state students be unpaid researchers for Pearson?”

“NYSUT Vice President Catalina Fortino, who oversees the union’s Research and Educational Services Division, added, “These upcoming field tests mean that, once again, too many students in New York state will be spending time on standardized tests that take time away from real learning.”