Archives for category: Standardized Testing

Heather Vogell, a stellar reporter for the Atlanta Journal-Constitution, has done in-depth investigative reporting on the standardized tests that now are used to determine the fate of students, teachers, principals, and schools.

She has found a surprising number of errors, though not surprising to those familiar with the testing industry.

Read this article. How should a student respond to questions where all the answers are wrong?

What does it do to students when they realize the questions or answers are wrong?

Here is an idea for this tireless reporter: investigate how much money the testing industry spends to lobby Congress and the states to maintain their hold over the minds of our students and the very definition of education.

Readers, after you read Heather Vogell’s excellent articles, please read Todd Farley’s eye-popping exposé of the testing industry called “Making the Grades.”

You will never forget his description of how student constructed responses are scored and who is doing it (minimum wage temps).

A group called the Campaign for High School Equity made
news the other day when it criticized Arne Duncan’s NCLB waivers
and complained that the waivers might reduce the amount of
high-stakes testing for poor and minority students. Mike Petrilli
at the conservative think tank Thomas B. Fordham Institute
challenged me to admit that the civil rights groups were leading
the charge to protect high-stakes testing. I accepted his
challenge. It didn’t make sense, on the face of it, that civil
rights groups would want more testing. Every standardized test in
the world reflects socioeconomic status, family education and
income. Testing measures advantage and disadvantage. Some kids defy
the odds, but the odds strongly predict that the have-not kids will
be at the bottom of the bell curve. They will be labeled as
failures. They may get help, they may not. But one thing is sure:
standardized testing is not a tool to advance civil rights. Testing
is not teaching. Low scores do not produce more resources or higher
achievement. More testing does not improve learning. It increase
rote learning, teaching to the test, narrowing the curriculum, and
sometimes, cheating. So who is this group and why does it want more
testing. First,
the article that Mike forwarded to me
. It says that the
waivers are allowing too many schools to avoid the consequences of
being low-performing. In other words, the Campaign for High School
Equity prefers the draconian consequences of No Child Left Behind
and the punitive labels attached to schools based on high-stakes
testing. Of course, their statement also makes it appear that Arne
Duncan is trying to water down punishments and high-stakes testing,
when nothing could be further from the truth. Who is part of the
Campaign for High School Equity? It includes the following groups:
National
Urban League
National
Council of La Raza
National
Association for the Advancement of Colored People
The
Leadership Conference Education Fund
Mexican
American Legal Defense and Educational Fund
League
of United Latin American Citizens
National
Association of Latino Elected and Appointed Officials Educational
Fund
Alliance
for Excellent Education
National
Indian Education Association
Southeast
Asia Resource Action Center
Why are they in favor of
high-stakes testing, even though the evidence is overwhelming that
NCLB has failed the children they represent? I can’t say for sure,
but this I do know. The Campaign for High School Equity is funded
by the Gates Foundation. It received a grant of nearly $500,000.
Some if not all of its members have also received grants from Gates
to support the CHSE. The NAACP
received $1 million
from Gates to do so. LULAC
received $600,000
to support the CHSE. The Alliance
for Excellent Education received $2.6 million
“to promote
public will for effective high school reform.” The Leadership
Conference on Civil Rights Fund received
$375,000 from the Gates Foundation
to support CHSE. The
National
Association of Latino Appointed
and Elected Officials is
Gates-funded, though not for this specific program. The National
Indian Education Fund received
Gates funding
to participate in CHSE. The Southeast Asia
Resource
Action Center was funded by Gates
to participate in CHSE. The others are not Gates-funded.

When CHSE demands more high-stakes testing,
more labeling of schools as “failed,” more public school closings,
more sanctions, more punishments, they are not speaking for communities
of color. They are speaking for the Gates Foundation.

Whoever is actually speaking for minority communities and children of color is
advocating for more pre-school education, smaller class sizes,
equitable resources, more funding of special education, more
funding for children who are learning English, experienced
teachers, restoration of budget cuts, the hiring of social workers
and guidance counselors where they are needed, after-school
programs, and access to medical care for children and their
families.

The folks at the Thomas B. Fordham Institute are struggling to come to terms with the New York testing disaster. They certainly will not retreat from their deep faith in standardized testing, and they insist that there must be more parent choice, even though parents are sick of the excessive testing and most continue to choose their neighborhood school, if they still have one.

This is my favorite line:

“Reform critics like Diane Ravitch often question why we don’t push reforms that would create a “Sidwell Friends” for every student. Putting aside where we would find the extra $1.6 trillion it would take to make that possible, there is a simpler answer: some of us don’t want Sidwell Friends. And just because some believe the elite culture of the top 1 percent is what’s best for all children, doesn’t mean all parents share that belief.”

I can’t say where that $1.6 trillion number comes from. I went to ordinary public schools that did not face annual budget crisis, that did not squander millions on standardized testing, that provided arts programming and daily physical education and foreign languages, that did not fire teachers if students got low test scores. But people who did not go to ordinary public schools may not know that.

What I want to challenge here is the assertion that “some of us don’t want” what the best private schools have to offer.

Who wouldn’t want what Sidwell offers? Or Exeter? Or Lakeside Academy in Seattle?

Who wouldn’t want classes of 12-15 instead of 35-40?

Who wouldn’t want a beautiful campus?

Who wouldn’t want experienced, respected teachers?

Who wouldn’t want a rich curriculum with science labs, history projects, drama and music, and lots of sports every day?

Who wouldn’t want to go to a school that never gave standardized tests and didn’t judge teachers by students test scores?

Maybe there are such people. I have never met them. Maybe they work at Fordham or the Gates Foundation, but I doubt it.

I usually agree with Matt Di Carlo. He is one smart guy.

But not always.

That’s okay. Friends can disagree and still be friends (I proved that by blogging with Deborah Meier for five years).

I think that value-added methods of using test scores to rate teacher quality are “junk science.”

Matt disagrees

Now, granted, I am but a historian, not a social scientist, but I do read lots of social science. I noted that the National Academy of Education and AERA held a briefing on Capitol Hill and issued a joint statement warning about the pitfalls of VAM. Here is a salient point from their report: “With respect to value-added measures of student achievement tied to individual teachers, current research suggests that high-stakes, individual-level decisions, or comparisons across highly dissimilar schools or student populations, should be avoided.”

Edmund Gordon, one of our nation’s most eminent psychologists, recently led a commission to study assessment practices. He concluded that the overemphasis and misuse of standardized testing to hold students, teachers, and schools accountable is not only ineffective but “immoral.”

I would say that “immoral” is an even stronger condemnation than “junk science.”

Campbell’s Law suggests that the use of high-stakes testing degrades education. Threatening to fire teachers if their students’ scores don’t go up does not produce better education. It produces worse education. It promotes narrowing of the curriculum. It promotes cheating. It encourages teachers and schools to avoid the neediest students.

Teaching is so much more than test scores. To think that teachers may be defined significantly by the rise or fall of the test scores of their students requires a belief in the intrinsic value of standardized tests that I do not share. We may learn something from wide assessments with no-stakes, like NAEP. But using these flawed instruments to fire teachers and close schools is–in my judgment–wrong. They were not designed for those purposes. And the first rule of psychometrics is that tests should be used only for the purpose for which they were designed.

All things considered, the term “junk science” seems appropriate, as does Dr. Gordon’s phrase: ineffective and immoral.

One reason parents flee public schools, if they can afford it, is to escape the dead hand of testing that now strangles learning. The sooner we can put testing in its place as a diagnostic tool for teachers to assess what they have taught (not as a Pearson-designed 14-hour ordeal), the sooner we will restore the rightful purposes of education and the dignity of the teaching profession.

Anthony Cody met teacher Michelle Gunderson at Occupy the DOE. When he heard her ideas about testing, he invited her to write a guest blog.

Gunderson explained that she has seen test used to sort children, to punish children, and now–to privatize schools.

She has developed her own credo for the ethical use of tests.

Please read it.

The essence of the pledge is that a student’s test scores should be treated as confidential.

I wholeheartedly agree.

I would go farther.

We don’t expect our doctors to turn over our test results to the state, why should teachers give children’s data to anyone but them and their parent or guardian?

If you live anywhere near Cambridge, you should plan to attend this FairTest event on May 9 when FairTest will honor Jonathan Kozol.

FairTest will present Jonathan with the Deborah W. Meier Award for Heroes in Education for his lifelong commitment to education, children, and human rights.

Jonathan is indeed a hero in education.

He is a champion for children, for teachers, and for public schools.

I am happy to add him to this blog’s honor roll for his courage, his eloquence, and his clear vision of a just, decent, and humane society.

 

FairTest has been a watchdog for the testing industry for many years.

The latest news is that the SAT will be overhauled (again), this time to align it with the Common Core standards.

No big surprise, since the head of the College Board, David Coleman, was the lead player in developing the Common Core standards.

Fairtest reports that more than 800 colleges and universities no longer require the SAT for admission.

A test coaching industry has grown up to prep students for the SAT.

Thus, one’s SAT scores have become, more than ever, an indicator of a family’s ability to pay for test prep.

The FairTest newsletter reports:

“Responding to the College Board’s ongoing failure to address the exam’s flaws, the number of schools dropping the SAT has surged. Since 2005, nearly 90 colleges and universities have eliminated testing requirements for all or many applicants. That brings the total to more than 800. The test-optional list now includes 140 institutions ranked in the top tier of their respective categories by U.S. News & World Report.”

Tom McMorran was named Connecticut’s principal of the year in 2012. Here he offers a lesson to our nation’s politicians about the Common Core standards and high-stakes testing. Send this to your state legislators and your member of Congress and the Senate.

 

Tom sent the following comment:

 

It is time to school our politicians about CCSS and High-Stakes testing.
Here is a day in course level 101.

Tom McMorran
2012 High School Principal of the Year NASSP

Philosophy 101:

In order for an argument to carry weight and cause one not only (1) to believe it, but also (2) to take action based on that belief, the argument must have warrant. There is nothing subtle here. The weakest form of argument is some version of “I am in power and I say so…” Or, in any teen’s mother’s words: “Because I am the parent!”

When the person presenting the argument relies on some authority to shore up his/her argument, then we have a duty to test the reliability of the authority. In philosophy or rhetoric or simply argumentation this is known as an appeal to authority.

Last week Gina, Mary Ann, and I attended another workshop at the Connecticut Association of Schools (CAS). This is the body that is, in theory, an institution that is independent from the State Department of Education. The presentation was made by Dr. Diane Ulman, who is the Chief Talent Officer at the DOE. She was appointed by Commissioner Pryor.

As part of her presentation, Dr. Ulman reminded us that the Governor’s Council, The Gates Foundation, a range of other foundations and 46 states have signed on to CCSS. In other words, she offered an appeal to authority. Now, for an appeal to authority to work, credentials must be established. And any group that has a personal, financial interest in public policy must make their bias known. So, let’s ask a very basic question: Where’s the money? For Pearson, Houghton Mifflin, and other publishing companies the prospects are enormous. Smarter Balance, the private, for-profit company received half a billion federal dollars to develop the next generation of assessments, which will replace the CMT and CAPT and be administered in about 26 states. You may recall the President’s State of the Union Address; he all but bragged about the 4.3 billion for Race to the Top (RTTT) funding, and how it was amazingly inexpensive for the Federal government to get these 46 cash-strapped states to sign on.

So, when you hear the proponents of the Common Core State Standards and High-Stakes Testing appeal to authority, you have a duty to weigh the degree to which the authority has sufficient warrant to be believed. Here, let me try it: Elvis is still alive. Evidence? 50 million Elvis fans cannot be wrong.

Statistics 101:

Before meaningful inferences can be drawn from any data set, the researcher has a duty to ensure that the social phenomenon under consideration has not been conflated with other factors. In other words, if you want to give a test that measures the contributions of a teacher to a student’s growth, you must account for and guard against any other factor that might conflate with the primary inquiry. It works like this:

1. We want to know if the teacher’s skill as a reading teacher leads to observable reading skills in her/his students.
2. Therefore, if we give all students the same reading assessment, we should be able to conduct a comparison between teacher A’s students and teacher B’s students.
3. From that comparison we can tell if one teacher is better than another at teaching reading.

So, what’s wrong with that?
A. If the assessment was designed to measure student performance, it can only be used for teacher evaluation by an act of hopeful extension. If the assessment had been designed to measure teacher performance, then it could only be used to measure student performance indirectly.
B. In order for teacher A to be compared with teacher B, the context for all potentially confounding factors for the experiment must be the same. In other words, the only factor that can be measured is, in this case, reading.

But wait, Tienken, Lynch, Turnanian, and Tramaglini have something to say about this in “Use of Community Wealth Demographics to Predict Statewide Test Results in Grades 6 & 7.”

Here’s the very short version: If you tell these researchers three out-of-school demographic variables, then they can tell you a New Jersey school system’s 6th Language Arts scores on the New Jersey Assessment of Knowledge for grade 6 (NJASK6). Tell them (a) the percentage of lone parent households in the community, (b) the percentage of people with advance degrees, and (c) the percentage of people without a high school diploma, and they can plug those data points into a formula that will predict the scores within an acceptable range.

If confounding factors such as a town’s wealth are predictors of performance, then how can we use a reading assessment designed to measure a student’s performance in order to decide whether or not a teacher has effectively taught the skills or knowledge measured by the test?

Here is another wee complication: In New York the APPR rating system that is a year ahead of Connecticut’s uses a growth over time model, which sounds great. But, if you are the unlucky teacher who earned the highest rating in your first year and then for some reason you “slipped” to proficient in your second year, you have not shown growth over time, have you?

Economics 101:

The foundation of the CCSS argument has been negative comparisons between international assessments of 15 year olds in which Americans appear to come out near the middle of the testing range. The argument runs like this: The future economy needs 21st Century Skills. Other countries are out-scoring us, therefore the strength of our economy is threatened over the next few decades.

But, if we recall our faculty reading of Yong Zhao’s Catching Up, or Leading the Way, we recall that there is an inverse relationship between performance on a standardized international assessment and productivity over time. Yes, that’s right. The same group of 14 yr olds who came in dead last in the First International Math Study (TIMS) is now a group of the 60-somethings who control the American economy, which is still rated among the top three most productive economies according to the World Economic Forum.

So, to make the international comparisons look bad, the proponents of this argument have to place the USA into a comparison with the 58 countries for which there is competitive data. Yikes, it looks like the mid-21st century will be dominated by Bulgaria; didn’t see that coming, but that’s what the tests show. If, on the other hand, one compares the US to the G-20 or G-7 Economies, the negative comparisons cease to be statistically valid.

Also, let’s just pause for a minute here and consider the PISA study of 15 year olds. You have to be 15 to take the test. So, if an American kid averages 170 days of school attendance a year, and among those days are mid-years, finals, and field trips, then let’s say there is a good chance for 140 days of instruction. But Asian countries regularly offer up to 240 days of school, so let’s knock off twenty and call it 220. Should an American student be able to compete with his/her counterparts in math? Well, actually, even on the much-vaunted PISA fully one out of four students performing at level five, the highest level, is an American.

So, if we follow the scores-to-economics argument, we would be likely to engage in behaviors that promote success on a test, but this will lead to lower creativity and productivity in the adult world!

Sociology 101:

Campbell’s Law: 1975 “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social process it was intended to monitor.”

Here is what Nichols and Berliner have to say at the end of a comprehensive examination of NCLB and high-stakes testing: “We are going to do something unheard of in the history of academic research. In this concluding chapter, we are not going to call for more research. There is absolutely no need for new research on high-stakes testing! Sufficient evidence to declare that high-stakes testing does not work already exists.” (2006, Collateral Damage, p. 175).

CONCLUSION:

1. I am NOT saying that we should have no standards. I am not saying that a standards-based curriculum is a bad thing; in fact, I am in favor of it.
2. I am NOT saying that we shouldn’t desire excellence for all students. I am not saying that all students should be able to have meaningful adult lives.
3. I am NOT saying that teachers shouldn’t link their performance to student achievement. I am not saying that we should avoid standardized assessments.

I AM SAYING that the worn out application of so-called hard-nosed business practices (which I do not believe business men or women apply to their own concerns) have any place in a school. I AM SAYING that there is a better way, and it is for all of us educators to embrace our responsibilities as professionals and act from Informed Professional Judgment. I AM SAYING that we can either define ourselves or accept the so-called reform that is happening to us.

It might be that we have to acknowledge and optimistically embrace the following proposition: The High School Structure that has served us so well is not broken; it is obsolete, and it is time for us to transform it!

Tom

An article in the Wall Street Journal goes on a rant against critics of standardized testing. It was written by a charter school advocate in Texas and a professor at ultra-conservative Hillsdale College in Michigan. The authors are shocked that so many parents and local school boards in Texas want to reduce the number of tests needed to graduate high from 15 to only three or four.

They insist that American students are really incredibly stupid and the best way to make sure they gain the wisdom of the ages is to demand more of Pearson’s multiple choice tests.

You can see that they really care about the Higher Things because they drop names like Homer, Milton, Melville, and Shakespeare. They also drop some references to the Founding Fathers.

Two things are odd about this article (in addition to the fact that the statistics they cite were based on a telephone survey of 1,200 students, who were asked multiple-choice questions and had no reason to take the survey seriously).*

First, when American students were classically educated, many eons ago, as the authors yearn for, they were not taking any standardized tests. None. Zero. Zip. They were writing essays and examined orally by their teachers. It seems the authors yearn for the good old days of 1910, when the high school graduation rate was about 10%.

And then there is the irony that the authors are the sort who usually rant about the importance of respecting parental choice. Why do they deny the choice that so many Texas parents so clearly and passionately want: an education where more time and resources are devoted to teaching, not testing?

Gosh, with more time for teaching and learning, the students would actually have time to read Homer, Shakespeare, Melville, and Milton, instead of test prep.

*Full disclosure: I was co-chair of the organization that commissioned the survey and co-authored the introduction. The organization, named Common Core, has no connection to the Common Core State Standards. It was created to advocate for the liberal arts and sciences, not for testing them. I resigned from it in 2009.

P.S. a comment below points out that Hillsdale College attracts many home-schoolers who do not take batteries of standardized tests annually.

I just received this press release. This is a great action by the Providence Student Union.

Governor Chafee, take the test!

State Superintendent Deborah Gist, take the test!

MEDIA ADVISORY

March 13, 2013

CONTACT: Aaron Regunberg | Aaron@ProvidenceStudentUnion.org | 847-809-6039 (cell)

STUDENTS INVITE LEADERS, POLICY MAKERS TO “TAKE THE TEST” –

WOULD THEY GRADUATE UNDER NEW NECAP POLICY?

WHAT: To lend a deeper perspective to the debate over Rhode Island’s new high-stakes testing diploma system, members of the Providence Student Union (PSU) have invited community leaders and policy makers to put themselves in students’ shoes and take a shortened version of the NECAP exam that is now being used as a make-or-break graduation requirement for the state’s young people. Currently 40 state senators, state representatives, city council members, school board members, non-profit directors, lawyers, reporters, and education officials are planning to participate in this student-administered, student-proctored event.

DATE: Saturday, March 16th

WHERE: Knight Memorial Library, 275 Elmwood Avenue, Providence

WHEN: 12:15 p.m. event begins, test-takers start their exams; 1:25 p.m. suggested time for press to arrive to catch the last minutes of test-taking, see a short PSU presentation to the media, and interview test-takers.

The event will have strong visuals. Students and adult participants will be available for interviews.
###

The Providence Student Union is a youth-led student advocacy organization bringing high school students together to ensure youth have a real voice in decisions affecting their education. Learn