Testing expert Fred Smith explains here why New York City Chancellor Carmen Farina should say no to the Pearson field tests.
The field tests waste instructional time. They benefit the publisher, not the students.
“Here are some arguments the chancellor could use:
*Because students know the stand-alone field tests don’t count and are of no consequence to them, they are not motivated to do well, especially in lovely June weather. This skews the data and fails to provide Pearson with reliable “intelligence” needed to furnish good exams.
*Proof that stand-alone field testing is an unworkable approach to test development lies in the poorly constructed ELA and math exams that were given in 2012 and 2013. Witness the criticism from teachers and parents across the state on both exams.
*The field tests have proceeded because the state has created a top-down system that inhibits principals and teachers from telling parents about them or seeking permission for their children to take them.
*A definitive analysis of federal legislation and state rules and regulations has found no legal basis requiring schools to give, or parents to go along with, the tests.”
Diane Ravitch's blog 
The good news is that if students “..know the stand-alone field tests don’t count and are of no consequence to them, they are not motivated to do well, especially in lovely June weather. This skews the data and fails to provide Pearson with reliable “intelligence” needed to furnish good exams.” they will not become stressed about taking the exam during the lovely June weather.
So, Mr. free market economist, would you support mrslewischem’s proposal below?
Yes, I await TE’s response. Students are often exploited by the testing industry.
Sorry for the delay, I was walking home from the university.
I think it is a fine idea. They should probably pay more for higher scores than for lower scores in order to avoid the incentive problems expressed in the original post and will need to be careful to control for any selection issues that arise by allowing some students to choose to take the exam and others choosing not to take the exam.
Perhaps students should be compensated as test subjects who willingly participate in fields tests of new products are. The free market ideology seems to operate in one direction only.
mrlewishchem: I think you just made the “invisible hand” of the marketplace slap its “invisible face.”
Amazing how much “invisible noise” it makes!
😱
Let’s see how much “rigor” Pearson can take and how much “grit” it can demonstrate as it discharges its fiscal obligations to self-regulating behavior—on the part of parents, taxpayers, students and teachers.
Aren’t willing to pay?
“I reject that mind-set.” [Michelle Rhee*]
*Even a broken clock is right twice a day.*
😎
A koan for Market Fundamentalists: What’s the sound of an Invisible Hand clapping?
Great idea.
How about also compensating the teachers who must be present to proctor the tests and the school systems that must have open buildings, AC, computer access, etc.? Since these duties are outside the scope of educating your children and are squarely in the realm of R and D for a private corporation they should be outside the regular school day, IMO.
Free market, right?
Seems like a good idea as well.
How about compensating the entire country for a lost generation of students whose educations were narrowed and distorted and trivialized by test prep because of these tests?
Reparation for testing. Exactly how much do they owe us?
And this IS exactly what has happened-is still happening-to our students. So sad, so infuriating…..this is gives me the courage to ask questions and to speak up against poor educational practices.
Yes on paying the kids. See another post on Diane’s blog in which students suggested that very thing. Will be fun to turn the tables on these boobs. Teachers can invoice for payment as well. Who cares if the check never arrives. Let’s use these actions to make our point across the USA.
Everyone can enjoy creating impressive graphics to advertise their “contractor” services. Some kids can volunteer to get paid for taking tests more than once. Why not Summer Work Programs in which teens get paid to score the tests, upload the data, analyze it and then draft national education policy proposals based on the “results”. Civics lesson will be when youth and adults take a field trip to small claims court in an effort to obtain compensation for services rendered. There are so many creative, enjoyable ways to take back our freedom, our democracy and our inventive public schools!
Maybe the boys will charge more than we girls, or vice versa?
Fred Smith has demonstrated repeatedly that the practice of late-in-the-year stand-alone field tests is a fig leaf to cover the inadequacy of Pearson’s test development process. The retort of NYSED is that to develop the tests properly is more expensive — and testing opponents wouldn’t want that, would they? The answer is so simple it escapes all our state education officials: STOP THE NON-STOP TESTING! As Alfie Kohn and many others have repeatedly shown, standardized tests mainly track family income and DON’T measure the kinds of learning that leads to life-long intellectual growth, so what use are they, really?
A modest proposal: when class sizes in all public schools match those in private schools, when all public-school kids have a chance to learn a musical instrument, to do experiments in a well-equipped science lab, and to study in a well-stocked library full of books, not computer screens — well, then let’s decide if we have money to burn on tests that are, at best, redundant confirmations of what teachers and parents already know about how their kids are progressing. Until then, suspend the phenomenally wasteful, destructive practice of testing all kids in all schools every year.
There should be some monetary motivation/compensation to take these field tests, either to the schools and/or to the students. Why should Pearson be able to test their product free of charge then turn around and charge the school districts for the finished product (whose results the recipient is not allowed examine, even after the exam has been completed, to see if it is a quality product).
If we are using a business model for education, consider what sort of company would put up with this nonsense. The answer is: None would!
and the tests are a FRAUD… PERIOD.
YES, YES, Susan! This IS the point–everything else is mere commentary. STOP THIS TESTING!!!!
Forgot to add simple anagram: OPTS~STOP!
Please take the time to read, sign, and circulate the petition entitled:
STOP COMMON CORE TESTING.
http://www.petition2congress.com/15080/stop-common-core-testing/?m=5265435
Yet no one stops this madness. DeBlasio, Farina and the idiot Mulgrew have some vested interest in this torture.
I was proud to opt out and will NEVER allow my kids to be tortured by this nonsense.
Please take the time to read, sign, and circulate the petition entitled:
STOP COMMON CORE TESTING.
http://www.petition2congress.com/15080/stop-common-core-testing/?m=5265435
Another PARCC Mugging
I think that it would be criminal malfeasance for officials of the state of New York to proceed with the PARCC examinations in ELA. Here are my reasons:
1.The PARCC ELA exams are invalid.
First, much of attainment in ELA consists in world knowledge (knowledge of what—the stuff of declarative memories of subject matter). The “standards” being tested cover almost no world knowledge and so the tests based on those standards miss much of what constitutes attainment in this subject. Imagine a test of biology that left out almost all world knowledge about biology and covered only biology “skills” like—I don’t know—slide-staining ability—and you’ll get what I mean here. This has been a problem with all of these summative standardized tests in ELA since their inception.
Second, much of attainment in ELA consists in procedural knowledge (knowledge of what—the stuff of procedural memories of subject matter). The “standards” being tested define skills so vaguely and so generally that they cannot be validly operationalized for testing purposes as written.
Third, nothing that students do on these exams EVEN REMOTELY resembles real reading and writing as it is actually done in the real world. The test consists largely of what I call New Criticism Lite, or New Criticism for Dummies—inane exercises on identification of examples of literary elements that for the most part skip over entirely what is being communicated in the piece of writing. In other words, these are tests of literature that for the most part skip over the literature, tests of the reading of informative texts that for the most part skip over the content of those texts. Since what is done on these tests does not resemble, even remotely, what actual readers and writers do in the real world when they actually read and write, the tests, ipso facto, cannot be valid tests of real reading and writing.
Fourth, standard standardized test development practice requires that the testing instrument be validated. Such validation requires that the test maker show that the test correlates strongly with other accepted measures of what is being tested, both generally and specifically (that is, with regard to specific materials and/or skills being tested). No such validation was done for these tests. NONE. And as they are written, based on the standards they are based upon, none COULD BE done. Where is the independent measure of proficiency in CCSS.Literacy.ELA.11-12.4b against which the items in PARCC that are supposed to measure that standard on this test have been validated? Answer: There is no such measure. None. And PARCC has not been validated against it, obviously LOL. So, the tests fail to meet a minimal standard for a high-stakes standardized assessment—that they have been independently validated.
2. The test formats are inappropriate.
First, the tests consist largely of objective-format items (multiple-choice and EBSR). These item types are most appropriate for testing very low-level skills (e.g., recall of factual detail). However, on these tests, such item formats are pressed into a kind of service for which they are, generally, not appropriate. They are used to test “higher-order thinking.” The test questions therefore tend to be tricky and convoluted. The test makers, these days, all insist on answer choices all being plausible. Well, what does plausible mean? Well, at a minimum, plausible means “reasonable.” So, the questions are supposed to deal with higher-order thinking, and the wrong answers are all supposed to be plausible, so the test questions end up being extraordinarily complex and confusing and tricky, all because the “experts” who designed these tests didn’t understand the most basic stuff about creating assessments–that objective question formats are generally not great for testing higher-order thinking, for example. For many of the sample released questions, there is, arguably, no answer among the answer choices that is correct or more than one answer that is correct, or the question simply is not, arguably, actually answerable as written.
Second, at the early grades, the tests end up being as much a test of keyboarding skills as of attainment in ELA. The online testing format is entirely inappropriate for most third graders.
3. The tests are diagnostically and instructionally useless.
Many kinds of assessment—diagnostic assessment, formative assessment, performative assessment, some classroom summative assessment—have instructional value. They can be used to inform instruction and/or are themselves instructive. The results of these tests are not broken down in any way that is of diagnostic or instructional use. Teachers and students cannot even see the tests to find out what students got wrong on them and why. So the tests are of no diagnostic or instructional value. None. None whatsoever.
4. The tests have enormous costs.
First, they steal away valuable instructional time. Administrators at many schools now report that they spend as much as a third of the school year preparing students to take these tests. That time includes the actual time spent taking the tests, the time spent taking pretests and benchmark tests and other practice tests, the time spent on test prep materials, the time spent doing exercises and activities in textbooks and online materials that have been modeled on the test questions in order to prepare kids to answer questions of those kinds, and the time spent on reporting, data analysis, data chats, proctoring, and other test housekeeping. So, the tests have enormous opportunity costs.
Second, they have enormous cost in dollars. In 2010-11, the US spent 1.7 billion on state standardized testing alone. Under CCSS, this increases. The PARCC contract by itself is worth over a billion dollars to Pearson in the first three years, and you have to add the cost of SBAC and the other state tests (another billion and a half?), to that. No one, to my knowledge, has accurately estimated the cost of the computer upgrades that will be necessary for online testing of every child, but those costs probably run to 50 or 60 billion. This is money that could be spent on stuff that matters—on making sure that poor kids have eye exams and warm clothes and food in their bellies, on making sure that libraries are open and that schools have nurses on duty to keep kids from dying. How many dead kids is all this testing worth, given that it is, again, of no instructional value? IF THE ANSWER TO THAT IS NOT OBVIOUS TO YOU, YOU SHOULD NOT BE ALLOWED ANYWHERE NEAR A SCHOOL OR AN EDUCATIONAL POLICY-MAKING DESK.
5. The tests distort curricula and pedagogy.
The tests drive how and what people teach, and they much of what is created by curriculum developers. This is a vast subject, so I won’t go into it in this brief note. Suffice it to say that the distortions are grave. In U.S. curriculum development today, the tail is wagging the dog.
6. The tests are abusive and demotivating.
Our prime directive as educators is to nurture intrinsic motivation—to create independent, life-long learners. The tests create climates of anxiety and fear. Both science and common sense teach that extrinsic punishment and reward systems like this testing system are highly DEMOTIVATING for cognitive task. The summative standardized testing system is a really, really backward extrinsic punishment and reward approach to motivation. It reminds me of the line from the alphabet in the Puritan New England Primer, the first textbook published on these shores:
F
The idle Fool
Is whip’t in school.
7. The tests have shown no positive results.
We have had more than a decade, now, of standards-and-testing-based accountability under NCLB. We have seen only miniscule increases in outcomes, and those are well within the margin of error of the calculations. Simply from the Hawthorne Effect, we should have seen SOME improvement!!! And that suggests that the testing has actually DECREASED OUTCOMES, which is consistent with what we know about the demotivational effects of extrinsic punishment and reward systems. It’s the height of stupidity to look at a clearly failed approach and to say, “Gee, we should to a lot more of that.”
PARCC: Spell that backward.
cx: The tests drive how and what people teach, and they drive much of what is created by curriculum developers.
cx: Both science and common sense teach that extrinsic punishment and reward systems like this testing system are highly DEMOTIVATING for cognitive tasks.
And I failed to mention that the tests will ALMOST CERTAINLY acerbate both the white/black and girl/boy achievement gaps and will dramatically affect enrollments of blacks and boys in colleges that will be, increasingly, basing admission decisions in part on these tests (This is one thing that the deformers are pushing for; they want a seamless testing and accountability-based system PreK-college; here in Florida, the governor has just convened a task force of higher ed people to look at how to incorporate test results into admission decisions, decisions about what courses kids will be allowed to take, etc).
Robert,
I was under the impression that males, especially African American males, do better on standardized tests than their grades would predict. Do you dispute this?
If you do not dispute this, I don’t understand how you can think that more emphasis in standardized test scores will lead to fewer boys going to college. The more colleges and universities depend simply on grades (and in my institution at least it is unweighted GPA which does not include any grades from university classes taken while in high school), the fewer males will be in higher education.
TE, you are doubtless referring to the SAT and the ACT. However, I was writing about the new Common Core tests and their effects. New York was the first state in the union to give these, in the 2012-13 school year. 31.4 percent of girls scored proficient (at levels 3 or 4). Only 26.9 percent of males. So, a HUGE gap, favoring females.
It is beyond question that we already losing a lot of our boys. The CCSS is leading to more regimented curricula. This will lead to more disaffection among boys. I expect to see that gap that we saw on the NY test widen.
I am thinking about the research out of Georgia Tech which shows that boys do much better on standardized tests than predicted by the teacher assigned grades. You data about boys and girl failure is perfectly consistent with the finding that boys do even worse compared to girls on teacher assigned grades.
I also think it is important to distinguish between a thing and measurement of a thing. I could devise a measure of poverty that would show higher levels of poverty than the current measure and I could devise a measure of poverty that would show a lower level of poverty than the current measure. Neither would have any impact on actual poverty.
I was not confusing things and measurements of things, TE. I was talking about the consequences of tests. These are not simply measurements without stakes attached and taking place outside any social context and without social consequences.
What I said, TE, is that boys are performing worse on the new CCSS tests. That was the question at issue. That was the question that you raised.
And boys perform even worse if teacher assigned grades are the measure. Is that more accurate?
Looking at those CCSS test results, I would say that that is NOT accurate. In 2010, 8.5 percent of high-school boys dropped out and 6.3 percent of girls. So, a difference of 2.3 percent. The gender gap for CCSS scores was quite a bit higher.
But there is an even more significant issue and that is the issue of attitudes toward schooling. I suspect that boys hate it a lot more. They are told that they must be successful. From Day 1, they are told this. Be a man! Carry that ball! Beat that opponent! And then they get into highly regimented school environments ant they HATE them, and they look forward and they see little chance of living up to the expectations. And so they graduate, but then they don’t go on to college. Why bother if they are just going to fail anyway and end up in some low-level job? We have a lot of disaffected boys in this country right now. It’s a real problem. If we had this big a gender gap in college and it were reversed, people would be screaming bloody murder. We’re failing our boys. I am convinced that the CCSS standardization and regimentation and testing regime is going to make this much, much worse.
TE, those figures are not for teacher-assigned grades. Those are the outcomes on the CCSS tests given to millions of boys and girls in the state of New York in 2012-13. The GIRLS DID FAR BETTER on the CCSS summative standardized state test–the first trial run of PARCC (spell that backward).
Let me give you the working paper link again. Unfortunately the published version is behind a pay wall.
Let me remind you of the major finding of the study:
“Using data from the 1998-99 ECLS-K cohort, we show that the grades awarded by
teachers are not aligned with test scores. Girls in every racial category outperform boys on reading tests, while boys score at least as well on math and science tests as girls. However, boys in all racial categories across all subject areas are not represented in grade distributions where their test scores would predict. Boys who perform equally as well as girls on reading, math and science tests are graded less favorably by their teachers but this less favorable treatment essentially vanishes when non-cognitive skills are taken into account. For some specifications there is evidence of a grade “bonus” for boys with test scores and behavior like their girl counterparts.”
Here is the link: http://people.terry.uga.edu/cornwl/research/cmvp.genderdiffs.pdf
It is still the case in our culture that boys are given a lot more latitude to be independent, to speak their minds, to be rebellious and mischievous than girls are. Girls are still microconditioned to an an enormous extent to get along and go along. As school has become increasingly regimented over the past couple of decades, boys have fared much, much worse. And we now have a huge gender gap at both the undergraduate and graduate levels. 60 percent of undergrads are female. 2/3rds of students in Master’s degree programs. The CCSS and the associated testing are already leading to a lot MORE regimentation. And both the ACT and the SAT are redoing their tests to make them mirror the CCSS. It will be very interesting to see what happens to boys’ score on those.
It is certainly the case that this increased emphasis on test scores will do great harm to children of color.
For all the license given boys, 60 percent of college students are female.
Let me quote from another NBER working paper:
“The decline in high school graduation is of interest in its own right as a measure of the
performance of American schools. It has important implications for interpreting a wide variety of educational statistics. For example, part of the slowdown in male college attendance rates documented by Card and Lemieux (2001) is due to declining rates of high school graduation among males. In addition, half of the growing gap in female versus male college enrollments documented by Goldin, Katz and Kuziemko (2006) can be attributed to higher levels of high school graduation 4 among females and larger declines in male graduation rates.”
Reliance on teacher assigned grades as the monoculture of evaluation will only increase this trend.
For the full paper, including the conclusion that high school graduation rates peaked at 80% in the 1960’s and declined through 2007, go to this link: http://www.nber.org/papers/w13670.pdf
Thanks for the link, TE.
The unions, representing their members, should call for a national boycott and opt-out, and not just because this serves the members’ self interests but because the unions represent TEACHERS, who, if they are worthy of the name, CARE ABOUT KIDS AND DO NOT WANT TO PARTICIPATE IN ABUSING THEM.
These tests are child abuse.
We are supposed to report child abuse.
But when our “authorities” are the ones REQUIRING the child abuse, who ya gonna call?
I commented about the field tests in many earlier posts, stating that even lab rats receive food rewards (while our children are being used as such, and get zip!).* If my child were currently in school, I would DEFINITELY refuse to have her take any such tests–as it were, she probably would have refused, on her own, to take them. I’ve said over and over that, even if parents have been afraid to opt out of the other tests, the field tests were CERTAINLY those that they could/should refuse. As it stands, does anyone SERIOUSLY think that Pear$on is going to look at ANYTHING coming out of this so as to–ha,ha–IMPROVE their beyond faulty, flaw-filled, age-&-grade-inappropriate tests? To quote the great Duane, “my answer is NO!”
I wonder how much money Pear$on MADE on these tests?!
*Of course, the major point is that these tests are an unforgivable waste of precious learning time, and should not be administered at all. There is no compensation–monetary (although, surely, Pear$on would di$agree w/that–it’$ ALL about the money!) or otherwise–that can make up for a lost education.
Please take the time to read, sign, and circulate the petition entitled:
STOP COMMON CORE TESTING.
4,400 letters/emails sent to Congress and President Obama to date.
http://www.petition2congress.com/15080/stop-common-core-testing/?m=5265435
All my different congressmen and senaters, all, reply with the same form letter. Somehow I don’t think they’re hearing us, lol.
Based on the extremely lame and utterly clueless form letter I got from Chuck Schumer, I will never vote for him again
Every form letter I receive tells me much about the people I will not vote for in the next election. If someone chooses to not respond, that tells me something too. And if I actually receive an intelligent thoughtful response, that candidate has my attention. I think it’s worthwhile to write. You learn quite a lot from the responses.
My senators and representative don’t even respond. I live in a VERY red state, so these people having nothing to fear. They will be reelected, regardless, even though one of my senators, up for reelection in four years, will be 84 when he is reelected.
done
I got form letters back from my reps and senators saying how much they care about education and so have put their energies into making sure the Common Core and testing will go forward.
Me: End the tests
Them: I share your concern about education. That’s why I have backed turning all our schools into test prep factories.
They disgust me. All of them. GO GREEN!
I have had it with both the Dim-ocrats and the Repug-licans. Two rival criminal gangs fighting over who gets the grift.
They ARE two rival gangs.
Bob, you have hit yet another nail on its head.
For the dims, it’s not about getting the power to help the ordinary working person.
It’s about the power to grab the power in order to keep the power and get more of it. That’s the name of the game for the dimocraps . . .
With the GOP, this has always been the case more or less, and you know that what you see is what you get.
Not so with the dimocrats . . . .
Most of both parties are more rotten, deranged, and twisted than Jocelyn Wildenstein and her ex-husband.