This post was written by Fred Smith, who worked for many years as a testing expert in the New York City Board of Education. In recent years, he has advised anti-testing groups like Change the Stakes.
I’ve passed the point of exasperation. But, after 40 years in New York’s testing trenches, giving up now is a luxury I can’t afford. Call it over-investment.
And here today is this email I get from a friend sending me the following link from Education Dive (a new one to me), which does a very short summary of a July report from the National Center for Educational Statistics. It compares common core standards and core-aligned test results across the states. Headline: New York Tops List of States with Most Difficult Tests.
http://www.educationdive.com/news/new-york-tops-list-of-states-with-most-difficult-tests/402129/
The study examines proficiency cutoff scores and equates statewide performance on reading and math exams with corresponding results on the National Assessment of Educational Progress. It shows the relative standing of each state in terms of the NAEP scale.
Given the high regard in which NCES and NAEP are held, their methodology and findings must be respected. Here is the link to the report
Mapping State Proficiency Standards Onto NAEP Scales: Results From the 2013 NAEP Reading and Mathematics
I have two immediate reactions to this. First, making a test difficult in a statistical sense–does not make it valid, nor does it make it rigorous, a measure of critical thinking or more challenging—as proclaimed by proponents of core-aligned exams. More than 200,000 children, whose parents opted them out of the exams this year, loudly reject this proclamation.
Confusing, badly constructed items, inadequate time limits, developmentally inappropriate content–make a test “more difficult.” So does taking exams in an unlighted classroom, or when you’re hungry, homeless, sleep-deprived, just learning English or have special needs.
The study itself issued this caution about interpreting the results. The analyses in this report do not address questions about the content, format, exclusion criteria, or conduct of state assessments in comparison with NAEP.
Second, the fact that 2013 is the centerpiece of this report is important. According to the New York State Education Department (SED), 2013 was supposed to be the foundational year—the baseline, if you will, that would usher in the common core and assessments against which progress toward meeting the standards would be measured. NCES’s report merely shows that New York produced the most difficult tests. And…??
Finally, we come back to exasperation. Two recent events let us know that despite our protests and the importance of gaining necessary reform, we the public, remain where the politicians want us, on the outside looking in. The New York State legislature in the shoddiest, last-minute way possible just passed the weakest test-related bill imaginable. It does nothing to require truth in testing, which had been the focus of proposed legislation that was evidently abandoned. (A real T-in-T measure could have passed in this session. It was an extraneous item insignificantly glommed onto a much larger omnibus bill that satisfied the diverse interests of the legislators and was crafted to pass.)
And SED managed to award the next 5-year testing contract to an outfit called Questar Assessments. It has worked closely on testing projects with Pearson, Inc. in the past. And, before the contract was awarded, Pearson was quietly granted a one-year extension of its expiring 5-year contract. It will work to assure a smooth hand-off to Questar, especially in the matter of field testing, which may project Pearson’s involvement beyond 2016. We all know how sound Pearson’s test development expertise proved to be. Why not reward it for more of the same.
Will Pearsona non grata silently become the subcontractor, running the testing program from the shadows. Again, SED has gone about its decision-making with no transparency—leaving us to find out the details too late.
Diane Ravitch's blog 
I immediately wondered…….what if……..there was an all out drive to make difficulty of state tests the number one, super publicized, taken for granted as valid, emphasizing just how vitally important to each state’s value of their children? Could it provoke a level of backlash against the testing industry that is long overdue? Would some of the “reformers” in a few states genuinely not appreciate being made to be look bad by New York. Cardinals were terrible last night, but beat Colorado on a walk off walk……made me look at a few things, probably the wrong way this morning.
The value is to deem Public Education along with the teachers as a failure and then swoop in with the profit driven pseudo solution. It’s simple; create chaos when there is none and reap the financial rewards and leave the shattered pieces behind when you are done pillaging.
“We’re number one — for hard tests”
We have the highest fails
Of that we’re really proud
They can’t make heads or tails
And flipping ain’t allowed
Love it, SomeDAM Poet.
With all the money they’ve poured into promoting Common Core I think it’s weird there’s so little information given to parents on the “cut score” process. Are we just to accept whatever 1 thru 5 label they hand down as completely valid? Based on what?
It’s even stranger in Ohio because I’m not clear what I’m supposed to do with this information. If my kid is labeled a “3” in English and the state of Ohio just told me the test they chose is not good enough and will no longer be used, why would I accept the next test and the next label? Is that one any good? Aren’t the same set of people picking it?
They’ve been testing kids in this state for 15 years under an ed reform regime and with the CC tests they announced the tests were too easy and Arne Duncan says various people were “lying” to us. Not him. Just those other people. Have they gotten better at their jobs? Are the liars gone? It looks like the same crew of ed reformers to me. Why is this test valid when those were not?
For kids that are currently in the middle of their K-12 progression, like my son, will the entire remainder of their public school be made up of fine-tuning Common Core tests?
How is that fair to them? Shouldn’t these adults have had a plan past “market this aggressively”? I think it’s great the adults selected a “sacrifice cohort” but maybe they should have given that some thought as far as how it affects individual students who happen to be in the middle when they launched their nationwide, experimental; program. They never would have done this to adults, because adults would be screaming bloody murder if they changed the rules midstream. That’s why we have the whole concept of “grandfathered”.
Stop all these unnecessary testing!
The NYTimes article about StudentsFirstNY published yesterday explains the pressures pushing NYS educational policy.
Why are NYS’s tests hard? Because difficult tests impress the small group of wealthy men and women who are driving policy.
Aaron, the tests are so hard because they are intended to make the public schools look bad, thus encouraging more privatization of public resources
Thanks, Diane.
A defense of public education is a better message than something merely negative.
Also, somewhere mixed in there, certain people believe that school (and life) should be a competition, and the winners should be further separated from the losers.
I’m wondering if we’ll see parents and kids seize on the easy 1-5 scale and use it as a proxy for everything. I suspect they will just because people love rankings and they’ve made it so easy for people to follow that inclination and “numbers don’t lie!” Except when they do.
I wonder how it changes things for younger children to have a national one-digit number attached to them as early as 3rd grade. I don’t have any confidence the Best and Brightest have considered any of this, given what I see as their general recklessness and their own love of and embrace of numbers as proxy for a whole range of human value.
It reminds me of that part in the Giver, where all of the children are numbers, not names, until they are “given” to their “parents.” The parents would sometimes call the children by their number instead of their name. Isn’t that the same thing here? Should I be calling my students by the number of their ELA test score (“Thank you, 4 for helping 1 over there with her spelling.”)? BTW, that last line was sarcasm.
1-5 = A-F = MMoOO!
People shouldn’t focus exclusively on test scores when measuring children, except the entire governmental apparatus does, and so does the entire ed reform “movement”, and they drive that message home daily:
“Minn. student test scores show little change despite vow to improve Star Tribune: In reading, nearly 60 percent of students mastered state standards, compared with 59 percent in 2014. In math, 60 percent of students met math standards, down from nearly 62 percent in 2014. White students continued to outperform students of color by more than 20 percentage points on average.”
Do they not see the contradiction? How they say one thing and do another, every single day? Nothing has changed in ed reformland except the campaign rhetoric and even that isn’t consistent.
When I saw Questar I knew something was fishy! They came out of nowhere and there was no complaint from Pearson, I just had a feeling. They are a shill for Pearson, the puzzle is fitting together now. We are so out of our league, it’s as if we are fighting a guerrilla jungle war and we are lined up like we’re in the American Revolution Continental style formation. We’re following rules and they are gangster sociopaths. Rules don’t apply!
“We’re following rules . . . ” Who is the we?
Frontline educators. I was a School Counselor (Licensed Clinical Social Worker w/ K/12 teaching credential) for 24 yrs. jumped through every hoop put in front of me,including having to get another credential because I was not “highly qualified” as per State. This inspite of being licensed to hang out a shingle & take insurance, having done my job and much more, for twenty years at that point in time. With positive evaluations based on my teaching Family Life Education, Nutrition & health,6th grade mathematics, because none of the administrators understood COUNSELING!!! I was run out because the district is buying iPads and paying WestEd for useless trainings for big bucks. We are developmentally inappropriate and I fear for our future.
These worthless test results are due back in November. Please could someone tell me how in the world I will be given value added going from the Ohio Achievement Assessment to PARCC? I have already accepted the fact that I will be rated ineffective next year. The “skilled” rating from my principal cannot overcome this unfair, silly nonsense. I will do what one blogger said “Make a basket with the unopened envelope.” I had some students finishing in 30 minutes with a scribbled writing. I’m sure they will make me look like “Teacher of the Year.” No more of my family members will suffer in this crazy “career” or “job” or whatever we want to call this abuse. 😜😜😜
Crooked is as crooked does!
There is nothing trustworthy about the NAEP as an independent governing body or developer of tests. I say that because the Vice-Chair is now Susan Pimentel, founding partner of the nonprofit Student Achievement Partners, a lead writer of the Common Core State Standards for English Language Arts and Literacy, and the chief architect of the antecedent American Diploma Project Benchmarks marketed as essential for college and career readiness.
So the entrapment of almost all public K-12 and teacher education into so-called Common Core State Standards (CCSS) is rounded out by this appointment. It makes a mockery of the objective of “maintaining the integrity and independence of The Nation’s Report Card ( NAEP) as a trusted yardstick of student achievement since 1969.“
It is useful to recall that the Common Core State Standards (CCSS) were published in June 2010. For good and well-documented reasons, the CCSS are standards that Bill Gates wanted and paid for.
The first year the CCSS could have had any presence in schools for serious collaborative study by teachers and administrators would have been 2010-2011.
Let’s assume all educators in NY (or any state that signed up for the CCSS) had nothing to do all of that year other than comprehend the implications of the 1,620 standards (including parts a-f). That is an absurd assumption. Even so, 2010-2011 should have been the year that personnel in every CCSS state conducted a grade-by-grade comparison of their current standards and the CCSS so they could transition to the new CCSS standards in 2011-2012 and also acquire needed resources.
Now, assume that kind of concordance/alignment analysis was actually completed for every grade and subject, in order to get a clear idea of the degree of “fit” between the old standards, the CCSS, and the existing curriculum resources. That kind of analysis requires steps such as annotating how existing standards and the CCSS are alike and different with attention to the breadth, complexity, and “rigor” of content and skillsas well as their grade-by-grade distribution. There is much more to such alignment studies, but the point is that this lengthy exercise should be accomplished by teachers–the best way to see how everybody is implicated in the changes. And that work should have been completed in time to order new CCSS-compliant resources as needed for the school year 2011-2012.
Of course the logic of this process differs from the realities of life in schools, and by some changing instructions to publishers of resources, and the budget issues, and time to get acuainted with all of the new materials before the start of school. Given all of these contingencies, it is probable that most schools started the 2011-2012 year with a mix of still “useful” teaching resources for standards approximating the CCSS and relatively few rigorously compliant new resources. In effect, coherent and “almost CCSS-compliant” instruction might begin this year, assuming all miracles have been accomplished.
But here is also the oopsy for 2011-2012. The only students who will experience all of the full wonders and labyrinths in the CCSS are in Kindergarten. The 2011-2012 cohort of first graders is already behind because, if you don’t know, the CCSS are organized around tight learning progressions to be mastered on time, grade-by-grade. The poor first graders missed all of the “challenging” Kindergarten pre-requisites, so they and their teachers have to do something pretty amazing to stay on track to college and career readiness. They have to master all the Kindergarten progressions they missed and all of their first grade progressions as well.
By the same CCSS logic, the make-up burden for the 2011-2012 cohort of 2nd graders, includes the learning progressions from Kindergarten and first grade in addition to those at their grade level. And so it goes up the grades. The eight graders in 2011-2012 have not had the claimed benefits of all of the learning progressions in prior years—the content and skills they were supposed to master in Kindergarten and in grades 2, 3, 4, 5, 6, 7. This phenomenon of being perpetually behind is the result of cluess policy-makers telling teachers to “fly the plane while building it.”
Now comes school year 2012-2013. In NY, and perhaps other states, the students in grades 3-8 are tested on the CCSS in math and English language arts. The eight graders have been thrust into the new CCSS-compliant instructional program without the benefit of K-7 instruction in the learning progressions—about which there has been so much hoo-ha.
Recall that the big hoo-ha of the standard-setting process was that all of the pre-requisites for college and career readiness were “backmapped” (reverse engineered). If students began in Kindergarten and met or exceeded the standards for each grade—completed all of their grade-by-grade learning progression on time—they would be ready to ace the one-size-fits-all standards for entry into college and/or career.
In New York State, the 2013 end-of-year tests for grade level mastery of the CCSS were either unfair by plan or by ignorance of the logic around which the CCSS were organized, marketed, and foisted on students and teachers. And that logic had everything to do curriculum and instruction from the get go. The claim that the “standards had nothing to do with curriculum or instruction” was false.
Here is an if-then proposition. If, in 2012-2013, the implementation of the CCSS started with the highest level of fidelity to the standards, then tests for the Kindergarten cohort in the Spring of 2013 year might plausibly measure the “progress” of those students moving along toward the grade one expectations. by the logic of the year-to-year theory of moving toward college and career readiness, all of the students in grades 3-8 tested in Spring of 2013 , should have been given something like a handicap score (a golfing term) to compensate for being tested before they were taught the content/skills from prior grades.
If the Kindergarten class of 2012-2013 is tested every year, under conditions of sustained high fidelity to the CCSS (with no re-formy distractions) then in theory (only in theory) they can be fairly tested every year until 2025, at which time there would be the very first proof of the efficacy of this grand experiment. That assumes the tests are fair. Clearly they are not.
The logic and theory of action imbedded in the CCSS is beyond salvation by any program of testing. Any testing program based on it has a high probability of being invalid in what it purports to measure. That invalidity will be exacerbated by using those scores to evaluaate teachers.
The frauds attached to the CCSS are made worse by the positionheld of the Vice Chair of the NAEP Governing Board– one of the main authors of this deeply flawed architecture for education. I believe there are good reasons to call for her immediate resignation so that the good name and reputation of NAEP as the independent measure of achievement can be preserved.
If it is, relevant, I speak from experience with standards-based curriculum development since the 1960s, as well as item development and interpretation of results from the first two NAEP visual art assessments.