New York’s Phony Test Score Increases

By dianeravitch

August 15, 2014 //

Yesterday, state officials were celebrating the latest test scores. English was flat; math was up a few points.

But stop the party!

It turns out that the passing mark was lowered.

Categories Common Core, Testing

32 Comments Post your own or leave a trackback: Trackback URL

NY teacher says:

August 15, 2014 at 1:54 pm

No statistician would ever use the phrase “a 0.1% IMPROVEMENT” in comparing the 2013 and 2014 ELA test results.

Especially after changing at least two variables (instruction and the tests).

One tenth of one percent is clearly within the margin of measuring error regardless of the sample size of the test taking group.

Reply
Arthur Camins says:

August 15, 2014 at 1:56 pm

Different cut scores between proficiency levels might be the result of manipulation for desired results. However, small shifts from year-to-year in one direction or another when attempting to make comparisons across forms of a test (or across years) is not only not unusual, but a necessary process. The process is called, equating. See: http://en.wikipedia.org/wiki/Equating. For any high stakes test of this sort, the test makers should make public in its technical manuals regarding how the process worked to justify shifts in the relationships among raw scores, scale scores and proficiency levels.

Reply
- Duane Swacker says:
  
  August 15, 2014 at 5:53 pm
  
  “. . . raw scores, scale scores and proficiency levels.”
  
  All are psychometric mental masturbations to make the process appear “scientific”.
  
  Reply
kathy says:

August 15, 2014 at 1:56 pm

and don’t forget, these are results of different students!

Reply
- concerned mom says:
  
  August 15, 2014 at 7:24 pm
  
  I never understood why they call it growth when the students change.
  
  Reply
  - Cupcake says:
    
    August 15, 2014 at 9:15 pm
    
    You don’t understand it because you obviously have a brain in your head.
  - Duane Swacker says:
    
    August 16, 2014 at 2:32 pm
    
    TAGO, Cupcake!
    
    Have to nominate that one for quip of the year!
schoolgal says:

August 15, 2014 at 2:02 pm

Isn’t the bottom line really that this is an election year. And after the fallout from parents, this was a strategic move. btw, each state sets their own cut offs which really makes comparisons from state to state unfair as well.

Reply
Steve Cohen says:

August 15, 2014 at 2:02 pm

It all works out if one uses the VAM model Chetty developed.

Reply
- Duane Swacker says:
  
  August 15, 2014 at 5:53 pm
  
  Subtle but TAGO!
  
  Reply
Frustrated in Brooklyn says:

August 15, 2014 at 2:14 pm

Also, the scores have not been released to the individual students or teachers yet, at least in NYC. So much for us teachers “using the tests to drive instruction” — the release date is later than it has ever been. When will the individual student results be available for parents, teachers and students — the people who really matter? Oh, wait, it’s all about the big guys, not us at all!

Reply
- dianeravitch says:
  
  August 15, 2014 at 2:44 pm
  
  If you can’t see the questions, what good will the scores do for teachers or students?
  
  Reply
  - NY teacher says:
    
    August 15, 2014 at 2:49 pm
    
    And if you did get see them like I did, you would realize that the test items were so poorly constructed (especially ELA) that they made the scores irrelevant. Bad tests cannot inform good instruction.
  - Duane Swacker says:
    
    August 15, 2014 at 6:04 pm
    
    And the vast majority of tests, even many made by the classroom teacher, are bad from an epistemological* and ontological** view.
    
    *Epistemology (from Greek ἐπιστήμη – epistēmē, meaning “knowledge, understanding”, and λόγος – logos, meaning “study of”) is the branch of philosophy concerned with the nature and scope of knowledge[1][2] and is also referred to as “theory of knowledge”. It questions what knowledge is and how it can be acquired, and the extent to which any given subject or entity can be known. (from Wiki)
    
    ** Ontology is the philosophical study of the nature of being, becoming, existence, or reality, as well as the basic categories of being and their relations. Traditionally listed as a part of the major branch of philosophy known as metaphysics, ontology deals with questions concerning what entities exist or can be said to exist, and how such entities can be grouped, related within a hierarchy, and subdivided according to similarities and differences. (from Wiki)
ira shor says:

August 15, 2014 at 2:28 pm

Good find! How do we get these manipulations of data into media circulation? What’s a media strategy to counter bogus claims about “improvement”?

Reply
- Akademos says:
  
  August 17, 2014 at 7:43 pm
  
  New York Post: Cut Offs Lowered
  
  http://nypost.com/2014/08/17/dept-of-ed-officials-adjust-proficiency-thresholds-for-common-core/
  
  Reply
  - Akademos says:
    
    August 17, 2014 at 7:48 pm
    
    Cutoffs
Michele Hamilton says:

August 15, 2014 at 4:41 pm

Is that why NYSED needed until mid August to release scores of tests that were taken by students, and graded by teachers, in April?!

Reply
- Duane Swacker says:
  
  August 15, 2014 at 6:05 pm
  
  YEP!
  
  Reply
retiredbutmissthekids says:

August 15, 2014 at 5:07 pm

Same silly bragging going on in Chicago. And, you know, what’s the special du jour?
One day, it’s state D.o.Ed. heads bemoaning the low scores–gasp!–MORE “failing” schools! At other times, it’s boasting about GREAT test scores (& they’re only talking mere %age points of “improvement”). ALL of us know that this is nothing more than utter nonsense–of course, the tests themselves are NOT standardized–neither valid nor reliable, much less worth the paper (all the trees killed for this, not to mention all those #2 pencils!) they’re printed on. :If you can’t dazzle them w/your brilliance, baffle ’em w/your b.s.” NO–we WON’T be fooled. Lee County FL School Board–GO! All you folks listed on the Fair Test links–GO! Districts, towns, cities, suburbs, counties, states–ALL over the U.S.–OPT OUT NOW!!!

Reply
Chiara says:

August 15, 2014 at 5:50 pm

It’s such a joke that we’re telling these kids tests are just “assessments” to “see where they are” when all the adults are treating test results like they’re the focus of everything.
They’re not idiots. They know the tests scores are the be-all and end-all. They know it because the whole focus is on test scores.
The least we could do is admit the obvious “we measure you, your school, your teachers and the political leaders of schools on your test scores”
This ridiculous denial must make them wonder. At least in South Korea they tell them the truth.
They all know it anyway. My son does, and he has spent his entire time in public schools post-NCLB and RttT.
We base our entire system on test scores, and have for a decade. Adults made that decision.
They should quit lying about it and pretending it’s any more nuanced than that.

Reply
concerned mom says:

August 15, 2014 at 7:22 pm

They did the same in NC we went from scores of 1-4 to scores of 1-5. A 3 is passing, but a 4 is college ready.

Reply
rratto says:

August 15, 2014 at 8:42 pm

Manipulating cut scores fits right into to MO of the NYS Board of Regents. They will do anything to make their point. Let’s not forget how they manipulated college readiness data and remedial help in community colleges. Let’s not forget how they ignore high graduation rates and gaps that have been steadily closing. Let’s not forget how they ignore inadequate funding and research on class size.

So they released scores… just how does that inform my instruction on brand new students with much different needs than my class last year? I don’t even know which questions my class did well on and where I should focus.
In the meantime NYS marches right along as we prepare to abuse our students yet again with hours and hours of tests..

Reply
Joseph says:

August 15, 2014 at 9:11 pm

These are the tools to dumb down our progeny. Everything right on target, Eva Moskowitz’s Success Academy had a 94% pass rate. That should trigger an immediate investigation, just based on statistical probability of her population.

Reply
- NY teacher says:
  
  August 16, 2014 at 7:33 am
  
  Maybe she got some friendly test prep advice from She Who Shall Not Be Named aka “Eraser-head”.
  
  Reply
- spw says:
  
  August 17, 2014 at 7:30 am
  
  You are right! Her schools scored significantly higher on these tests than did the other charter schools-The same thing occurred last year.. Even the NYC Charter Association mentioned S.A.”s high scores and mentioned how some would question these high scores.. Something is rotten in the State of Success Academy Schools and they should be investigated. If a traditional public school showed this kind of unexpected growth, it’s testing procedures would be -and should be-investigated.
  Seth
  
  Reply
Joseph says:

August 15, 2014 at 9:15 pm

Eva, how do you do it, and you never being a teacher? http://www.nydailynews.com/opinion/score-core-article-1.1904103

Reply
Lily says:

August 15, 2014 at 10:38 pm

Are the state exams taken by children in NY charter school scored by public school teachers or by other charter school teachers? Who exactly grades the charter school exams?

Reply
Lily says:

August 15, 2014 at 10:39 pm

*charter schools

Reply
Anne says:

August 15, 2014 at 11:14 pm

This reminds me of how I used to marvel over the years at how scores had changed on passing and failing. Used to be 95-100 was an A, then 94-100 was an A, now 93-100 is an A. And, so it followed down the line with all the other grades. Used to be 70% was a failing grade. By 2012, 67% was a failing grade. I kept up with it as a teacher, over the years. It never ceased to amaze me how it would gradually get lower and lower. What does that tell you?
I also meant to say this happened in Louisiana. They weren’t fooling anybody. It made me sick. Just to have more children who would pass the tests in class.

Reply
- Duane Swacker says:
  
  August 16, 2014 at 2:41 pm
  
  Sucked into the whole “grading students” canard are you, Anne???
  
  Grades as such are pure bovine excrement (and that’s exactly what I tell my students).
  
  No grade is accurate enough to even worry about those minor adjustments, not to mention all grades in different classes are based on different things, etc. . . . Grades are bullshit and egregious educational malpractices that should be abolished.
  
  From Wilson:
  
  It requires an enormous suspension of rational thinking to believe that the best way to describe the complexity of any human achievement, any person’s skill in a complex field of human endeavour, is with a number [grade] that is determined by the number of test items they got correct. Yet so conditioned are we that it takes a few moments of strict logical reflection to appreciate the absurdity of this.
  
  Reply
Blind Noise says:

August 16, 2014 at 6:41 pm

This is also true of the new, rigorous, Common Core-aligned high school equivalency exam. It is indeed harder, especially the math and essay sections. However, one needs a 1, rather than 2, to pass the essay. A nontrivial number of Students who failed all or some sections of the GED passed this test, though they almost universally reported struggling through it. Many students walked out, or did not return for the second day of testing. Somehow, though, the passing grades are making it possible for students who are not necessarily ready for college to pass. Data is so very important; it can be so easily manipulated.

Reply