School officials across the nation keep warning that the new tests are “harder,” and passing rates will drop by 30% or more.
Why?
The passing mark–or cut score–on tests is not determined by science. It is a judgment call.
Those who are in charge decide where to place the passing mark.
If the scores go down on a test that is new, it is because the officials set the passing mark with foreknowledge that scores and passing rates would fall.
In addition, the state education department piloted the questions, and they know with a high level of precision which are hard and which are easy.
If the scores fall, they were designed to fall.
This is what they want.
Why? I don’t know. Ask them.
Diane Ravitch's blog 
Well then , they have the legal authority to convert to the much, MUCH pushed charters. Isn’t that convenient. What happens to Public education at that point??? What happens to the the special needs children that are not served by charters?? I don’t understand the HUGE push for charters…
we all know why, so that the crooked politicians and their corporate paymasters like Gates, the Waltons, Broad, Bloomberg etc. can point to public schools and say “see, the public schools are failing our children, we need charters to save them”
these people should be lynched by the public for the harm they’re doing!
-ask questions, start trouble-
I am glad to see this post. It is the simple truth of the manipulation of a testing system that can easily yield whatever results its manipulators wish.
In the end, the setting of cutoff scores makes the seeming “objective” standardized tests capriciously subjective.
This is so very true.
Nothing is going to happen until the parents find out exactly what is going on..
Several lawsuits may put these TESTING HIERARCHIES IN THEIR PLACE.
I am often amazed and disheartened at the number of people, including teachers, who do not know that cut scores are arbitrary. Or that the tests themselves are subjective. Many teachers think that the testing suppliers have some knowledge that they themselves are missing, that allow the companies to create these magical unproblematic assessments. Just another symptom of the undermining of the teaching profession, I suppose.
Is it to benchmark where they are starting from with the new Core Curriculum standards?
It is harder now so that they can then lower the passing scores and state how wonderful the curriculum is, how well it is working, look how many are now passing since all the reforms!
I believe it so the cut passing scores can later be lowered, or the test made easier. Then these people will point with pride and say how well the “reforms” have worked, look how much growth has been shown…
Let us field test this by the prominent Rhode Island volunteer testees and see how they do. Since many of them would not have graduated from high school, according to their results, we need to ask for their assistance again to make sense out of this nonsense.
Puts craziness in perspective. Do we keep moving the goal line in football, add extra holes in golf, raise the net in tennis, ask soccer players to play water polo? Silly, you bet! At ,east kids and tchrs are not paying the price for this mean-spirited practice. Absurd!
And that’s only part of the story of high-stakes testing.
The quantity and frequency of tests are beyond excessive.
These tests are now to be the basis of anywhere from 20% (NY) to 50% (CO) of a teacher’s evaluation. So – the definition of “proficient” is arbitrary and the new tests are not yet valid or reliable (statistically speaking) but they going to be used to label teachers and principals?
So let’s see, in NY, the cut scores were arbitrarily changed three years ago… then new standards were introduced… there has been zero time for curriculum mapping to align with the standards (like them or not)… but the new tests will be based on those standards so scores will drop yet again… and the state still has not decided if they will use a value-added measure (VAM) or tradition measures to be applied to teacher evaluations… and soon the tests will be untested in practice but still in play online with up to 10 hours of testing 3rd graders.
How about analysis of cost-benefit (voice over of car salesman commercial commentator: “And all this for the low, low cost of $700 million”)?
How about an analysis of the state’s value-added (they are caught up with teaching value-addedness but how much value is being added by this change everything-at-once approach)?
How about a statistical analysis of reliability and validity before using the tests and of this ever-moving target?
Can’t see the forest for the trees and there’s a lot of tree thinking going on in state departments and DC.
Ha❢ If they ever get as far as seeing a whole live tree, that would be an improvement. There’s nothing but knothole thinkers coming out of that woodwork.
How many privileged parents will pull their children out of public schools as a result?
Perhaps Rick Hess really had it just about right:
Garbage in; garbage out— GIgo
Marge
Sasha and Malia don’t have to do this…..it’s time to call out all the policy makers on the choices they make for their own children.
Do you think it is expected because all states as members of the consortium are not allowed to determine their own cut scores? Under the agreements, signed by the governor and Superintendent of Education from each state, they must have a uniform cut score as determined by the governing states of the consortium. Everyone must be judged under the same criteria in order for the data to be consistent. This is a huge abdication of state control.
My understanding is that common core will bring with it a revolutionary change in test scores because of the nature of the testing itself. Students will no longer be asked to select a, b, c, or d, but will instead have to show their thinking through written discourse, illustrations, etc. There will be no more guessing at answers. Math ability will be judged on the conceptual understanding of the students as well as on the answer they come up with. Students will have to produce answers on their own, not select from a menu. Understandably, test scores will plummet. But there will be no point in comparing multiple choice style test scores to common core tests. It would be like comparing candle wax to orangutans.
Sounds good. I assume educated professionals will then review these billions of students responses to assess them?
O, no, that’s right, Pearson Education will get paid hundreds of millions to do that job even if they–and the whole testing industry–have a documented history of scoring errors. Read Todd Farley’s “Making the Grades: My Misadventures in the Standardized Testing Industry.”
Or, maybe they’ll score all those open-ended student responses with Automated Essay Scoring Engines THAT CANNOT READ.
true, true.
They were designed so that all public schools will be labeled failing. Then parents will beg for charters, private schools, vouchers, etc. This was obvious from the beginning. This is the kill plan for the good suburban public schools. I am in one of the top 200 public high schools, and I estimate that a maximum of 30% of our students would pass these tests. Imagine how rural and inner-city school students will do on these tests. The elites simply had to devise a plan that would label all public schools (even the high-peforming suburban public schools) as failing. Let’s see if the public goes for it. I wonder how many adults would pass..ha ha. I would love to give these tests to Exxon executives, senators, etc. It would be hilarious.
John — you are absolutely right. The purpose of a system is what it does.
Marge
bingo!
ACT says it has studied the scores of students on its tests to see which students earn A’s’ or B’s in the freshman year of college. It would be interesting to see what a neutral person thinks of their analysis. And of course, earning an “A” or “B” in the freshman year does not necessarily mean you will go on to be an active, positive person.
Otherwise, as noted, the “cut” score on statewide tests often is a judgement.
You’re discussing a pet peeve. In Virginia, I have served on a number of DOE test-related committees. A few years ago, I served on a committee which had as its final task to recommend failure (!) rates for end-of-course tests. The committee (of my peers!) recommended that students of the subject which I taught should have a higher failure rate because they were older and more knowledgeable. Therefore, the test should be tougher to ensure that more students failed. The group leader, I understand that she was a Pearson employee, explained that this would be done by choosing enough questions of sufficient difficulty to provide the expected results. Translation: my bright ESL students and hard-working students with special needs would have to cope with a more challenging vocabulary to figure out what the heck they were asking. Of course we were all sworn to secrecy on all the particulars of the meeting. I hope my comments are sufficiently general to not violate that oath.
Of course, they have other ways to make certain that enough children are left behind. I understand that in Social Studies, only one test has ever been released to the public,and that was after the standards were changed.
KR in VA: there is absolutely nothing unusual or unexpected or surprising about your firsthand experiences with high-stakes standardized testing.
If you have not already read the following, I suggest you consider getting them and reading them in the following order:
1), Todd Farley, MAKING THE GRADES: MY MISADVENTURES IN THE STANDARDIZED TESTING INDUSTRY (2009);
2), Phillip Harris, Bruce M. Smith, and Joan Harris, THE MYTHS OF STANDARDIZED TESTS (2011);
3), Daniel Koretz, MEASURING UP: WHAT EDUCATIONAL TESTING REALLY TELLS US (2008).
If you find them worthwhile, recommend to others.
FWIW: The first two are much slimmer and easier reads because they are destined for broader reading audiences; they will prepare you for the third, which is by a genuine psychometrician, i.e., a numbers/stats guy who literally is a professional standardized test explainer/designer/evaluator.
I will not presume to know what your reactions might be, but I learned more than I ever really wanted to know about designing, administering and scoring high-stakes standardized tests, and at times the learning process involved feeling alternately uncomfortable, outraged and nauseous.
Thank you for your posting.
Education is a huge market. High cut scores create mounds of failure. The failure creates a market for test prep materials. Doesn’t Pearson quadruple dip: standards, tests (for teachers and students), curricula, and test prep.
You GOT IT RIGHT!!
They are on bed with the publishing co and especially the one you mentioned..Big Time!!
The assessments are harder simply because the CCSS standards are higher. The better question is why the standards are higher, which is a valid question. The report we’ve gotten is that such standards support college readiness, which makes sense, but I agree that there needs to be more than professional judgement/opinion used to develop expectations.
As an analogy, imagine if we decided to set different child development expectations because we decided that kids needed to have more self-control at an earlier age. This wouldn’t make sense, because child development benchmarks aren’t something we decide, but something we discover. This isn’t to say we can’t do anything to speed up or support some of the processes, but we can’t do something like deciding abstract reasoning should come earlier in the process of neurological development.
Still, none of this has to do with the assessments, but with the new standards. The assessments seem to make perfect sense given the context of the standards.
Read again.
They are the most ridiculous questions …
I haven’t read the CCSS assessments, but I can’t imagine that question selection is making them 30% harder. Without any more evidence, it seems to make a lot more sense that those tests are harder because they’re attempting to assess more difficult material.
However, I’m open to being wrong – have you read a report/research that suggests that question selection is making tests that much more difficult?
Ed, there is no such things as “tests are harder.” Those who construct decide how many difficult questions to include–those that very few kids can answer; how many that almost all can answer; and how many in the middle. Those in charge also choose the cut point.
It is all subjective.
Diane, I’m not sure we are talking about the same idea then when we say “harder.” For example, constructing a test in which a greater number of children score higher on the measure would make it easier, which could easily be measured. Maybe I don’t understand what you mean when you say “harder.”
Ed, state officials say that the failure rate will be 30% higher because the tests are harder. They assume that by some magic, hard tests will make kids smarter. Magical thinking. Testing companies and state officials determine how many will fail because they decide where to put the passing mark. They can make it hard or easy, whatever they want. Their choice. It’s arbitrary.
I think I see what you are saying now, Diane. I think we’ve been talking about 2 different things. In reality, there are at least 3 ways for 30% more kids to fail:
1) The test style is harder, but the content is the same. (e.g., greater percentage of question types that are difficult).
2) The test style is the same, but the content is more difficult.
3) Both the test style and content are the same, but the cut score is raised so that fewer kids score that high.
It seems that you are assuming that only item #3 is happening, and I’m wondering why. CCSS is known to include more difficult content at an earlier age, so why have you ruled out #2 as a potential reason for the 30% decline?
Ed,
Test developers know exactly how every question will perform. They always have the ability to make tests harder or easier by weighting it with harder or easier questions. And they can adjust the cut score to produce the result they want. It’s not rocket science. It’s arbitrary, completely and totally arbitrary. Nothing objective at all
Diane, I’m still not seeing how you are drawing the conclusion that cut scores are arbitrary. It seems that you are making a (valid) statement about test construction in terms of question difficulty, and it is true that cut scores have the potential to be arbitrary, but they also have the potential to be scientifically based as well. For example, it would be possible to examine correlations between performance on one grade-level assessment and a future grade-level assessment, and based on that correlation determine cut scores that predict future performance. So, if a child scored 80% or higher that may correlate with “low-risk” for poor performance on a future assessment. In that case, the cut score is not arbitrary, but determined to predict future performance, and identify kids that might be at risk for future difficulty.
Exactly. As a teacher who has half of a schedule with AP seniors, I create a vicious test every few months to remind them that their laziness will not be tolerated. I have created some monsters that are fair but exceedingly difficult. On the other hand, I could create cupcake questions to guarantee A’s. It’s all up to the test maker.
Point blank. Thanks, Diane. Please keep saying it in these simple terms. I have a PhD in psychology (expertise in children and schools) so I do understand more about testing than the typical mom would. It is a JOKE how they are already playing this one…setting everyone up to fear the “drop.” And it is working. The principal at my children’s school handed out articles about the Kentucky drop to our leadership council.
It isn’t the same test. It has not been established as having any type of reliability or validity. It won’t have the same cut scores, which are set arbitrarily, and it is clearly going to be developmentally inappropriate for at least the younger end of the age range. Children (and some adolescents) don’t reason conceptually in the ways these tests will demand. The curriculum, apparently developed by working backwards from 12th grade without regard for the COLLECTIVE WISDOM OF YEARS OF RESEARCH by professionals across the fields of education, child development, psychology and others, utterly disregards what we know about children. Nancy Carlson-Paige and others have already written about this issue and Diane has shared these concerns on her blog. I keep hoping Matt Damon is making a documentary about this stomach-turning travesty in education. Surely he has been tempted to turn on the camera after hearing his mother and others to whom he would be exposed explain the situation. It could be the game changer. I fear even Merrow’s new information on Rhee will not suffice in turning the tide. Those people have no shame.
Any progress with Michael Moore investigating things like this?
Good point! I’d like to see that happen!
It is a well known phenomenon that when you change the test, even if by the same test maker in a similar format, the test scores go down and then go up in subsequent years even if nothing different happens in regard to instruction and student population. That is why new superintendents benefit when a new test comes in with them (that benchmark will inevitably rise no matter the competence fo the leader or the supports he does or does not give to staff.
It is not necessarily a plot to make the tests harder. The new tests will result in lower scores no matter what the cognitive demands. Kids who are familiar with problems formatted as: 1
+2
will often have difficulty with the same problem when presented as 1 + 2.
But all the tests are different at this point (supposedly) because the states have their curriculum that they are testing. So even if something as simple as formatting might create changes in some states, why are we ALL being prepared for the drop? It is a conspiracy, of large proportions.
Another thing that annoys the heck out of me is that good test development usually involves lots of pilot studies and correlational studies, etc. It should take a good long while to develop a test and get it right. A test of anything should correlate to some positive degree with a test that has been proven to test that same “anything.” This process establishes validity. It helps to answer the question, “how confident are we that the test measures the construct we say it is measuring?” A new test should also be given to a group and then given to that same group again to see if they perform in the same ballpark. Are the scores all over the map or do we tend to get the same basic result (assuming not too much time has transpired and taking into account practice effects). A new test should also be studied for bias. Subgroup results should be examined to see how people of different gender, race, etc. perform. The goal isn’t to avoid any differences at all among subgroups, but to establish that there isn’t some unintended bias in the test being developed. Individual items should be analyzed and bad items should be tossed while items that get at what you hope to measure should be kept.
So, I am no test development expert…these are just very basic ideas. But I wonder if any of it is happening before this new round of insanity to which we will be subjecting our children. It seems to me the testing industry, with the able assist from NCLB And RTTT, has redefined achievement. Wow, that is a major accomplishment. And all so my little ones can be deemed college and career ready, or not. What a joke. It is all for profit!
Say it again!
Well Kay,the whole process of making “standards” and standardized test is fraught with error that renders the whole thing invalid and by definition unreliable. Read Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 .
Since the whole process is invalid any conclusions/inferences drawn from it are, as Wilson states, “vain and illusory”. In other words a complete waste of time, resources and money.
Scare tactics. Our union had a “question the Superintendent” forum last week. In discussion of the contract violating, mandatory summer CC training required of teachers, the Superintendent tossed out this 30% figure. Teachers all across the room gasped. “Oh no!”, agreeing this was bad. Sheep!
30% less that what? Who cares? Where’s the beef? I went into denial, refusing to believe what I was seeing and hearing.
If everyone drops 30%, that is bad? If everyone was expected to increase by 30%, would our “problems” be solved?
It makes sense now because Obama’s RTTT is really just a continuation of the untenable plan to have 100% proficiency under GW Bush’s NCLB by 2014 or be deemed a failure. This is similar to GH Bush’s Goals 2000 plan to have 100% of children ready for school by 2000, when nothing was put in place to assure even a modicum of success. Politicians set these unrealistic goals of 100% compliance, while educators scratched their heads and wondered why their country’s leaders were setting them up for failure.
Why that was done initially was unclear, but it is obvious today. The point is to prove that public education is a bust, so that it can be dismantled and handed over to privatizers who can profit from it, as has been happening in our cities. The suburbs are next.
Milton Friedman’s plan, initiated under Reagan and instigated by the fear-mongering of “A Nation at Risk” in 1983, is coming to fruition. Who could have predicted that it would be neo-liberal Democrats casting the final blows to public education, in order to serve the “free” markets?
Public educators are working very hard to prepare our students for their futures. I, for one, am not concerned about standardized test scores. I want my students to be well read, creative, and able to problem solve. I also like to help them become good little people in this big messed up world. We are being forced to teach in a ridiculous manner and follow outrageous les that do not make sense. All at the expense of our hildren’s education. Our hands are tied.
You are right to “TeachWithYourHeart” and to give to children what you know cognitively they are needing as well.
When what surrounds us and is supposed to direct us is not in the best interests of children, we have to take a stand and do what is right for children, because that is precisely why we are there.
“Bless the beasts and the children, for in this world they have no voice, they have no choice.” http://www.youtube.com/watch?v=dnNlU433i4Q
What is missing is independent verification that the current cut scores and standardized tests weed out or hold back students who are not ready for work and adult life and keep them in school until they are ready. There is no proof that the Common Core will produce “Career and College Readiness”. Here in Florida, the state added harder standards and raised the cut score on the FCAT Writes 2012. When 2/3 of the students failed the test, the state said, essentially, “My bad.” and lowered the cut score. Something similar may happen in this case. Teachers are concerned that students make real progress; politicians prefer illusory progress that they can use to win votes in the next election cycle.
These exams are a joke. A waste of money and valuable instruction time. America, or at least those in charge of education, had better wake up. We just keep falling behind.
They obviously want us to fall behind.
Yes, Cosmic Tinker. You and I see things the same way. Both parties are flip sides of the same coin. The illusion of choice. Two choices, and either way they win! You can choose 27 types of toothpaste though, so enjoy your “freedom” to buy things. I will take it a step further. After they destroy public education, they will run charters for awhile, but ultimately education will only be for those who can afford it (money). They don’t want to be in the education business, especially not for the poor (no money). They just want to kill public education, and they can’t just come out and say that (not yet), so we get these games and pretend reforms. We are going back to where perhaps 20% will have an education of some kind, and other 80% will be illiterate. History repeats itself. This is kind of like a new technological Dark Ages. Yes, we have figured out the whole game, but so what. Power will do what it wants to do. We are as powerless as serfs in the Middle Ages. We are just scribbling on an alley wall in a big city full of entertainment. As my professor from Harvard always told me, “It doesn’t matter what you think; it only matters what those in power think!” I suggest escapism through reading. You are always free in your mind. I like gardening and bird watching…ha ha.
Cosmic tinker & John- there are many examples of huge corporations doing something incredibly stupid being called on it through social media and losing hundreds, thousands and perhaps millions of customers within a short period of time. Amazon has very foolishly decided to house the students’ data on their systems. If every teacher boycotted Amazon for a year, what would be the result on Amazon’s bottom line?? The powerful only respond when it impacts their pockets. Who do they think buy books?? It’s teachers & principals!! Join me in boycotting Amazon and we’ll make a difference.
Marge
Yes, John, I also wonder if this is an insidious plan to ultimately do away with education for the masses entirely. I don’t know if they are planning to overturn child labor laws, too, a la New Gingrich, or what though, because without compulsory education, there are probably going to be a hell of a lot of unsupervised low income children on the streets every day.
Yes, I agree, Marge. I had already decided to boycott Amazon. I now have an incredibly long list of companies that I’ve been boycotting.
Dear Cosmic Tinker — Please share your list— I have just added Toys R Us ; Walmart & Sam’s Club to mine.
Thank you!!
Marge
Marge, Those are on my list, too. Here’s a long list of products made by the Koch Brothers that I won’t buy: http://kochwatch.org/?q=node/28
Cosmic Tinker — Thank you so much for the list of Koch Georgia Pacific Products. I will also add them to my list of products to boycott. I began my boycott of Amazon. Reminiscent of the movie You’ve Got Mail, I went to a darling Bookstore in Brockport , New York called Liftbridge. I will be buying my books from them from now on. It feels great to support small businesses!!
Marge
Marge, you have a great idea!
How great it would be to call for a week of boycotting Amazon?
Microsoft’s Windows 8?
And Walmart and Sam’s Club?
Carrie, It will take much more than a week of boycotting to make a difference. It requires a long-range commitment and spreading the word amongst other consumers.
I’ve been boycotting Walmart for about a decade, due to their poor labor practices. Too many of my colleagues did not get on board, because they didn’t recognize the threat to their own unions and jobs. Now that it’s so well known how anti-union, anti-public education and right-wing the Waltons are, they’re starting to realize and boycott, too.
Let’s add WalMart and Microsoft to that list.
We need an app to filter out the corporate “reformers’ when doing any online shopping. It should give a little alert when making a purchase from whole bunch.
Disruptive technology.
Oh, my! The tests for 2nd graders have been already very hard. This year as I’ve been preparing my students to pass the STAR tests I’ve been saying that the makers of the test are tricksters, so that it’s up to them to see the tests as if they’re on a treasure hunt to find the best answer.
As we have been going over the practice tests together I ask them to first find the “dumbest answer” and toss it out right away. Next find the two which are most alike, and as a detective would zero in on the one that makes the most sense. I say it’s you against the test makers and don’t let them get the better of you by not caring, instead care, and beat them at their game.
It’s so nice you have made it like a game, but what nonsense this is for the children.
Fake failure = marketing op for privatizers to scam taxpayers by offering fake improvement. This is unethical marketing via legislation, not reform based on science. There is nothing honest about it.
Follow the money. Cui bono? To control others? Ignorance? Meanness? What motivates these people with insatiable egos? It’s upside down. It’s also the business model that has nowhere in education.
.So that our children “fail” meaning the educators in CPS have failed, justifying the chaos of school closings, the need for charters. It is to justify corporate tax brakes to the wealthy and politically well connected. To ensure todays disenfranchised youth continue to provide Walmart with a surplus of umder educated, unskilled, minimum wage labor.
I’ve heard the 30% more failures line several times recently. If that’s the case, will we also have a 30% increase in AIS teachers to deal with all the failures? I think not !!! We don’t have enough extra help teachers this year. What a contrived plot to make us fail…..Why? To break the unions? To make money by privatizing?