Here is the reason for the collapse of test scores in New York City and New York State.
State officials decided that New York test scores should be aligned with the achievement levels of the federally-administered National Assessment of Educational Progress (NAEP).
This is an excerpt from a press release prepared by Mayor Bloomberg’s office:
“The new State test results are in line with previous results for student’s readiness for college and careers and show New York City students have maintained gains made over the past decade. The percentage of New York City students meeting the new, higher bar for proficiency in math (29.6 percent) is similar to the percent of students measured proficient on the 2011 National Assessment for Educational Progress (NAEP) tests (28.0 percent) – up from 20.5 percent on the NAEP test in 2003. The percentage of New York City students meeting the new, higher bar for proficiency in English (26.4 percent) is similar to the results on the most recent NAEP English test (26.5 percent), up from 22.0 percent on the NAEP test in 2003.”
Now, leave aside for the moment the odd fact that the mayor is boasting about the appallingly low percentage of students in New York City who met the new proficiency standards after a decade of his control of the schools. The key point here is that the mayor, his Chancellor Dennis Walcott, Regents’ Chancellor Merryl Tisch, and State Commissioner John King all agreed that the state and city scores should be comparable to the NAEP “proficiency” level.
That is a huge mistake. It explains why the scores are invalid.
The state didn’t just “raise the bar.” It aligned its passing mark to a completely inappropriate model.
The state scores have four levels: level 4 is the highest, level 1 is the lowest. In the present scoring scheme, students who do not reach level 3 and 4 have “failed.”
NAEP has three levels: “Advanced” is the highest (only about 3-8% of students reach this level). “Proficient” is defined by the National Assessment Governing Board as “solid academic performance for each grade assessed. This is a very high level of academic achievement.”). “Basic” is “partial mastery” of the skills and knowledge needed at each grade tested.
“Proficient” on NAEP is what most people would consider to be the equivalent of an A. When I was a member of the NAEP governing board, we certainly considered proficient to be very high level achievement.
New York’s city and state officials have decided that NAEP’s “proficiency” level should be the passing mark.
They don’t understand that a student who is proficient on NAEP has attained “a very high level of academic achievement.”
Any state that expects all or most students to achieve an A on the state tests is setting most students up for failure.
If students need to reach “proficiency” just to pass, there will obviously be a very large number of students who “fail.”
B students and C students will fail.
The NAEP achievement levels have always been controversial. Many researchers and scholarly bodies have said they were unreasonably high and thus “fundamentally flawed.” That term “fundamentally flawed” occurs again and again in the literature of NAEP critics. This article by James Harvey is a good summary of these arguments.
Some on this blog have asked whether NAEP is a criterion-referenced test, and the answer is no. A criterion-referenced test is one that almost everyone can pass if they master the requisite skills. A test to get a drivers’ license is a criterion-referenced test. Anyone who studies the laws can pass the written test and qualify for a drivers’ license.
NAEP is not a criterion-referenced test. Massachusetts is the only state where as much as 50% of the students (and only in fourth grade) are rated proficient in reading. The NAEP tests are not designed to be criterion-referenced tests; they are a mix of questions that are easy, moderate, and difficult.
The achievement levels were created when Checker Finn was chair of NAGB. I think they are defensible if people understand that the achievement levels do not represent grade levels. If the public wants a measure of “grade level,” then “basic” probably comes closest to grade level. “Proficient” is not grade level; as NAGB documents state, it represents “a very high level of academic achievement.”
More important, the NAEP achievement levels were never intended to be measures of grade level, and New York officials are wrong to interpret them as such, especially when they mistakenly use “proficient” as the passing mark.
Any state that uses NAEP “proficient” as its definition of “grade level” is making a huge mistake; it will set the bar unreasonably high and will mislabel many students and misjudge the quality of many schools.
And that is exactly what happened in the New York testing fiasco.
If the state sticks to its present course of using NAEP “proficient” as its passing mark, it will encourage criticism of the Common Core standards as unrealistic and stoke parental outrage about Common Core testing.
People know their children, and they know their own school. The politicians may convince them that American education is floundering (even if it is not), but they can’t convince them that their own child and their own school are “failing” when parents know from their own experience that it is not true.
The corporate reformers now using the Shock Doctrine to bash the schools and disparage students may find that their tactic has backfired. They succeed only in adding fuel to the growing movement to stop the misuse of standardized testing.
What is happening in New York is likely to undermine public confidence in the state’s highest education officials and create new converts to the Opt-Out of Testing movement.
The Shock Doctrine may be a boomerang that helps to bring down the madness of No Child Left Behind, Race to the Top, Common Core, the Pearson empire, and every other part of the reformy enterprise.
New York may have inadvertently created by the most powerful recruiting tool for the Opt Out movement.
Diane Ravitch's blog 
State Opt Out/Refusal Guides found HERE.
http://unitedoptout.com/opt-outrefusal-guides-for-each-state/
OMG. My jaw dropped as I read Harvey’s excerpt. The NAEP is invalid! In shorthand, that means the scores are meaningless & we should not use them to make any assumptions of learning (or no learning)
If schools used invalid assessments to place kids in Special Education they could be sued- and rightly so.
jcgrim,
“The NAEP is invalid! In shorthand, that means the scores are meaningless & we should not use them to make any assumptions of learning (or no learning)”
Ding ding ding ding ding ding ding ding! Come on down and collect that prize.
Rocinante still stands strong in my quest!
I challenge all to read and understand what Noel Wilson has elucidated in his 1997 dissertation “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
You’re using words that are too big. The people who are educated enough to understand what you’re saying have already done enough research and/or possess a good understanding of history and economics and already understand this new form of government-mandated child abuse is not okay. Those who don’t are the same ones who are still voting for the traitors in office who are intentionally dumbing down America and teaching them concepts that will ultimately prove our downfall.
Interested in joining the NY fight against high-stakes testing? New York State Allies for Public Education was formed recently by parents to unite opt out groups from every corner of the Empire State. As parents we cannot allow another year of our children’s education to be harmed. Visit NYSAPE.org for more information.
I also read on twitter some parents are considering sending the individual results back to King, return to sender, marked: I N V A L I D .
That is a great idea. It would overwhelm the post office but would be a very powerful and visible gesture.
It has been suggested by Kris Nielsen that parents from across the state send test results back, yes. He has listed the address on the various Opt Out Facebook pages.
Send copies to your state Senators and Assembly members. The legislature elects the regents, who appoint the commissioner, and they’re all supposed to be working for us. Remind them often.
Mail it Certified Mail w/ Return Receipt.
Makes an impact.
I like the idea of sending this stuff back labeled
JUNK (science) MAIL
Oh, I think they understand. I think they understand VERY well.
Indeed, yes they do!
I have to give credit to ETS for staff development in Massachusetts as MCAS was being conceived; here is only one example of what ETS taught us about testing. Using the local curriculum, and the standardized test selected by the board in the local district, the annual tests were reviewed . This quote is from data in the Public Schools of Scituate MA: Selecting Scituate Standards (led by Guido Risi Principal and the superintendent at that time)
QUOTE:
PURPOSE: “first and foremost, this standard-setting study is being conducted primarily for the purpose of program planning and not solely for the purpose of categorizing or slotting students in the Scituate Public Schools.”
Scituate faculty in their schools looked at suggested/possible standards Level 1 standard: would identify those students who are non-masters and require a follow-up program in skills development (for example reading comprehension). a Level 2 Standard that identifies students who would benefit from but do not require a program developed for the Level 1 group (sometimes in reading called a “corrective program” although this may be arcane language by 2013 expressions) and a Level 3 Standard that would identify students who would benefit from some additional help through a developmental program (i.e., text structures; inferential comprehension, etc.) This multiple standard approach resulted in the following (for that cohort of students in that year and not generalized to any other groups). Level 1 4% Level 2 7% and Level 3 3% ….this is a lot different from saying “your students have failed.”
NY has gone about this with the PURPOSE being overgeneralized ; instead of program planning it is “devastate the students” and they have produced flawed data. This is unconscionable, irresponsible, immoral and unethical. Parents need to get better information from their local school boards.
ETS provided valued staff development with up to 35 districts before any MCAS tests were ever administered. and ETS trained teachers in holistic scoring so that the understandings would be clear on reading student essays . One summer we did oversee reading of 10,000 student essays and this project reading essays indicated that there was a teacher effect along with the curriculum effect (manual thesaurus, computer, etc. plus a good teacher)
Could Jean Sanders please tell us the source of her information on the state’s use of ETS for staff development in Massachusetts before MCAS tests were administered. I was on the original assessment development team for grade 4 from 1996 on and a member of the department from 1999-2003, and I am unaware of any work by ETS staff relating to MCAS. Sandra Stotsky
it was earlier — when MEAP was being developed. ETS produced a report that is in ERIC; goes back before 1996 — when Greg Anrig Left MA he went to work with ETS…. Perhaps they realized what a good job he had done in MA??? (I can’t verify the latter but I can verify that MEAP was introduced and schools were building standards and choosing tests and learning to do analytic scoring of essays. ETS was our leading presenter especially on setting standards. I have forgotten the names of the two people from ETS office in MA.
found the study 1974.
Manual for Project Management. MEC-ETS Standard Setting Studies Conducted as a Part of MEC’s Improving Basic Skills Project.
Educational Testing Service, Wellesley Hills, MA.
This document is designed to assist school districts to implement the Massachusetts State Board of Education regulations requiring the setting of performance standards in basic skills at three grade levels. The appropriate procedures are described in the manual: (1) setting of basic skills objectives, at three grade levels; (2) the selection of tests or other measures that adequately cover the objectives; (3) the setting of a minimum acceptable score on each test or measure; and (4) the reporting of the number and characteristics (sex, race, etc.) of those students achieving below the local standard. Information is appended on tests currently available for assessing basic skills development at the high school level and general achievement test series, as well as on holistic scoring. (Author/George elford)
This second edition of the project managers’ manual for the MEC-ETS standard setting study includes-several changes based on the review of the earlier draft by Dr.Michaelieky(OS,Princeton).
April, 1974
Jean,
That standard-setting process was not used for either NAEP or Common Core. The CCCS tests adopted the NAEP levels, which are inappropriately applied. I explained why in my book “Reign of Error.”
Thanks, I know it got lost along the way. This work was very, very old. It was a small pilot “study” in MA with George Elford of ETS — if I recollect Dean Childress was instrumental in discussing the goals for the state and he worked at BU. Anrig was commissioner and shortly after that he went to work at ETS. This work was done prior to MEAP — the first iteration of the MA test… Along the way, things got derailed. The CA Educational Commissioner and the NY Educational Commissioner were trying to find ways they could “work together” …. and what started out as local curriculum and administration and teachers working together to identify local standards became a huge corporate undertaking. We then were left with the credo of Coleman/Colman and the MCAS– it was not the intention of the policy and practice at the time the manual was written. I know because I worked with George Alford in the ETS office and we had about 15 towns that each did their own local standard setting process. MA Department of Ed was not involved directly with this ETS project so they probably didn’t see the usefulness. At my last visit to Albany NY I went to the archives of the Government and the Department of Ed and had the librarians dig up all the records and the documents that were clear that things were forming: A Lab/Center was formed for New England and we lost all the local input that was originally intended — so instead of “grass roots” with actual practitioners who worked daily with students, the corporate world took it over. I was simply responding (belatedly) to Sandra Stotsky who was asking if I could verify the words of the principal Guido from Hatherly School in Scituate. And, most of the curriculum people and principals are now deceased or are certainly retired and yet we knew … There was a lot of political upheaval in the state. Anrig was gone; governors switched, we ended up with Cellucci , and then eventually David Driscoll. The early history was from the descriptions of MEAP and the process to get to that point– the first iteration of the MA test. Just wanted people to know some of the background in MA. And I don’t fault Anrig or the NY Commissioner for the “mess” we ended up with. (p.s. the New England/New York “Lab & center” project was called NEREX… Later it evolved and now most everything for that domain is at Brown. (Annenberg at Brown turns out some good things; I am hesitant to recommend any of the other studies at Brown especially the one on “grit” in our students and the recent one where they say there is no virus transmission in schools of the lethal virus. ) If there is anything that is incorrect I hope people will tell me because it is important to keep a record of the happenings … and get it on the hard drive whenever possible.
ERIC ED 183 619
Title : Manual for Project Management. MEC-ETS…. things got much more “political” in the state ; I left that job about 1992 and went to adjunct faculty at Northeastern U. and U MA Lowell. When there were funds for a “Lab & Center” it was extremely competitive and the corporate world kept moving in closer and closer to the money…. I do not fault ETS or Greg Anrig for this. BU was strong at that time but Silber took a lot of bad press and lawsuits. Wish I could write the whole story but that is above my pay grade; I feel like the nurses who are now overwhelmed in the hospitals and “working at the top of their license” as our HHS people like to brag about very smugly.
cx spelling. Zieky, M. 1. (1982). Passing scores: A manual for setting standards of performance on educational and occupational tests. Princeton, NJ:
Educational Testing Service. George Alford. date 1979 (not 1974)
as you can see by the ERIC ED record, all of these developments occurred before the tenure of Ms. Stotsky. I was impressed when she worked at the state department because I knew of the work of the Calabasas Center for Civic Education and how much Ted Kennedy supported their work. These arrangements and collaborative efforts were in the very early stages of the state developing standards. https://files.eric.ed.gov/fulltext/ED225569.pdf
These are the archives I visited in Albany NY to set the record straight for myself…. NEREX was formed with New England and NY … At some point the universities seem to be left behind or cut out……. and we ended up with Pearson……… if anyone has a differing opinion I would love to hear it… Gordon Ambach I believed was an outstanding leader… The commissioners met to form NEREX and gave their blessing but then they were more in the background… Other political forces took over.
http://www.archives.nysed.gov/research/edpolicy/research/res_conference_bios_ambach.shtml
Diane, any plans to visit the NY State legislature to testify on this? I suppose there would have to be an opportunity for discussion, perhaps when an education-related bill is on the table. Is there a public TV or televised public access discussion group where this can be revealed? How about another article in theDaily News?
Given that the charter schools in NYC saw a higher percentage drop in their success rates as a result of the change to the new tests, there are several conclusions to be drawn
1. By the standard the “reformers” like Bloomberg want to use, they were never all that successful, because they were only achieving “basic” level
2. there seems to be clear indication that their students were being geared to pass (perhaps barely) the tests that were being used.
The second point is important. In theory they could regear to the new tests to improve their performance – on the tests.
What I am still waiting to see are the longitudinal studies which track students from charters through post secondary education, starting with the percentage that have to take remedial courses because they are not prepared to do college-level work.
I think the best longitudinal data comes from the US Census: http://www.census.gov/newsroom/releases/archives/education/cb12-33.html
Why aren’t politicians and policy-makers trumpeting this statistic? It seems like all of our harping on higher standards is starting to make an impact.
30% of Americans over the age of 25 hold at least a Bachelor’s degree. This is amazing! Let’s throw a party!
I suppose that narrative wouldn’t support neoliberal efforts to privatize the $650-B annual public education budget.
Congratulations to NYSAPE.org –
A similar coalition exists in Texas – tamsatx.org. TAMSA parents are doing the homework on Pearson’s billion dollar testing fiasco and legislators are listening.
TAMSA leaders make presentations across the state at regional service centers, local businesses, chambers, public schools, etc. The coalition keeps parents informed through social media. As a result, the balance of power is shifting away from Pearson’s lobbyists.
http://www.tamsatx.org
Click to access 2013-01-13-tamsa_overview.pdf
The majority of people outside of the education world do not understand the difference between criterion-referenced and norm-referenced tests. Both are necessary and appropriate in the correct time and place. But, to set the bar with a norm-referenced test is wrong. We are not making widgets here.
For those of you who are getting frustrated with current issues in education, you might want to read “Real Talk for Real Teachers” by Rafe Esquith. This man speaks the truth.
Amazing how convinced Bill Gates is and how he bought EVERYBODY to spin this giant web. He now has higher education in his cross-hairs by spouting that degrees are not necessary and educational experts are to be ignored…it is only about skills. Oh yeah? How’s it working for NYC now?
They keep stomping through giant vants of tests, cut-off scores, stats, GE, NAEP, CCSS….stomping grapes without yielding a drinkable wine. Sour Batch to be thrown away!
Just because the guy is filthy rich does not mean he is educated or cultured. But, the greedy will sell their 1st born to grab a couple of bucks as Gates creates chaos. All in the name of doing good with his Billion$. Test your own values: would you follow him if he were NOT RICH? College drop-out, disrespects scholars, teachers, leaders of countries, his own employees, college professors, children, parents….
Waiting for the next spin interpretation of the NYC test results. Reality TV? As the Deformers churn!!
Personally, I think that Gates wants to feel less self conscious that he didn’t get a degree, so he denigrates those with degrees any chance he gets–including teachers.
I don’t know if the Opt-Out movement has a logo yet, but would suggest O2 (unable to write a subscripted 2), the formula for the diatomic oxygen molecule. Like O2, Opt-Out may be necessary “breath” for public schools to survive.
•Superscript: Press CTRL+SHIFT+=
•Subscript: Press CTRL+=
That’s from my Word 2003.
Diane I agree that the state license exam is criterion referenced, I was pointing out that there are reading and math tests that are criterion referenced rather than the traditional Bell Curve, which is manipulated. Your argument about the proficiency levels is strong. This needs to change. Perhaps we can use an international comparison. One idea that has not been expressed is that after 10 years of NCLB, the children’s scores are actually plummeting and they have been systematically dumbed down.
“New York may have inadvertently created by the most powerful recruiting tool for the Opt Out movement.”
Excellent point, and I am glad to read it.
The scores HAD to be invalid. The scores HAD to be devised ahead of time. Without manufactured failure, Pearson (the company lining the politicians’ pockets) couldn’t sell the cure! NYS parents do NOT fall for this and spread the word!
If NAEP is the “standard”, why create brand new tests? Why not just use the NAEP tests?
This needs to be shared widely with parents. If this isn’t sufficient cause for outrage, I don’t know what is.
Michael Lambert, the short answer is: If students don’t get an A, they fail. That’s the New York way.
Just so I understand clearly — a “3” on the NYS tests equals a “3” on the NAEP tests? Or does a NAEP 2 equal an NYS 3?
The metric currently is that 3/4 on NYS = Proficient/Advanced respectively (since proficient is the lowest possible “passing” mark and 3 is the lowest acceptable passing mark it can’t be otherwise). All students need to be 3s and 4s.
1s and 2s are lumped into Basic (failing)
Level 2s are considered “partially proficient,” and they were the biggest scoring group in New York State (37.1%). I’m not sure why everyone is saying that being partially proficient is “failing”: it kind of seems like a “reformy” tactic to me. Especially following a year where kids in NYS were taught God-knows-what.
Right, I know what the NYS scores mean — I teach English 6-8 in NYS. I’m confused how these numbers correspond to the other one.
“Now, leave aside for the moment the odd fact that the mayor is boasting about the appallingly low percentage of students in New York City who met the new proficiency standards after a decade of his control of the schools.”
Which is it, Diane? Are the schools fine and is this just a manufactured crisis, or are the schools not fine, with appallingly low proficiency rates after 11 years of mayoral control?
And the excerpt from the press release doesn’t conclusively say that the state test’s proficiency levels were aligned with NAEP’s; it’s merely offering them as a comparison. The cut scores were determined by a panel of 100 state educators. (I highly encourage everyone to read this first-hand account, which makes absolutely no mention of NAEP: http://theline.edublogs.org/2013/08/08/new-york-state-cut-scores-from-the-inside/ ) Commissioner King, of course, set the cut scores exactly where the panel recommended.
Perhaps there is a smoking gun that shows that the NYSED cut scores were benchmarked to NAEP, but I don’t think this press release is it.
A partial mea culpa: this description of the cut-score setting process does mention that panel referred to NAEP, SAT, and college preparedness rates, although it does not go so far as to say the state cut scores were actually aligned with NAEP.
Diane,
Wasn’t NAEP what was used to form “A Nation at Risk” that kicked off this whole fiasco about 30 years ago?
Was it valid to draw that conclusion? Why or why not? I genuinely do not know the nuances of that report as it pertains to NAEP tests.
As that report is one of the major root causes of what we’re seeing today, I’d be most interested to find out that the bar has been too high for years and that’s why our schools are perpetually in crisis mode.
No, NAEP was not used as the basis for “Nation at Risk.” It relied mainly on international comparisons.
In 1983, when the report appeared, NAEP was still in its infancy. The first NAEP tests were given in the early 1970s.
There were no state by state comparisons until 1992.
If you recall, Nation at Risk said the sky was falling and that we were at risk of losing “our very identity as a nation…”
Thirty years later, we have not lost our economy, our military pre-eminence, or our identity.
I deconstruct the international test scores in my new book, which will be out in mid-September.
on international test scores. Among those who have taken them apart were the late Gerald Bracey and Iris Rotberg. When I was taking a graduate course in educational policy with Rotberg, she brought in someone from WESTAT to talk with us about international tests (TIMMS – and this was in the fall of 2000 IIRC). Among the things we learned from that and from being allowed to poke around in a lot of data :
– some countries did not give the tests to non-native speakers of the national language
– some countries without a coastline dropped questions about coastal biology – I would think for a relevant comparison someone in KS or NE, which unlike MD with state standards including the Chesapeake Bay has a focus on such biology, perhaps should also not have been asked those questions to provide a fair comparison
– we were asking questions in math where we knew our students had not had the math because we wanted to see how they did
– students in some countries being tested (Norway?) had 2 years of physics, while a substantial portion of American students did not have physics, which is not required for graduation in many states.
Remember that last point. When I examined the data there was one question on which American kids blew away every other country. It was a physics question. It had to do with the stopping distance of car. Now, is what was being tested academic knowledge or personal experience?
Diane, looking forward to reading (?and reviewing? hint, hint) your forthcoming book.
Peace.
“Thirty years later, we have not lost our economy, our military pre-eminence, or our identity.”
Is that supposedly “military pre-eminence” supposed to be a good thing?
Personally, I see it as a societal sickness.
Why doesn’t this blog have a “Like” button?
KenS,
Thanks for the “like”. I sometimes feel quite alone in expressing that thought..
Duane
This is a PR nightmare for NYC & NYS school officials. They know now parents will begin opting their kids out of testing.
THAT’s when they madness will end. When there is no more PROFIT to be made on our kids.
This is for Kenneth Bernstein, above – this one on PISA. The entire testing system is in shambles, and still the reformers use it to justify the destruction of public education:
Is Pisa Fundamentally Flawed?
http://www.tes.co.uk/article.aspx?storycode=6344672#.Ufi3lQOu51U.twitter
I spent this past Monday at the Governor’s Conference on K-12 Education Reform listening to Virginia politicians, think-tankers, TFAers, and others of that ilk bemoan our students’ diminishing ability to compete with students of other industrialized nations. We saw many PowerPoint slides with NAEP, PISA, and TIMSS data “proving” that our schools are failing to prepare our children for life in the 21st century.
Here in Virginia, we’re preparing to dismantle our public schools and emulate, of all places, Louisiana. Our legislature last winter wrote a version of Louisiana’s Recovery School District (we call it Opportunity Educational Institution) and school A-F grading system into Virginia state law.
Disgusting.
“We saw many PowerPoint slides with NAEP, PISA, and TIMSS data “proving” that our schools are failing to prepare our children for life in the 21st century.”
Well, all you saw were a bunch of lies, prevarications and falsehoods all dandied and dollied up to appear to be “the truth” about American (sic) education.
Oh, no doubt, Mr. Swacker, no doubt.
The explanation of NAEP scoring makes sense. The numbers need to be redistributed. If a score of 3 was an A and 4 was for advanced students, then the 1s and 2s (the majority of the score) need to be broken down even further, so kids truly in need can be identified. Perhaps a 1 to 5 or even 1 to 6 scale could be used. It is unrealistic to expect everyone to be an A student, just as it is unrealistic for everyone to be able to pass 5 or more Regents exams. Children are not commodities and schools should not be run like a business. Education involves our flesh and blood and we want them to be treated with respect and dignity. This scoring system of an invalid test (with questions which are not in the realm of a normal learning curve) amounts to child abuse. It is causing stress for teachers, administrators, and most of all, our children.
It really does feel like an abusive situation. Kids will be told not to “worry” about the pain that their poor test scores are causing them. Meanwhile, they are completely justified in worrying about it. It’s like an abusive parent telling their child not to worry about the bruise they just got because they were just trying to make them a better person.
As a 6-8 math teacher in NYS, I appreciate the sentiment of the post and many comments. However, what I would like to see or find commentary on is this question, “What qualifications do the Governor, Mayor, Commissioner King, and others have to pinpoint precise benchmarks toward ‘college and career ready’ in 3rd – 8th graders”? I just don’t believe the glide path toward readiness is accurate. Each student’s path may be as varied and diverse as their own backgrounds and motivations.
Diane, I wish you were able to come to the rally at Comsewogue High School on Long Island this Saturday coming up!
Great article and discussion. I’m not quite clear, is Opt Out opposed to all testing or just poorly done testing?
Opt Out of tests that ave high stakes, punishments and rewards for students, teachers, principals
As a man who has lived a long life listening to women scream every ten minutes about ANYTHING in which they might perceive themselves at some disadvantage, I do not trust ANYTHING verbal about our schools that discusses only some uni-sex entity called “students” or “children”. It is simply impossible NOT to suspect nefarious intent in our women-dominated K-12 “education” industry, to suspect that much is being hidden about the only group in America today without a lobby – America’s boys. As a man of some experience and intelligence, I want to know how well “boys” are doing vis à vis “girls” in those schools; any American man would be a fool not to. There is a REASON why those schools now routinely ensure twice as many university degrees for women as for men, why gender has been censored out of the discussion, but all I ever see is self-serving propaganda.
Why Are American Men So Dumb? – http://invincibleprobity.wordpress.com/2011/03/28/why-are-american-men-so-dumb/
Ms. Ravich, I must respectfully disagree with your comments:
Particularly your suggestion that the “bar has been set unreasonably high”. That’s NOT what happened. There was no bar. The tests were set to be CONFUSING. Not high achieving. The tests were pure deception.
My daughter who regularly scores in the 99th percentile didn’t say the questions were difficult. She said they were ambiguous: e.g. more than one correct answer, neither being more right than the other. A question for which you could find all the same words under the same listing in a thesaurus. Do the math: If any of four answers is as right as every other, the percentage of students who pick the “right” one is 25%. That’s right where the kids scored.
The tests weren’t hard. The kids were set up not to be able to select one “correct” answer so that Pearson and Achieve and John King and Arne Duncan could come in and “rescue” them.
The distinction is HUGE and it needs to be highlighted in order to see what the agenda really is and get Pearson and Achieve out of our education system, not just making money with different questions.
“B students and C students will fail.”
Exactly… because the test is supposed to assess MASTERY of the standard. Not progress toward a goal, but reaching the goal. Questions then arise: Is that truly what should be measured with these tests? Do the questions actually measure the standard accurately and fairly? Are the testing processes fair and reasonable?
Have the problems of implementation been thoroughly examined?
In my opinion, while the Common Core Standards have some good features, the logistics of implementation and basic rationale underlying the assessment processes have gone horribly awry. The children will be the losers.
So only the winners of the Race to the Top get the laurels.
But whatever happened to “equality of educational opportunity,” equity, closing the achievement gaps?
Or was that just blather?
[…] later, when mind-boggling proof proved Ravitch proper. Then, the exams have been, ridiculously, aligned to the NAEP. Of course, the success were being twisted to indicate that UFT instructors ended up some sort of […]
[…] on, when frustrating proof proved Ravitch appropriate. Then, the assessments were, ridiculously, aligned to the NAEP. Of class, the outcomes had been twisted to indicate that UFT lecturers were some kind of […]