I first learned about the Common Core standards while attending a briefing for Congressional staff at a conference sponsored by the Aspen Institute in 2009. it was held at Wye Plantation, a lovely and isolated conference center in Maryland. Dane Linn of the National Governors
Association described the development work. I was invited to talk about the history of standards in the U.S.
In the discussion following Linn’s presentation, I recommended field
testing. In my experience in working on state standards in California, the standards needed to be vetted by experienced teachers. There needed to be a feedback process in which teachers used the standards and had the chance to tell someone in charge what worked and what didn’t, what was placed in the wrong grade (too hard or too easy), and which expectations were unrealistic.
In 2010, I was invited to the White House to meet with
three top officials–Melody Barnes, director of the Domestic Policy
Council, Rahm Emanuel, the President’s chief of staff, and Roberto
Rodriguez, the President’s education advisor. We talked for an hour
(Emanuel got bored and left early). They asked many questions, like what do you think of merit pay? i told them it had been tried repeatedly and failed every time. they responded that the Obama administration was putting $1 billion into merit pay.
Another of their questions was:
“What do you think of the Common Core standards?” My answer: “They
should have a trial in a few states for a few years before they are
made national standards. You need to find out what needs fixing.
They might be so rigorous that they increase achievement gaps and
hurt the kids who are not doing well now, especially poor kids and
kids of color.” I had a suggestion: “Why don’t you try the
standards out in three to five states, offer grants to those that
want to do it, and see how they work and what consequences they
have?” They were not interested. I left the meeting at the White House feeling that I had just spent an hour talking to people who heard nothing that I said. They told me what they planned to do, but they never engaged in dialogue about whether it was a good idea. Oh, and before Rahm Emanuel abruptly left the meeting to do more important things, his first question was “What can we learn from Catholic schools?”
For four years, I sat on the fence, waiting for evidence one way or the other about the Common Core standards. It really bothered me that no one cared to find out how they worked in real classrooms before imposing them. Then, earlier this year, I wrote a post
explaining that the way the standards were imposed, with
no trial, no feedback, no way to update them, made it impossible
for me to support them. Critics responded that standards need no evidence, but I don’t believe it. No big corporation would roll out a big product without field testing. Why should an entire nation accept education standards without finding out how they work?
Now we do have evidence. This is what
we know: the Common Core tests cause a huge decline in test scores.
Passing rates fell 30% in Kentucky and about the same in New York.
What is worse is that the achievement gaps grew larger. As
Carol Burris recently wrote, the test results were
especially devastating for black and Latino children. “The results
expanded the black/white achievement gap. In 2012, there was a
12-point black/white achievement gap between average third grade
English Language Arts scores, and a 14-point gap in eighth grade
ELA scores. This year, the respective gaps grew to 19 and 25
points. In 2012, there was an 8-point gap between black/white
third-grade math scores and a 13-point gap between eighth-grade
math scores. The respective gaps are now 14 and 18 points. The gap
expansion extended to other groups as well. The achievement gap
between White and Latino students in eighth-grade ELA grew from 3
points to 22 points. Students who already believe they are not as
academically successful as their more affluent peers, will further
internalize defeat. “The percentage of black students who scored
“below basic” in third-grade English Language Arts rose from 15.5
percent to 50 percent. In seventh-grade math, black students
labeled “below basic” jumped from 16.5 percent to a staggering 70
percent. Nearly one-third of all New York children scored “below
basic” across the grade level tests. Students often score “below
basic” because they guess or give up. Principals and teachers
cannot get accurate feedback on student learning. Although Ms.
Tisch [chancellor of the New York State Board of Regents] may say
that “this does not mean there’s no learning going on,” what will
parents think? Students will now need to be placed in remediation,
or Academic Intervention Services. Schools that serve a
predominately minority, poor student body will be fiscally
overwhelmed as they try to meet the needs of so many children.
Those who truly need the additional support will find that support
is watered-down.”
Maybe the standards are okay, but the tests are
not.
Who is in charge? By law, the U.S. Department of Education is not
allowed to interfere with curriculum or instruction. Who
can find out what went wrong? Or will we feel okay about imposing
reforms that widen the gaps? In New York, the charter schools did
no better than the public schools. Where are we heading? It won’t
do to keep saying, as Secretary Duncan likes to, that only
extremists oppose the standards. Reasonable people question them as
well. To whom should we turn for a careful, thoughtful analysis of
what is going terribly wrong?

Standardized testing does not create or enlarge the “achievement gap”, all it can do is reveal that the gap exists. If we changed exams and cut scores next year so that almost all students are rated proficient, it would not mean that the “achievement gap” had disappeared.
LikeLike
When you make the test harder and with little preparation you increase the gap…as long as there is an income, opportunity, resource gap there will always been an achievement gap. Do not pose a question. I will not answer.
LikeLike
My reply was a little bit longwinded, but it’s essentially what you said. Good call.
LikeLike
You do not change the size of the “achievement gap” by changing how you measure it, you only change your perception of the gap. Making it more difficult to achieve proficiency on an exam does not make the underlying achievement gap any larger just as making it easier to achieve proficiency on an exam does not make the underlying achievement gap any smaller. It is important to distinguish between the thing you are trying to measure and the thing itself.
LikeLike
TE, we eagerly await the day when your tenure, your salary, and your job depend on the rise or fall of your students’ standardized test scores.
LikeLike
I await the day that I am tenured, but after teaching for 20+ years off the tenure track I fear that ship has already sailed.
It does seem to me that this thread is concerned with the measurement of the achievement gap, not anything to do with teacher evaluation. Have I mistaken the thrust of the original post?
LikeLike
I am addressing that very issue, quite explicitly. You raised a very good question, TE, and I am attempting to answer it. That answer looks at one standard and its measurement and the rational reaction to that measurement and the consequences of that reaction for widening the achievement gap. Again, the devil, here, is in the details.
LikeLike
Please see my notes, below, which deal with one standard from Grade 8. I chose one pretty much at random as an illustration of how testing the standard and then attempting to do instruction on the standard, as written, in response to that testing would doubtless increase the achievement gap with regard to THAT standard. It’s not sufficient to talk about these matters in the abstract.
LikeLike
So, let’s imagine two groups of 8th-grade students, A and B. Both take a high-stakes test with a question on it that measures standard If standard CCSS-ELA-L.8.1a: “Explain the function of verbals (gerunds, participles, infinitives) in general and their function in particular sentences.” The question on the test asks them to explain the function of the gerund in this sentence:
Yolanda taught swimming to Waldo.
Both groups bomb. So, we create some explicit instruction about how gerunds and infinitives are both substantives and can function in various roles in sentences, just as other substantives can–as subjects, indirect objects, direct objects, predicate nominatives, etc.
Now, suppose that the students in Group A have grown up in home environments in which they are frequently exposed to complex SAE syntax and students in Group B haven’t. The students in Group A will respond to the gawdawful remedial instruction in explaining the functions of verbals, but the students in Group B, of course, won’t, because they need instruction of a very different kind first. They need to build the internal schemata for verbals that the students in Group A already have. The instruction created in response to the standard, as written, would, therefore, exacerbate and increase the already existing achievement gap with regard to that standard.
But really, the instruction that would be necessary in order for students to meet the standard–to be able to EXPLAIN THE FUNCTIONS OF verbals–should be very low on our list of priorities, if it appears on that list at all, for who the hell needs to be able to explain the functions of verbals? That’s a useful skill for people who want to be professional linguists when they grow up, I suppose, but what we really want is for kids to use verbals in their speech and writing and to do so correctly, for them to be able to comprehend verbals when they encounter them in speech and in their reading. And that’s a DIFFERENT set of skills than the skills described by the standard, and that set of skills has to be approached differently.
I’m sorry to have belabored this, but that sort of belaboring is exactly what is required if one is going to put together standards–lists of the outcomes that we are going to test for. That’s what WASN’T DONE when these standards [sic] were put together, and there will be lots and lots of very specific negative unintended consequences of that’s not having been done.
LikeLike
TE,
Your comments are interesting and reveal some issues with standardized testing that I did not consider before. I think once again people are getting hung up on the test results, instead of considering what the actual test results really show us.
LikeLike
It was somewhat infelicitous to say that the tests enlarge the achievement gap. Their immediate effect is, indeed, to change people’s ideas about that gap. However, those changed ideas in turn lead to the implementation of policies that do affect the achievement gap, and that’s why I used the qualifier “somewhat.” In particular, failure on the tests leads to test-oriented instruction that, in turn, clearly does widen the achievement gap, as our 10-year experience with NCLB has shown. What could be more idiotic than looking at the utter failure of having turned our schools, under NCLB, into test prep factories and saying, “Gee. We ought to do a lot more of that”?
LikeLike
Robert, thanks for excellent explanation via the “explain the gerund” example. You’re dead-on that such grammar-based language instruction is “gawdawful” even though it has dominated teaching of writing, no research that such scholastic patter increases student writing competence, but here it is again.
LikeLike
TE, You have no formal training or teaching experience in K12 education so please stop making declarations as if you are an expert just because you think you teach “13th grade.” How much standardized testing is imposed on students when they are adults in college? NONE (though that is likely to soon change in this zeitgeist).
LikeLike
You believe standardized tests do create achievement gaps, not just reflect differences in learning? If so, perhaps eliminating standardized tests will also eliminate achievement gap.
The highest stakes standardized tests for students at my university are probably the USMLE, though I think there are others outside of medicine as well.
LikeLike
Standardized tests reflect opportunity gaps. They are mirror images of socioeconomic status.
LikeLike
I agree with you that the achievement gap has a great deal to do with socioeconomic status. The common core exam did not change the socioeconomic status of any student, so I find it unlikely that the common core exam changed the achievement gap.
LikeLike
As with exams for teacher certification, CPAs, and lawyers, the USMLE is a professional exam, not a standardized test administered by colleges.
LikeLike
I think that these professional exams are standardized exams. What is your definition of standardized exams?
LikeLike
Those tests are mandated by legislation, required by professional organizations for entry into the profession and administered externally. They are not college tests.
LikeLike
Your initial point was “How much standardized testing is imposed on students when they are adults in college? NONE (though that is likely to soon change in this zeitgeist).”
You criticize my response by saying the examples I cite are “mandated by legislation, required by professional organizations for entry into the profession and administered externally.”
It seems to me that tests “mandated by legislation” are imposed on students in exactly the same way that common core tests are imposed on students. I do not see a difference here.
I also don’t know why external administration of a legislatively mandated exam is an important distinction. If the common core tests were administered by an independent agency would you change your opinions about them?
LikeLike
TE, as Carol Burris pointed out, we could close the gap by making the test so hard that only 1% passed it.
The point you must bear in mind is that the passing mark on these tests is subjectively determined. There is no scientific determination of what kids ought to know at a particular grade level. Those who make the decision decide in advance how many should pass and how many should fail. They could move the mark in either direction.
LikeLike
Or we could close the measured gap by making it so easy that 99% of students passed. Neather would accurately measure the underlying differences in academic achievement.
LikeLike
Standardized testing based on flawed standards can make people think that there are enormous gaps that are DUE TO inadequate “coverage” of the standards, which will in turn lead to more “coverage” of those standards–to standards-based pedagogy and curricula. If the standards themselves were poorly conceived, which these are, then one ends up misidentifying what the problems are and coming up with the wrong sorts of solutions for dealing with those problems. The tests are not simple litmus tests that show, clearly, the presence or absence of some X. The devil, here, is in the details, in what is being tested, precisely, and in what people do in response to those tests.
Let me give one example:
If standard L.8.1a says that students will be able to “Explain the function of verbals (gerunds, participles, infinitives) in general and their function in particular sentences,” and if students bomb on that standard, then people will create curricula and pedagogical strategies aimed at teaching students how to explain the functions of verbals. But that’s a TERRIBLE way to approach developing students’ command of use of verbals in their speech in writing and their ability to comprehend sentences that they encounter, in listening and in reading, that contain verbals. And if we approach this standards via explicit instruction in the grammar of verbals as traditionally conceived, we will have done nothing to compensate for the fact that kids grow up in different linguistic environments in which their internal devices for intuiting syntactic structures have different levels of exposure to complex syntax, and not addressing that but, instead, taking that explicit approach, will have an opportunity cost that will widen the achievement gap. The wording of the standard, the call for students to be able to “explain” the function of verbals has dramatic consequences and reflects a lack of understanding of how kids acquire the grammars of their languages.
LikeLike
When the tests are culturally and socio-economically biased, yes, they create an “achievement gap”. We could create an opposite “achievement gap” by focusing the tests on things that urban minority children have to learn just to stay alive, such as:
Which gang controls 63rd Street between Damen and Wood?
If you were attending Manierre and now have to attend Jenner, how many additional gang boundaries are you going to have to cross?
Which park has the highest murder rate in Chicago?
There, I just created an “achievement gap” showing how far behind their peers white students are.
LikeLike
I certainly think different exams can more or less accurately measure the achievement gap, but changing the exam does not impact the underlying gap.
LikeLike
Please see my notes about one standard, an 8th-grade Language standard, above, and how that standard will end up affecting the measured achievement gap. Again, the devil is in the details. One cannot talk about these matters simply in general and in the abstract. One has to look at actual, specific consequences of measurement of actual, specific standards, as written and what people do in response to that measurement and what consequences those responses have for the gap.
LikeLike
Touche, Dienne! Brilliant, in fact–a summary of everything anyone ever needed to know about tests.
And–BTW–bears repeating–these “standardized” tests are neither valid nor reliable, so are NOT, in fact, standardized. The tests are faulty, notwithstanding bias.
Remember the “Pineapple Question.” If my child were in public school today (she’s graduated college), NO WAY would I allow her to waste her time on such ridiculous tests.
EVERYONE needs to OPT OUT. No one takes the test–case closed!
LikeLike
It is actually much more important to distinguish between what is being measured and those who have the power to measure it.
Remember Campbell’s law:
“The more any quantitative social indicator (or even some qualitative indicator) is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.”
LikeLike
Consider the sentences “”I have never known Alex to make unfounded personal attacks” and “You shouldn’t have let these attacks get to you.” An EXPLANATION of the infinitives, there, might go something like this: In both cases, we have an infinitive clause acting as the object of a verb. In the latter case the infinitive particle–the to before the get–is suppressed. Evidently, some verbs like know take an infinitival complement in which the particle is explicit, and some verbs like let take an infinitival complement in which the particle is suppressed. Moreover, this has changed over the course of the history of the language. See Shakespeare’s “I saw her coral lips to move” in The Taming of the Shrew.
The 8th-grade standard calls for eighth-grade students to EXPLAIN how verbals function. Well, that’s what an EXPLANATION of those verbals, those infinitives, might begin to look like. But that’s not what we need from students–explanations of verbals. If we respond to 8th-grade students having done poorly on this standard by giving them explicit 1960s-style Warriner’s English Grammar lessons in the structures of sentences containing verbals, we won’t have gotten far in training kids to comprehend verbals when they encounter them and to use them in their speech and writing. And that sort of instruction will have varying consequences for students who grew up in environments with different levels and types of syntactic complexity. So, the testing of that standard, as written, will lead to teaching to the standard (that’s what it’s supposed to do, right?), and that teaching will have different consequences for different groups of students. It will hurt those on the wrong side of the gap more.
LikeLike
This is the kind of thing that I mean when I say that these standards [sic] are sloppy and weren’t vetted. And in standard [sic] after standard [sic], one encounters these sorts of problems.
LikeLike
Exactly right, explanation of how a grammatical rule works does not tell us about communicative competence of any student. It only tells us that a student has meemorized the rule and can explain its function in an exemplary sentence. This is not demonstrating communicative competence in a real-life exchange, a meaningful rhetorical situation, where rules are intuitively internalized through experience, exposure, practice, feedback, consequences, and the desire to get something done or to make something happen through language use.
LikeLike
Thanks, Ira. People (including the authors of these dumb standards [sic]) don’t understand that “instruction in grammar” means a number of very different things. If our goal is to increase the range of the syntactic complexity of students’ written and spoken production, then explicit instruction in traditional parsing is largely irrelevant. The so-called “language standards” in the CCSS in ELA are a joke. The writing standards are even worse. Both encourage the creation of terrible, counterproductive curricula and pedagogical approaches.
If we wanted to be scientific about increasing the range of syntax that kids use in speech and writing and that they can comprehend when listening and reading, then we would gather samples of their speech and writing and analyze these to get some sort of picture of what their internalized grammars look like (that would be a significant diagnostic undertaking of a kind that is never done, now), and these internalized grammars would differ a lot from student to student. And then we would put students into interactive situations with language that both had engaging content and that employed those constructions that they have not internalized. But we are really in the dark ages with regard to all this–we’re not scientific about it at all–and the CCSS in ELA are guaranteed to keep people doing a LOT of really counterproductive crap.
One of the things that we’re currently doing is taking kids who are suffering from being brought up in syntactically and lexically narrow environments and PURPOSEFULLY exposing them ONLY to speech and writing that is syntactically and lexically impoverished. These days, because of the CCSS, publishers are doing Lexiles on everything they produce, which almost guarantees that kids’ internal devices for intuiting syntactic, morphological, and semantic structures from their ambient linguistic environments won’t have the material to work on that they need to have for real development. But, again, all this comes from egregious lack of understanding on the part of the standards [sic] developers of what we know about how kids learn language.
Don Marquis ends one of his great Archy and Mehitabel poems with this line: “Come, my dear, both of our professions are being ruined by amateurs.” I think of that line when I think of the folks who cooked up these standards [sic].
And here’s another issue: Those non-SAE environments are not always exactly what one legitimately call syntactically and semantically impoverished. There are studies detailing that some are, egregiously so. But many are simply different. The kid has internalized a grammar and acquired a vocabulary that is not SAE.
But we’re supposed to take a one-size-fits-all approach because that’s what our lords and masters tell us to do. One ring to rule them all!
Idiots.
LikeLike
The word idiot derives from the Greek idiotes, meaning “a layman, one lacking professional skill” and, ultimately, from idios, meaning “one’s own.” The CCSS in ELA were put together by people who lacked professional skill in English Language Arts education and who were presumptuous enough to think that their own views, as uninformed as they were, should override those of every teacher, curriculum coordinator, and curriculum developer in the country. The presumption of that is truly breathtaking.
LikeLike
I was thinking what Ira Shor commented when I came to his comment. Also, the ELA standards are not developmentally appropriate for early childhood grades students.
LikeLike
The “achievement gap” isn’t actually a thing. It’s a construct that doesn’t exist apart from a test, comparisons among test takers, and demographic breakdowns. For my money, it’s a bogus construct, simply because “achievement” is ill defined in the first place. A score on a multiple choice test doesn’t represent achievement, and to the extent that the test may be culturally biased, the score is even less valid. Once you make the test more difficult and more culturally and developmentally inappropriate and raise the “cut scores,” yes the “gap” may appear bigger, but the measurement becomes even more suspect.
I heard a speaker at the 2012 Save Our Schools convention point out that it isn’t an “achievement gap” we’re dealing with. It’s a “preparation gap,” and that gap starts to grow as soon as a child is born disadvantaged. I agreed. Alternatively, you might call it an “experience gap.” I think it would be much wiser to measure the inputs into the child’s preparation or experience than to obsess over test scores, which are already predictable by socioeconomic level.
Use the money being misallocated toward VAM and other misguided efforts to, instead, study the experience deficits of unsuccessful students and figure out the direct measures needed to correct deficits in experience and preparation. This is the kind of measure we need to focus on–actions to take–rather than endlessly playing around with measurements and analyses that are easily manipulated for political purposes.
So, why not try to pin down the salient factors behind the experience gap and fund measures that would address those factors head on? Plenty of research has been done on this, but we’re still hung up on raising test scores–and other peripheral concerns such a “standards”–instead of dealing directly with the origins of the so-called gap.
Beware that some proposed actions such as academic rigor for preschool kids, are likely to be counterproductive. Maybe we can start by giving books to the kids, and parents, who don’t have them.
LikeLike
“The “achievement gap” isn’t actually a thing. It’s a construct that doesn’t exist apart from a test, comparisons among test takers, and demographic breakdowns.”
Yes, exactly! You said it a lot better, but that’s basically what I was trying to get across with my example of testing things that urban kids know. We can create whatever “gap” we want just by the test we impose.
LikeLike
Do you really think that there is no difference in educational attainment between the children of the wealthy and the children of the poor? Can we eliminate learning disabilities by simply finding a test that all students do equally well taking?
It seems to me that there are real differences in student learning that exist independent of any test or other metric.
LikeLike
I am certainly happy to call it a preparation gap or experience gap,but I am not sure that changing the name makes it any more of a thing or easier to define.
LikeLike
teachingeconomist:
You may have missed my point. I’m saying that for many reasons the methodology that comes up with something called the “achievement gap” is invalid. Furthermore, these metrics are easily manipulable and prone to being used for political ends. Policy based on a shaky construct subject to easy manipulation will focus on peripheral concerns (although there will be lots of money in it for a few people), not the underlying problem. That’s what’s happening now.
By contrast, the tangible inputs needed to prepare a child for school success CAN be identified and measured. Once those are known–and they already are, to a large extent–smart policy steps can be taken to see that more and more children enter school prepared. Instead of funding questionable testing and data analysis to support reforms that don’t actually work, use the money to directly address factors shown to make a difference.
Which makes more sense? Firing a veteran teacher because her students haven’t bridged a mythical achievement gap? Or establishing literacy coaching programs for poor expectant mothers? Or how about building on the unique knowledge that children bring with them to school rather that discounting it because it may not be on the test?
LikeLike
Let us not forget that these “intervention classes”take time from a child’s day, time she could be in art, music, technology, something at which the student enjoys and excels which makes the school day bearable. So now all joy and success is gone and only failure and disgrace are left. Think of what that does to a child.
LikeLike
This is apropos nothing but I just heard on NPR that Nokia is being bought by Microsoft. Anyone else see the irony?
LikeLike
Nokia is one of Finland’s biggest industries.
LikeLike
The CCSS are not as bad as the implementation of the tests. The grade level appropriateness of the objectives taught and tested are the places where the whole thing unravels and falls apart. I don’t know what they were smokin’ in NY to come up with the 1st grade curricula. Exposure to those ideas, fine. Expecting that level of articulated responses, absurd. Having the education of America’s youth in the hands of Pearson is very, very inappropriate.
LikeLike
So many typos in that post that I am reposting it with corrections. My apologies.
It is hardly surprising that those who champion top-down, totalitarian mandates should be uninterested in rational discussion or debate of those mandates. The new standards [sic] basically overrule every teacher, curriculum coordinator, and curriculum developer in the country with regard to what the desirable measurable outcomes of education in the English should be.
The ELA standards [sic] were, clearly, rushed together by a small group of amateurs and foisted on the country with no vetting.
Anyone with any depth of knowledge of what we now know about how kids learn in the various domains that these standards [sic] cover will be horrified by them.
It grieves me that the committee that put these standards [sic] together would undertake so heedlessly a task as consequential as the creation of a national instrument that would be linked to assessments and evaluations with extremely high stakes for kids, teachers, and schools and extremely significant consequences for pedagogy and curricula. One would have expected, given the gravity of their undertaking, that those who developed these standards [sic] would have approached their undertaking with the high seriousness with which, for example, people approach designing airplanes. When people design airplanes, they conduct failure modes and effects analyses to figure out, in breathtaking detail, exactly what might go wrong with their designs. The design of standards, like the design of airplanes, is a serious and highly technical undertaking, or should be, one requiring expertise. We actually know things about how kids acquire grammar and vocabulary, for example, and what we know has significant consequences for how standards in those domains, if we are to have such things, should be conceptualized. But nothing of what we now know about learning in the domains that these standards [sic] cover is reflected in them. Again, they appear to have been rushed together by amateurs. And then they were foisted on the country with no vetting whatsoever.
So, can one really expect authoritarian types . . .
who like top-down mandates,
who think that they know other people’s jobs better than they do,
who are willing to give the job of development to amateurs, and
who think that their ideas need no testing whatsoever . . .
to be interested in discussion and debate?
Imagine getting a meeting with Darth Vader to talk about the desirability of the Death Star or Joseph Stalin to discuss modifications in his collectivization policy. Imagine being the defense attorney for Kim Jong-il’s girlfriend. These people are not interested in having their notions questioned.
And mind-blowingly, at the same time that they are imposing this authoritarian standards-and-testing regime on the country and not allowing any debate or testing of their ideas, they are CLAIMING THAT THEY ARE NOT IMPOSING THEM, that the states got together, on their own, and cooked up this stuff. We’ve heard nothing but Orwellean Newspeak from the administration on the subject of the standards rollout.
2 + 2 = 5. Get used to it. Your overlords have spoken.
LikeLike
In NY and other states, we need an independent commission to evaluate the “proficiency” tests. The tests lack validity and parents need guidance in how to deal with these results. For example, what test items were aligned with the grade level and how did their child do?
LikeLike
How about giving the parents a chance to be tested on the FIRST GRADE objectives that NY has come up with for students? That way, they could see if THEY think it is appropriate. All parents of all socio-economic levels. Should be interesting.
LikeLike
Diane, Could you please elaborate on why Rahm Emmanuel asked you about what could be learned from Catholic Schools.
LikeLike
You cannot accurately assess the achievement gap using percent passing/proficient. Using percent passing/proficient almost always provides an inaccurate assessment of the gap, but almost everyone uses this incorrect metric. Assessing the gap is actually pretty complicated, but using scale scores and standard deviations is a pretty easy way to get a decent estimate of the gap. As, if you read something professing the achievement gap is X or Y and that assessment is based on percent passing/proficient, then ignore it. Daniel Koretz has a great book called measuring Up that discuess this. Many of Gerald Bracey’s books do as well.
LikeLike
“You cannot accurately assess the achievement gap using percent passing/proficient.”
What exactly is the “achievement gap” being discussed? Is it the standardized test score differences?
The fact is that this “achievement gap” itself is “vain and illusory”. When one starts with a falsehood one more likely than not ends up with a falsehood. And using a standardized test score to discuss the supposed “achievement gap” is the epitome of “vain and illusory” due to the myriad errors involved in the process of making of educational standards, standardized testing and the labeling/grading of students that renders said process completely invalid.
I challenge all here, and anyone else, to refute/rebut what Noel Wilson has proven of the invalidity of said processes in his “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 . How such insane and inane educational malpractices have come to be the core of teaching and learning discourse is mind boggling.
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
A good point. One sees a lot of this reporting of comparisons of raw % passing/proficient scores, both in sloppy education journalism and in the reporting engines of online products keyed to the Common Core. Thank you for making this point, Dr. Fuller. I do hope that people will read, as well, with understanding Duane’s post below.
LikeLike
Sorry, but the Common Core tests did NOT cause a huge decline in test scores. That statement is illogical.
Tests result in an outcome. They are NOT the causal factor you are trying to portray in this post. The Common Core tests have shown our nation that we have been UNDERSTATING, not OVERSTATING, the achievement gap. And now that a new baseline has been established, we need to be more effective at teaching our children the skills they need to college and career ready. When you dumb down learning, you get inflated grades and overconfidence, until you get to college or your first job and realize you knew nothing, and that your colleagues from Singapore or Finland or China are a bit better prepared then you are!
We should have high standards, and our entire system, teachers included, should be accountable for raising achievement and narrowing the gap. it is an atrocity at how wide the gap is, and the new standards and assessments have simply shown us that the emperor had no clothes.
I for one believe we can work together to do better. Lets stop complaining, passing the buck, making excuses, and collaborate to get this right. These standards were not developed out of thin air, and the reason why your suggestions were not implemented was that it would require at least 25 years to see progress. We need to see progress NOW, or at least in 5-10 years.
Don’t get me wrong. The standards will need periodic modifications – they are not perfect. There has been much debate about the 8th grade algebra issue. But in their totality, they are far better than past standards and misinformation and propaganda from the political self interest groups have muddied the waters. These are not curriculum, only standards.
The bigger concern is whether there is now a perception at the national level that says “If Ms. Ravitch recommends option A, choose option B.”
LikeLike
Off-topic, but I would suggest using lower-case for the “d” in your username to make it clear that “ED” is short for “education” and is not an initialism for a medical condition.
LikeLike
I’m laughing inside, FLERP. This is what happens when folks don’t have anything intelligent to say. comment on this blog, or any blog, the way you would speak to your mother, or your best friend, or your student!
LikeLike
Hmmmm….yes, some fall into that category…I wonder who? LOL
LikeLike
“At the national level” referring to whom? Educators? Politicians? Reformers? People who do or don’t know anything about child development or grade level appropriate objectives? People who do or do not blame children for their luck of the draw insofar as parents, environment, wealth? I think this post is unnecessarily accusatory and inappropriate.
LikeLike
Read and learn from the words of seasoned educators and our blog host, an educational historian, or work your way through the same old reformy cliches, lies and slogans from the eduwannabes jumping on the popular teacher bashing bandwagon…..your choice. Same mantra, different day.
LikeLike
Who’s blaming children for the luck of the draw, or lack of luck? Don’t put words in my mouth, Deb. There are plenty of social and emotional learning programs that are having plenty of success with students who didn’t win the socioeconomic lottery! That’s the typical response from a member of the NEA – you think that teachers are the only folks who know something about child development. So keep ignoring folks who have a different perspective Deb. But please don’t take your frustrations out on me, or anyone else who disagrees with the edicts coming down from the education historian holy mountain…….
LikeLike
Most of the REAL educators here would climb Diane’s mountain any day rather than your pile of $h___!
It’s her blog…not yours. Start your own…oh wait….my bad…you already did. Ha!
LikeLike
Well, excuse me, Mr Ed. You are the one who came in here all high and mighty. All I did was ask some questions. I also posed either or questions as to who might be representative of your national levels of broad brushed bashing of Diane Ravitch’s research.
Then you, not I, stooped to name calling and telling another person they had nothing worth saying. You question my knowledge and credentials, making assumptions about my involvement with the NEA. You know NOTHING about me, sir. Nothing.
I haven’t seen you contribute to this blog. And even in your ranting, you share no solutions (which you imply that you have). You make assertions that are not accurate or on point. If you know of anecdotal success stories, fine. Good for you. Share. But as it is you seem to be a defensive critic.
The bottom line here is the objection to public money being skimmed off into the pockets of for-profit LLCs that have no real understanding of the broad spectrum of learning styles that exist. No one said they were all failures. No one said that no one outside of educational circles has valid thoughts. What kind of a teacher would not recognize that there are many roads to the same end?
Generally, when someone comes storming into a blog with a big, puffed up defensive attitude wanting to disrupt the focus, that person is called a troll. In any case, the only thing I will say is that you are rude and unkind. There are a few others on here that are similar, but you take the prize. Have a pleasant day.
LikeLike
If the tests are based upon deeply flawed, misconceived standards, then the outcomes that they measure are not to be trusted.
This post very well illustrates the attitude of the test-kids-until-they-scream crowd: These decisions have been made for you. We shall let you know when the Politburo meets again to modify what you are instructed to do.
“Let’s collaborate to get this right.” That’s an interesting statement in the context of a post that basically says, there wasn’t time to get these right; we had to put these out now, and if you don’t like what we’ve done, too bad. We’re not interested in hearing what you have to say.
“They are not perfect.” Oh my. Now there’s an understatement, certainly when it comes to the CCSS in ELA. Right now, throughout the country, curricula are being developed, pedagogical approaches are being decided upon, and every English teachers is undergoing “training” in the CCSS. The consequences of what has been done in these standards will be enormous, and many of those consequences, I believe, will be negative. The CCSS in ELA were sloppily prepared. They are a mess. And people with more sense can’t do anything about this but accept what has been foisted upon us all.
LikeLike
I will say it again. The passing mark on any test is a subjective judgment by humans. The test makers know when they construct the test how hard or easy the questions are. They determine what % will pass or fail. Usually it is a bell curve. NY officials predicted the outcome because they chose it.
LikeLike
dianerav: you and Randal Hendee and Duane Swacker have all hit the nail on the head.
When it comes to high-stakes standardized tests, criterion-referenced tests—even more than norm-referenced ones—are a predone deal. Given the many decades of developing, producing, evaluating and administering standardized tests by extremely well trained and educated [if narrowly so] test designers, the results are predictable.
It’s a rigged game. It’s set up that way. If you don’t believe it—hey, Uri Geller gonna bend some spoons for you with the power of his mind!
Rheeally!
If contrary to all evidence you think otherwise, you may find yourself in the unlike predicament of needing Marx’s advice:
“A child of five would understand this. Send someone to fetch a child of five.” [Marx]
Groucho, that is.
🙂
LikeLike
I don’t believe these new tests are norm referenced. If I am wrong, sorry. But, I have never seen any data in all the years of teaching with these tests that they are because the tests change year to year and the age of the child doesn’t seem to come into play at any time, just the grade level in which the child is currently placed.
There is an expectation that there will be “x number of students” who are “advanced” etc. When you give the students the IOWA tests or the TerraNova and InView tests you get a whole different set of numbers about the child’s ability with relation to others of the same age.
Student performance on these tests tend so produce a bell curve, depending on how large your sample is. I don’t believe the tests we’ve given in Ohio for the OAT or OAA or OGT have been normed at all. I don’t know if the tests given by PARCC will be normed. If so, where do they obtain this information for comparison?
The attitude that these reformers who come from outside of the educational circle have any clue as to what children should know is based on absolutely nothing of importance, imo. My perspective on this has come from the elementary education viewpoint which I think needs to be considered first and foremost. The elementary goals need to be accomplished as foundational to the applications they will be using as they go through school. But, we can’t just cram 6-7th grade level objectives in the first grade and expect them to ever be appropriate because the students aren’t ready to comprehend them at that age.
I do think there is a greater possibility that the CCSS might be worked out with secondary education to then provide a more college ready group of students, if that is even what we need. But, with the undermining of the worth of college degrees, which many of us have spent our lives pay back loans to afford and which have nearly bankrupted us as parents are becoming “worthless” to corporations who simply don’t want to bother to train employees, even if the applicant is a fast, willing and adaptable learner. I don’t know why anyone would take the risk of putting their child in college today. I wouldn’t. I would tell them to gain a skill in a trade so that they could at least have an income.
LikeLike
You are right, Deb. PARCC and Smarter-Balanced are not norm referenced. They are criterion referenced. And when you reference lousy criteria, you have a lousy test.
LikeLike
Do you know whether ACT’s Aspire is norm or criterion referenced?
LikeLike
YES! The tests are faulty & have always been. I’m sure they’re going to stop allowing read-aloud (Math, Science) accommodations for special ed. kids, because those of us who have been able to see/read the tests have seen–year after year–how many questions are faulty—no correct answer, more than one right answer, etc. Oh–not to mention the test-prep materials, which ALL teachers get to see (because we have to use them for lessons). Questions/answers in these materials often make no sense, as well.
People–we need to stop having discussions about these tests and start having discussions about STOPPING these tests…period.
In the special ed post, the power of retired teachers was discussed–well, we should be working–locally–on encouraging parents to opt their kids out.
LikeLike
The “achievement gap” is a red herring. It’s symptom. The ed-reformers turned it into a problem. The blame? Schools.
LikeLike
Diane is correct. They chose the outcome which is not hard statistically to do. If you raise the bar only those who are at the bar can go over it. So it becomes more select and elite as the bar goes up. Now if you take into consideration the CORE Federal
Waiver schools in California and think about that three of the 7 have from 17-24% of their students not coming to school everyday and they are at the bottom of the performance chart what would happen to the scores if those students were taking the tests because they were in school. Well the test scores would naturally drop but we would have a better society with more educated and less on the streets getting into trouble and eventually into the criminal justice system at a real high cost. I prefer students in school to high test scores perverted by driving off the low performers as the system demands now by its rules and regulations and test systems. If a teacher gives weekly tests they surely know what is happening if they are doing their job. They certainly knew when I went to school.
They always try to play numbers games to confuse people and considering the low level of thought in this country that is not hard to do. After all, how else do we have this mess?
LikeLike
For all readers..there is no quick fix…see this excerpt and full article, re: Massachusetts
Also noteworthy was what the reforms did not include. Parents were not offered vouchers for private schools. The state did not close poorly performing schools, eliminate tenure for teachers or add merit pay. The reforms did allow for some charter schools, but not many.
Then the state, by and large, stayed the course.
LikeLike
I don’t see why the testing mania had to take off as it did. Formative testing and the assisting students reach goals makes so much sense. Testing them to punish and never even giving them a chance to see how to correct their errors is a terrible way to use these tests. Continuing to use these bogus scores to evaluate districts and teachers is basically absurd.
LikeLike
“I left the meeting at the White House feeling that I had just spent an hour talking to people who heard nothing that I said. ”
This is how I feel at every staff meeting lately. What is the motive behind throwing out the baby with the bathwater?
LikeLike
We have sat in meetings with the rapid-fire curriculum director who spun in and out goibg from grade level meeting to the next, with our moutgs gaping in disbelief. She moved on to a charter school in another state, but she left us with our heads spinning. The experienced teachers always felt slapped in rhe face and the younger ones were just listening and hoping it would make sense. But just as Diane said, it didn’t matter what we said. It was always something new.
LikeLike
Another thought: we have had the point made th at other professionals must pass tests to get their licenses or certification. Teachers take the Praxis exams. NTE. Orals and finals to get masters degrees.
What other profession is “rated” by tests taken by someone else? Maybe our curticulum person should be rated by the teachers’ students’ performances. And pass it upward.
LikeLike
Thanks to many, but especially to Robert Shepherd, for your insightful comments. I’ve often wondered what this particular “faith-based initiative”
LikeLike
(whoops–to continue)
at the national level was not the subject of a more rigorous and thoughtful national debate. I’ve an essay on this subject, paying particular attention to the teaching of reading, that can be found by googling “jonathan’s edutalk” and looking at the most recent post, “aliterate readers and complex texts.” I’m told it takes about 15 minutes to read, so if you have the time I’d appreciate any responses you might have.
LikeLike
Here is my take on the use (and misuse) of the word “rigor” in the Common Core:
The Draconian definition (harsh; excessive sternness; something hard to endure) is the one that most often captures the attitude of education reformers and politicians who call for tougher standards, increased testing, and more (home) work in order to create an impression of making school more demanding. Their misuse of the word rigor fails to include higher order thinking skills, problem solving or the address the significance of imagination.
They misuse the word “rigor” the way the word “Inconceivable!” is misused in the film “The Princess Bride”.
If education reformers continue to misuse the word rigor to describe what is lacking in curriculum at every grade level, I propose teachers stop and correct the reformers, with or without Inigo Montoya’s accent, and say, “You keep using that word …I do not think you know the meaning of that word!”
FULL POST: http://usedbooksinclass.com/2013/09/03/inconceivable-rigor/
LikeLike
Hear! Hear! I never thought I’d see one of my all time favorite films put to such good use! I came to a similar perception during my years as a graduate student in English at that university just about 40 miles due south from you. My account (including “rigor” in appropriate quotes) can be found by visiting my “aliterate students and complex texts” blog entry at my “jonathan’s edutalk” blogsite.
LikeLike
As ever, a stimulating discussion of the latest topic. And, as always, Diane frames the question in clear, direct and commonsense language. Having been involved in the state education system for only 15 years, I cannot address the fine points you have all raised. However, from a policy point of view, I would make a couple of points:
1. Taking a testing system that is over a century old, that was always pointed towards rating and ranking kids and determining their status, educational opportunities and eventual economic and social success and turning it into a year-by-year measuring stick needed to involve parents and students. It has not. It has been done by the rarified political and educational elite in a near vacuum. The only public involvement has been announcements from on high to expect lower and lower scores. Parents and students see this as either a validation of their failure or a complete lack of validity of the system.
2. Just because a test is developed as a criterion-referenced exam, doesn’t mean it isn’t scored on a normative basis. That’s what post-equating is about. At the end of the day, the internal discussion and cut-score determination is based on what percentage of kids will pass or fail. Bank on that.
3. Education has become testing.
4. The 1% are quite simply trying to justify how well-deserving they and their children are in private, charter or religious schools; compared to how low-achieving everyone in public schools are. All the while proclaiming that this is about helping those poor wretches reach their level of “achievement.”
5. This is why I love Diane. She doesn’t fall into the pedantic discussions of educators about all the high-falutin’ “pedagogical” ramifications of a policy (please don’t take offense, but I was an economics major and I saw the same thing there!). She just lays it right out, Boom, in everyone’s faces in language and context they can understand. THAT is why the powers-that-be are so clearly frightened of her.
6. If these are truly criterion-referenced exams that get equated to past administrations, we will not see a significant increase in the percentage of students achieving proficiency in future administrations. It would take a HUGE increase in broad-based performance to move the needle, as any increase in achievement would have to reflect REAL learning gains, by definition. Given the skewed distribution of educational resources in our society, that is just not likely to happen. Particularly when we have wasted so much of our new educational spending on all this “reform”, while schools are closing and teachers are being laid off across the country. Talk about cognitive dissonance…
7. I’m afraid all we have gained from the bi-partisan NCLB and RTTT is that Washington thinks they know better and have proven they don’t. Do you remember next year was when all children were going to be proficient? Now, it is going to be very few children are proficient. And that’s only one turn of the political cycle.
8. Finally, if New York State, one of the leading educational states in the U.S. by all accounts before this reform started, has only 31% of its children proficient on a national standard and exam, what does that say about where the bar has been set? Hello, Mississippi?
LikeLike
I think the gap is due to “racism”. My son is the minority in his middle school. He is white. He tries his hardest to get into advanced classes because there aren’t many “minorities” in advanced classes. He wants to be in advanced classes because he can concentrate better. There aren’t kids talking and yelling all the time. He thinks the teachers don’t control the disruptive “minority” kids for fear of being accused of racism.
If those kids would shut the hell up, stop acting like animals and TRY to learn something…there wouldn’t be such a gap.
We live in South Florida…
LikeLike