Matt Di Carlo noticed an odd sentence in Mayor Michael Bloomberg’s “State of the City” address. The mayor, as is now customary, was applauding his administration for the amazing progress of the schools under his stewardship, and he said the following, in anticipation of the new Common Core assessments:
“But no matter where the definition of proficiency is arbitrarily set on the new tests, I expect that our students’ progress will continue outpacing the rest of the State’s[,] the only meaningful measurement of progress we have.”
Di Carlo skillfully takes apart that sentence to show that the mayor has no idea what he is talking about. If the “definition of proficiency” is “arbitrary,” as the mayor says, then the state tests cannot possibly by “the only meaningful measurement of progress we have.”
Di Carlo provides a graph to show that proficiency rates can be set arbitrarily, in which case they can give whatever answer you want, not a “meaningful measurement of progress.”
I know that Di Carlo won’t agree, but we won’t be able to reclaim education until we stop using statistics to measure what matters most in education. Nor will we be able to have good education until politicians give up their effort to impose their uninformed ideas on the schools and stop claiming credit for work they did not do.
Diane Ravitch's blog 
I agree. Test scores does not tell you everything about what children learn. Classroom learning is much more that taking a test. It is the experienced teacher.
Standardized testing is an expensive, wasteful numbers game. The testing companies are the big winners here.
Diane,
One of my major gripes with the reformy barbarians that have taken over urban education is the unbelievable level of conceptual, statistical and research illiteracy, particularly as these relate to crucial educational matters. The funny thing is that many think they know numbers and research based decision making (and some come from MBA programs where one would think they know something about statistics). Mounting defenses of reform effects on the basis of statistics and constructs many of them do not really understand is very iffy business. I recall vividly my conversations with some of the NYCDOE central office reformers and my superintendent about problems with the instability of very high and very low scores in the school grading scheme and whether the model DOE had adopted compensated for this. Blank stares and obfuscating responses is all I got (and I of course was told I had no choice but accepting accountability outcomes which I knew were fundamentally flawed). This specific problem with the school grading models has now been widely documented. My superintendent at the time, I am certain, does not even understand what a regression is and unfortunately neither do many principals. This is widespread in education – its just more obvious now since the reformers have driven the (flawed) data down everybody’s throats.
I agree with Darwin (well, both of them). Like the word “reform,” “reformers” are using terms, like “test data,” in a distorted manner. In high school and college political discussions and debates, we learned how to ask opponents to “define their terms.”
When they have close to a monopoly on the public dissemination of information, they can ignore that demand.
Testing and data, when used within the limits of validity, help teachers understand students’ strengths and weaknesses. Using low stakes student tests as high stakes measures of teachers’ and entire schools is a caricature of testing and research. As a weapon used to stigmatize an entire class of people (teachers), the abuse of testing and data place the “reformers” in the same general field of junk science as eugenics. It’s the Big Lie, repeated over and over, so that it acquires the aura of established truth.
Matt Di Carlo is usually right on the ball, and I hate to defend Make ’em Private Mike,
but I think Bloomberg’s statement holds up to scrutiny. The only meaningful measure, for Mike, is how New York compares to other areas of New York state, whatever test is used and whatever the definition of proficiency is. In other words, if NYC is doing better than Utica, Albany and Buffalo, the NYC DOE is doing a great job.
The real problem with this is the idea that the thing that has helped NYC make gains relative to the rest of New York State are NYC DOE policies. What is much more likely to be the the most important factor is one the reformers always tell us should be no excuse — socio-economic status. We all know poverty is a chief cause of low test scores. (Technically I should say ‘poverty is highly correlated with low test scores,’ but you get the point.) And, guess what? New York City is less poor compared to the rest of New York State than it was 12 years ago when our billionaire friend first ran for Mayor.
As our friend, the Jersey Jazzman posted (and which was also re-posted here), “Klein compares test score gains in NYC to the other cities in New York State. But he forgets to mention that child poverty has soared in Upstate while remaining relatively flat in NYC. The demographics of the “Big Five” cities in New York State changed dramatically during Klein’s tenure. Yet Klein doesn’t mention this when discussing his record of “success.”” http://jerseyjazzman.blogspot.in/2013/02/joel-klein-as-excellent-as-he-says-he.html
Now do you understand why Bloom’n’Klein want us to compare NYC to the other cities in New York? It’s because the other cities are falling apart.
Posted this on Facebook yesterday, thought I would share here as well…
Left school today feeling great because of what may seem like a small accomplishment of one of my students. Adding unlike fractions.You may think it is small, but when you take into consideration that this child entered the 5th grade unable to multiply and divide, you realize that that accomplishment is huge. Of course, none of that matters because we need to get ready for a test. When I listen to new teachers entering the profession and feeling frustrated because they can’t enjoy those small accomplishments my heart sinks. It has become so difficult to enjoy those accomplishments because we have to prepare for a test, which won’t even give the teachers or the students credit for those “small” accomplishments, achievements, and successes. *Just saying, in an “experiment” aren’t you supposed to keep a variable constant to properly assess the results?*
I’m thinking there is no constant, therefore what are we assessing?You think maybe it’s time to reexamine the system?.
“I know that Di Carlo won’t agree, but we won’t be able to reclaim education until we stop using statistics to measure what matters most in education.”
True.
Spreadsheets don’t teach, people do. Data doesn’t build better schools (Schleicher), people do.
Teach teach teach to the test. First graders reading 4th grade texts and trying to find the authors message? What is going on?! I should be called a professor and not a teacher. I’m teaching college to a bunch of 6 year olds who can only think about is lunch time. I have kids crying and saying their brain doesn’t work in first grade because they do not understand a Math Task. I find myself feeling bad for these kids. No playtime or centers. How boring. I wish I could give them the same educational experience I had.