Politico.com has a valuable new education blog in the morning, written by experienced education journalists.
This morning’s report by Nirvi Shah ponders whether the departure of Tony Bennett will show that his (and Jeb Bush’s) beloved A-F grading system is damaged goods. The discovery that Bennett toyed with the system to protect a school owned (and named for) a major GOP donor is reason enough to doubt its validity.
In fact, if you read the article closely, you will understand that the A-F system is intended to facilitate privatization. It sets up schools to fail and to be privatized. Once a school is labeled D or F, it goes into a cycle of decline that is usually irreversible as families leave, good teachers leave, funds and programs are cut, and the school dies, a victim of failed policies and malign neglect.
Unfortunately no critics of accountability like Bob Schaeffer of Fairtest or Paul Thomas of Furman University are quoted. There is a quote from a Néw America Foundation analyst but she seems to say, one, Bennett really did rig the numbers, but two, let’s not give up on test-based accountability.
I disagree. The evidence is now overwhelming that test-based accountability encourages a slew of negative behaviors, including teaching to the test, narrowing the curriculum, cheating, and gaming the system.
Bennett tried to game the system and got caught. New York State rigged the system to inflate scores but stopped after it was revealed in 2010. Beverly Hall gamed the system and will be tried for cheating. Schools across the nation have abandoned the arts or cut back on recess. The superintendent in El Paso is in jail for gaming the aystem.
How much more fraud and miseducation will be tolerated until thinkers and leaders step forward and admit that test-based accountability IS the problem
Diane Ravitch's blog 
I am still stuck on the question (as I tend to get): is there a generational infatuation with numbers and tallies? I ask this because I see elders I respected as a child sort of acquiescing and defending scores to determine pay, and I recall over the years how very impressed with statistics they have always been. Many Obama fans (who otherwise support traditional democratic values) truly believe in merit based pay as best practice. I don’t get it. So I have concluded that they are just more impressed with numbers and percentages and tallies than those of us now raising children. I am having quite the head to head with some of them. And unlike when I was a child, this time they are not able to convince me they are right.
Numbers do not talk louder to me, necessarily, than human worth. I know numbers give us information, but then we need to be cautious with how we handle that information. It can delude us into self-fulfilling prophecies.
Of course it was Obama that created Race to the Top which has spurred all this value added teacher evals, not to mention he appointed Duncan and has created all of the incentives for states to adopt the Com Core. Perhaps the real problem lies not in a generation, but in the politicizing of Am Education.
I know many “older” folks, who want students to be successful but also believe that the teacher represents educational authority in the classroom, and they dislike the idea if a prescripted curriculum. I don’t find many young educational leaders that value that autonomy in teachers. And I see the word “data” being used more by the younger set if admins and teachers then more experienced ones.
Politicizing education is definitely part of the problem. And since you mention Duncan, who is also a big part of the problem, we need a secretary of education who actually has classroom teaching experience, which would help keep some of the crazy in check. We’ve never had a classroom teacher in that position, so it’s no surprise to me that we always end up with policy that makes actual teachers shake their heads–at best.
Duncan is just a “boy” — immature and well … Can’t say it.
There may be an age divide, but I don’t think it’s at the point you seem to think. Or maybe you’re older than you seem. But anyway, the Moneyball generation is not young anymore. They were young in the 1980s, when statistical analysis started its journey toward mass appeal. Baseball was the perfect vehicle, because (1) it was a fun, popular game, and therefore had more mass appeal than the dismal science of economics; and (2) it had very rich primary data (i.e. data that recorded “primary facts,” such as home runs, hits, walks, etc.) that had been collected for almost a century. I first subscribed to Bill James’ newsletters when I was 9 years old. I bought every one of his Baseball Abstracts. I played Rotisserie Baseball against myself for the entire 1984 season, because I couldn’t find any other kids who knew what it was or were willing to put the time in. You had to log the stats daily by hand, compile the cumulative numbers by hand, and draw your own charts by hand. It was a royal pain in the a$$ but it was truly magical stuff to me.
Now I’m in my 40s. I don’t follow baseball much anymore, but I still believe that when you have reliable primary data, there are all kinds of important and unexpected things you can learn from it. Nate Silver is younger than I am, but he’s basically coming from the same place, although he probably had much better technology than I did and probably didn’t burn as many hours graphing by hand.
But the caveat is always the quality of the primary data. You can have an extremely high degree of confidence that a “home run” or a “hit” or a “base on balls” can be accurately recorded. And you can have a high degree of confidence about how important each of those data points are, although there are debates and reasonable minds can differ. Things get more debatable when you move from primary data to derivative data based on primary data (batting average, slugging percentage, etc.). And they get even more debatable when you try to create Super-Objective Statistics, which are derivative data based on other derivative data (such as “win shares,” or “wins above replacement,” whatever those are).
The problem with the Moneyball (or “sabermetric) approach to education is that it’s not clear what the primary data should be, what confidence we can have in our ability to record that data, and what significance those data have. So a big part of the data obsession in education policy comes from the desire to have better primary data. And a big part of the push for inBloom comes from the desire to standardize the data so we can analyze it like we did with baseball when we were kids. But it’s basically a very expensive experiment with very flawed information and all kinds of real-world consequences that weren’t in play when I was obsessed with Bill James and baseball in the early 1980s.
Thank you Flerp.
Makes sense.
I finished high school in 1991, btw. I have had my head in a music cloud most of my life, so numbers and their significance passed me by (although I “scored” high in math always—-and I did take calculus in high school). I just didn’t understand why that is coming into play now for education. Why now? Why not in 1963? So I figured the older generation had been pushing for it and finally got it (at least that’s how the cheerleaders for paying teachers based on scores I know seem).
I’m a hack musician myself. And I’m a far worse mathematician. I lament the latter, as it essentially makes me illiterate when it comes to all of the most important knowledge and discoveries in astronomy and physics.
That’s such a complex question, it’s far beyond my ability to give any kind of comprehensive answer. If I narrow your question to the reasons for the timing of the application of quantitative analysis to the profession of teaching, I think there are a couple short answers. First, there was no perceived need for that analysis in the mid-1960s, because education spending was extremely low. Teaching wasn’t yet regarded as a bureaucracy to be feared — the spending fears of that era were Medicare and other welfare spending, infused by race. Second, there was no popular language for that analysis in the mid-1960s. It would be 20 years before any politician would know what a “spreadsheet” was. And it’d be another 10-15 years before most politicians knew how to use a spreadsheet. That pretty much lines up with the first attempts at merit pay (in Tennessee, I think, spurred by Dr. Ravitch’s boss at the DOE).
Huge portions is the Democrat and Republican parties would have been horrified
Sorry, trailed off on the last typo-ridden thought. Not worth completing at this stage.
I believe any system of sorting, ranking and stacking children, teachers and schools by test scores is flawed. All of this is a disaster. How do we get people in charge to read and listen? The same people who promote the national standards and insist students read informational text and back up claims with evidence are unable and/or unwilling to apply the same standards to themselves.
Are you against teacher assigned grades? That is the most used system of sorting, ranking, and stacking of students.
Not to answer for Linda, but me, yes, I’m opposed to grading.
That’s a lie teachingeconomist. A grade is an individualized (I do believe oversimplification) of a student’s progress towards course goals. They are not provided to stack students against each other. At best they are used as an exit evaluation to determine if a student has a deep enough understanding of the material.
Any attempt to use grades to compare students is flawed at the outset since the ability to compare students to each other is not the intent of well-formed assessments.
The system makes judgements about individual students based on grades. This group of students gets to graduate from high school, this other group of students does not. This group gets to go on and take the next course, this other group does not. You don’t think you have sorted the students into different groups?
absalutly not. i think its a great idea ,then youll know where your at . and where to step up your game.
Andy Smarick (bellwether education/ and Fordham) has a posting on Education Next this morning defending Bennett again.
We are witnessing the death of shame in this country (and maybe globally). Name anything reprehensible that someone in power has done and I’ll find a line of people vociferously defending it. Kill an unarmed teenager? Flash your junk to unknown women? Lie to Congress? Drone bomb civilians (even American citizens)? Bankrupt the country with toxic mortgage swaps? Not a problem. Now, of course, if you’re an ordinary citizen who doesn’t get off the street fast enough during a protest, well, it’s off to jail with you!
I’m not a fan of “it was ever thus” arguments, because I believe we are living today in the strangest period of human history. That said, it’s good to have some perspective and resist the urge to look at every trend as a downtrend.
The Internet has definitely changed the way these phenomena are discussed, and technology has even made some of these phenomena possible in that they weren’t possible in the past. People have been “flashing junk” to strangers for a long time. They just did it in parking lots and not on Twitter.
People also have been killing unarmed teenagers in America for a long, long time, and they’ve been vociferously defended, too. What’s most notable about this phenomenon today is how unusual it is by comparison to the past. You want to see a lack of shame, read the records of the federal investigation of the Emmitt Till killing, or the transcripts of the murder trial.
And civilian bombings? Dresden? Hiroshima, Nagasaki? True, though, those bombs were dropped by pilots, not drones.
Flerp–
the same thing has crossed my mind. That’s why I am so Pollyanna all the time (well, that, and I am a preacher’s daughter so I was raised to be that way). But civil rights was, from what I’ve read, far more intense and the questions far more awakening of emotion and fear than anything right now. In fact, I suppose what is going on now is in many ways a reaction to civil rights or an offspring of some of the emotions triggered then and how they were handled at the time. We are still trying to sort it out. Even this morning on the radio was an interview with an author about a new book out about Eisenhower and how wars get handed down to presidents, etc.
I am eager to read a book I learned about whereby the Ford Foundation played into the Black Power movement and sort of redirected the energy (TOP DOWN, it’s called by Karen Ferguson). I sort of had my public education despite the black people around me (although some of my favorite and best teachers were black), but now I am so happy to look back and know that I had the interactions I did and that I am FB friends with minorities with whom I grew up. The period we are in right now (who knows what history will call it) has pointed me towards tons of reading about the civil rights era and before it. It helps keep things in perspective.
We started a petition to suspend school grading in Florida. Since 10 PM last night when I put it up, we’ve gotten over 900 signatures. Here’s the link:
http://youpower.democracyforamerica.com/petitions/suspend-florida-school-grading-1?source=facebook-share-button&time=1375408524
We need to suspend, or better yet, cease all school and teacher grading in each and every state!
My inner city neighborhood Detroit high school had an award winning chess team, athletic teams, math teams, dance, band, and choir, drama club, AP classes, everything a real high school should offer. Our principal kept the school relatively free from violence and gang influence. Students came from all across the city, some taking three busses! But the students could not “pass” the standardized test, so they gave us a grade of F, and over the course of three years, using Chicago’s “turnaround” model, destroyed us. This is criminal.
here is an excellent summary of the research on the A-F grading “system,” from Dr. Katie Brooks and Dr. Brooke Kandel-Cisco at the College of Education at Butler University: https://www.dropbox.com/s/1dhywd9zq070yww/A_F%20summary.docx
Let’s not forget the leaders of the NEA, UFT/AFT! They went along with all this nonsense. It’s one reason spurring the growth of the “Badass Teachers Association” because teachers feel the union leadership has let them down
Maine’s Commissioner of Education renewed his committment to Bennett-style reform yesterday. http://bangordailynews.com/2013/08/01/education/education-commissioner-stephen-bowen-defends-maine-school-grading-system-in-wake-of-florida-colleagues-ouster/
Not surprising that Maine’s commissioner supports Tony Bennett and his bad ideas. Stephen Bowen is a member of Jeb Bush’s Chiefs for Change. After the publication of this article, Bowen should have resigned. http://www.pressherald.com/news/virtual-schools-in-maine_2012-09-02.html?pagenum=full
Off Topic – I smell a rat!
Diane – you mention Politico starting an education blog. The Seattle Times (in Seattle!) is starting an education blog – even though we have 2 outstanding blogs already.
http://saveseattleschools.blogspot.com/2013/08/melissa-and-i-are-flattered.html
http://seattleducation2010.wordpress.com/
My experience with Politico is pretty limited, as I’ve found them to be stenographers of the Conventional Wisdom justifying why Geithner’s banks need coddling, pretending AHIP and Pharma aren’t just parasitical bandits … I would trust them to cover education in a manner pleasing to the ringwraiths of Bill Gate$.
I’ve lived in Seattle in ’89 when I was 29. It is a Powers-That-Be mouthpiece. It loves right wing lies on the economy or on working stiffs. it loves to support sell out Democrats who aren’t homophobic twits, racists, bigots or 13th century droolers on xx healthcare, but, are sell outs. Yawn. I repeated myself. Their education blog will be useful because you’ll be able to see what is getting concocted by the ringwraiths of Gate$-Ill-Vain-ia.
rmm
Rmurphy, politico blog does have the great investigative journalist Stephanie Simon. We will watch and see.
“How much more fraud and miseducation will be tolerated until thinkers and leaders step forward and admit that test-based accountability IS the problem”
there are no thinkers and leaders, only crooks and profiteers who need to suffer the “people’s justice”
Unfortunately it will probably require a LOT more revelations about the fraudulent profit based testing/accountability policies before things turn in the right direction. John Merrow, PBS reporter, was unable to get his excellent article exposing Michelle Rhee’s complete failure as Chancellor of Wash. D.C. schools published in any of the mainstream media outlets. One reason may be because the owners are part of the “club” that supports the reformy types. The Washington Post just posted very good profits. Most of the Post’s profits are derived from its subsidiary – The Kaplan Schools.
Part of the problem with reformers is they’re national, but they go from state to state. It’s impossible to compare what they say in one state to what they say in another for an ordinary person who can’t devote an hour a day to their school grading system(s).
I read their latest pitch, in Philadelphia, and they claim the newest product is fair and accurate, unlike that crappy Bennett system they sold to ten states, apparently. Since one of those states that got the Bennett system is mine and we border Pennsylvania, I found that amusing. Now we find out! Maybe the reformer who sold it in Ohio could make a return sales call and explain why Philadelphia gets the new and improved product.
You don’t roll out the prototype in ten states unless it works. If it failed in Indiana it should have been dumped or reworked in Indiana.
It’s like we have some terrible hybrid of the biggest idiots in the private sector, and the biggest idiots in the public sector. “Reform” is the worst of both worlds.
This explains the A-F system a little, Jen.
I’ve come to believe that this is all going to be the beginning of an urban “gentrification” push.
Yep.
To answer the question: Absolutely!
For the benefit of the new readers today and all those who don’t understand why any grading system is “fatally flawed” I refer you to Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 . See below for a summary and comments.
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
The piece that is missing from grading is the qualitative data that supports or undermines test-based accountability. People fail and cheat on a tests, because it’s not the best measure for that individual. Those who thrive on tests have testing taking skills that others don’t. Also, tests are biased based on those who make them. It is as bad as the unit tests concocted by teacher committees based again on teacher bias and for consumption by ALL students. The qualitative piece should discuss student potential, student interests and nurturing it, student strengths and qualities, and meeting those needs.
The implications of testing undermines what teachers have been taught to do which to differentiate because of individual needs. Standard based-grading and test-based accountability is full of hypocrisy and lot of others things. It is why teachers find themselves stressed out by the end of the day or should I say the beginning of the day as the pressure already begins to get Johnny to a Level 3 or 4 like the rest of the kids. Teachers know where their students are and their potential based on diagnostic testing (different than standardized tests) conferring and observation which is in contrast to teaching to the test and its results. Yes, we should have high expectations, but test makers shouldn’t be the judge of that.
I have read inspiring books on how teachers are able to up their test scores. While I applaud those heroic efforts, is teaching to the test and getting great results enhancing the real potential of students? Where does teaching them humanistic qualities fit in which is much more needed than a graded test? What these tests can’t do is to produce acceptable social behavior at the least but only encourages its adverse effects. Teachers can determine who are going to pass standardized tests, but yet the government spends billions because they don’t trust teachers.
Tests are a quick a dirty way of knowing an individual. In real life it takes more than a score. At the end, we want students to be productive citizens and they don’t all need a college degree for this to happen. Grading works for a certain population of kids and does damage to the rest not to mention the chaos it has caused in our education system.
Looks like Florida’s race-based learning goals are headed to court, alongside the unfair VAM teacher evaluation program.
http://www.bradenton.com/2013/08/02/4640691/fla-students-say-discrimination.html
The complaint reads like an anti-affirmative action brief. “The soft bigotry of low expectations.” Fine, eliminate the race- and ethnicity-based for test coals. Then the Southern Poverty Law Center will file a complaint saying that the disparities in test results are discriminatory.
test “goals”
I think you may be right if we assume that the goals were based on current levels of proficiency. The problem comes when assumptions about an individual student’s performance are based on a group prediction. Setting such a goal says absolutely nothing about an individual student.
Not to sound immodest, but I’m right no matter what the goals are based on. Whatever they’re based on, there are at least two arguments that the goals are discriminatory. They’re either discriminatory because the goals are different according to race, which runs afoul of President Bush’s warning against the “soft bigotry” and Justice Roberts’s maxim that “the best way to not discriminate on the basis of race is to stop discriminating on the basis of race”; or they’re discriminatory because at the end of the day, when we learn to no one’s surprise that minorities fall farther short of common goals than whites and Asians, the goals will have turned out to have disparate negative impacts on minorities and schools with high concentrations of minorities. And the Southern Poverty Law Center will make either argument, depending on which can be made.
I agree. I was just trying to allow that perhaps they had established these goals for other than arbitrary racist thinking.
Gotcha. I doubt it was arbitrary racist thinking. I think it’s just a clumsy attempt to prepare for the fallout. But it’s hard to imagine a non-clumsy way to prepare. It does surprise me, though, to see the SPLC parroting the main anti-affirmative action reasoning.
In the link Arnie Duncan praises Bennett instead of condemning him. Duncan is also still defending the flawed reform movement made possible by No Child Left Behind. Duncan is ignoring the law by implementing his Race To The Top.It’s scary that policy makers are ignoring the evidence in favor of ideology. Losers like Rhee, Duncan and Bennett have more influence over education policy than respected educators who base their ideas on sound research. The elites that run the country won’t be happy until they destroy it and steal all of it’s wealth and assets. Schools aren’t like a business and can’t effectively be run like one. The real losers are the children that are receiving a second rate education because of present education policies. Looking back in time I now see the seeds of the reform movement being sown in the 70’s. The people need to revolt and take back their educational systems from the Washington elites. This country needs to return to the time when education and educators were respected and considered important to the countries future and prosperity.