A regular reader who calls him- or herself “Democracy” wrote the following in response to my post about the hype and spin surrounding NAEP scores:
“Diane Ravitch writes this: “Anyone who takes them [NAEP scores} seriously is either a sports writer covering education or someone who thinks that education can be reduced to the scores on standardized tests.”
I don’t disagree. But there are, obviously, plenty of educators and citizens, perhaps even most, who do disagree. They buy into goofy arguments made by the testing business (the College Board, the ACT, Pearson, etc.). They spout the “data-driven” nonsense. They think SAT and ACT scores actually measure “learning” and “intelligence.” They believe that Advanced Placement courses really are “better” than other college preparatory classes. They adopt and implement teacher merit pay schemes based on student test scores. They tout the test scores of their graduates, and of their incoming freshman classes.
Who are these people? School superintendents and school board members. Teachers, Guidance counselors. College admissions officers, and college presidents and board of trustees members. Parents, Politicians.
These are the same people who gamely embraced No Child Left Behind, and who had neither cognitive presence, courage, nor professional conviction to oppose it until THEIR schools were directly threatened.
Many of these same people have now latched onto the Common Core, as a new and improved model of school “reform.” Unfortunately, it’s one that seeks to cure a disease (public schooling in “crisis”) that doesn’t exist. In the process, there’s an incredible waste of resources that might have been used to move in a different research-based direction and affect genuine, meaningful educational improvement.
And what about education reporting. It’s woeful. Or worse. People like Tom Friedman toss off dreadfully ignorant stuff on schooling and test scores. Amanda Ripley passes herself off as an “investigative journalist” and educational “expert.” Jay Mathews at The Post continues to push the “AP is better” myth, while his editors continue to heap praise on the Michelle Rhee-Kaya Henderson regime in DC. At The Educated Reporter and The Atlantic, they seem to have been former sports writers.
I appreciate Diane Ravitch’s efforts to help educate and enlighten those who disagree with her take on test scores. There are certainly a lot of them, and it’s quite an undertaking.”
Diane Ravitch's blog 
Nothing exceeds the arrogance of the ignorant when it concerns topics about which theiy no nothing. Let excellent teachers teach without having to test every five minutes, and watch our students grow.
I find this comment a collection of stereotyping of just about everything, most of which are true or not, depending on the context.
For example, ACT and SAT so what they were designd I do – predict freshman GPA. They don’t measure well at all what the school does. Advanced Placement and IB courses, math for sure, when taught they way they are intended, are better than most, but not all HS courses.
NAeP measures long term trends (like PISA). The problems arise when people misuse and misunderstand the scores.
Tests can have good and valid purposes. The Bar exam, medical boards, architects’ exams. My university did and still does give comprehensive exams in majors.
Tests can also be poorly designed and used for bad purposes. We know many of these.
Sweeping generalizations don’t help us sort things out.
Dear Peter,
Your comment illustrates that you are one of those people who buys into the testing malarkey. Both the ACT and the SAT are very poor predictors of college success.
The authors of a study in Ohio found the ACT has minimal predictive power. For example, the ACT composite score predicts about 5 percent of the variance in freshman-year Grade Point Average at Akron University, 10 percent at Bowling Green, 13 percent at Cincinnati, 8 percent at Kent State, 12 percent at Miami of Ohio, 9 percent at Ohio University, 15 percent at Ohio State, 13 percent at Toledo, and 17 percent for all others. Hardly anything to get all excited about.
The SAT is no better. College enrollment specialists say that their research finds the SAT predicts between 3 and 15 percent of freshman-year college grades, and after than that nothing. Shoe size would work as well, or better.
The SAT is used by colleges for their own nefarious reasons, like boosting their rankings in “best” colleges lists and leveraging financial aid (increasingly and especially for students who don’t need it).
Here’s Princeton Review founder John Katzman on the SAT (and Princeton Review does quite a bit if test prep for the SAT):
“The SAT is a scam. It has been around for 50 years. It has never measured anything. And it continues to measure nothing. And the whole game is that everybody who does well on it, is so delighted by their good fortune that they don’t want to attack it. And they are the people in charge. Because of course, the way you get to be in charge is by having high test scores. So it’s this terrific kind of rolling scam that every so often, somebody sort of looks and says–well, you know, does it measure intelligence? No. Does it predict college grades? No. Does it tell you how much you learned in high school? No. Does it predict life happiness or life success in any measure? No. It’s measuring nothing.”
The ACT and the SAT are used for the purpose of “financial-aid leveraging.” Instead of using a $20,000 scholarship for one needy student, schools can break that amount into four $5,000 grants for wealthier students who score higher, who will pay the rest of the tuition ($15,000 a year) and who will bring the school more cash and “will improve the school’s profile and thus its desirability.”
These are hardly “sweeping generalizaitons.”
And yes, Peter, AP is far more myth than reality.
One problem with looking at specific institutions is that they generally admit students based partially on scores, so they have a narrow range of scores. Because my institution has nearly open admissions but is also a research 1 university, we have experience with an unusually wide range of ACT scores, between 16 and 36. At my institution these scores do predict retention and graduation rates.
As for AP, Peter, here’s a brief run-down on the research:
• A 2002 National Research Council study of AP courses and tests –– an intense two-year, 563-page detailed content analysis –– concluded that AP courses and tests were a “mile wide and an inch deep” and they did not comport with well-established, research-based principles of learning.
• The main finding of the 2004 Geiser and Santelices study was that “the best predictor of both first- and second-year college grades” is unweighted high school grade point average, and a high school grade point average “weighted with a full bonus point for AP…is invariably the worst predictor of college performance.”
• Klopfenstein and Thomas (2005) found that AP students “…generally no more likely than non-AP students to return to school for a second year or to have higher first semester grades.” Moreover, they write that “close inspection of the [College Board] studies cited reveals that the existing evidence regarding the benefits of AP experience is questionable,” and “AP courses are not a necessary component of a rigorous curriculum.”
• Phillip Sadler said in 2009 that his research (with Tai) found “students who took and passed an A.P. science exam did about one-third of a letter grade better than their classmates with similar backgrounds who did not take an A.P. course.” Sadler also wrote in the 2010 book “AP: A Critical Examination” that “Students see AP courses on their transcripts as the ticket ensuring entry into the college of their choice,” yet, “there is a shortage of evidence about the efficacy, cost, and value of these programs.”
• In the 2006 ToolBox Revisited Cllifford Adelman, in a statistical analysis of the factors contributing to the earning of a bachelor’s degree, found that Advanced Placement did not reach the “threshold level of significance.”
• A 2006 MIT faculty report noted ““there is ‘a growing body of research’ that students who earn top AP scores and place out of institute introductory courses end up having ‘difficulty’ when taking the next course.” Two years prior, Harvard “conducted a study that found students who are allowed to skip introductory courses because they have passed a supposedly equivalent AP course do worse in subsequent courses than students who took the introductory courses at Harvard” (Seebach, 2004). Students admit that ““You’re not trying to get educated; you’re trying to look good;” and, “”The focus is on the test and not necessarily on the fundamental knowledge of the material.”
• The Sadler- and Klopfenstein-edited book, “AP” A Critical Examination” (2010) lays out the research that makes clear AP has become “the juggernaut of American high school education,” but “ the research evidence on its value is minimal.”
Public high schools reduce the education of their students to a single number and order the students from highest to lowest. Should that be condemned as well?
http://charlotte.news14.com/content/news/all_nc_news/701328/chapel-hill-carrboro-city-schools-considering-eliminating-class-ranking
This highly competitive district is considering eliminating class ranks.
Interesting to see. Only 13,000 districts to go.
Are you talking about class rank? It’s goofy. And it’s a big reason why students hop into AP courses…for the added weight given to those classes.
It may be goofy, but it is universally done and a traditional part of public education. In my state it does not provide any incentive to take AP classes as high schools do not calculate a weighted GPA.
TE,
Is class rank only a “traditional part” of public education? I thought many private schools also had class ranking.
Ang,
You are correct that many different types of schools traditionally compute GPAs and rank students accordingly. Do they reduce learning to a single number?
TE,
So, if you are aware that elite private schools also issue class rank, why did you refer to class ranking as a traditional part of PUBLIC school, as opposed to just school?
As to the rest of your “question”, I have no idea what that means.
Two reasons. First because 85% of students are educated in traditional public schools, second because a number of posters here speak nostolgically about public schools a couple of decades ago, but seldom mention private schools of the same vintage.
@ TE: Traditions and myths often die hard…It’s “tradition” that marriage is between “a man and a woman,” but restricting marriage to that violates the equal protection clause of the 14th amendment.
There is no good reason to maintain class rank.
Yes, it shoud be condemned TE!!
Our district has eliminated valedictorian and salutatorian and goes with the cum laude labels although I’m not sure how they come up with the designations.
My guess would be by GPA.
“Diane Ravitch writes this: “Anyone who takes them [NAEP scores} seriously is either a sports writer covering education or someone who thinks that education can be reduced to the scores on standardized tests.”
This is from a NYT interview with Arne Duncan, Mitch Daniels and John Engler (a former Michigan governor):
“Engler: I think we need to keep data and academic performance the way we keep it on sports. I mean, we know everything about where we stand in the league in football, but we could be last in the league in mathematics for a decade, and we’d never know it.”
http://economix.blogs.nytimes.com/2013/09/25/a-report-card-on-education-reform/
“Sports writer” isn’t far off, as it turns out.
Meanwhile, if I understand the newsletter I got from my local district, Ohio public school kids will be taking both the old standardized tests and the new standardized tests until we transition fully to the new standardized tests.
Ohio should follow California’s lead. California decided it was a waste of time and money to prepare students for two different tests, when one was about to be discarded. Arne Duncan threatened to cut their funding. The legislature voted to discontinue the old tests. Duncan went silent.
No question about it: Arne Duncan is a tool.
And Mitch Daniels and John Engler? They’re right there with Arne.
Duncan’s last name should be DUNCE.
YS-R,
It is, that’s the French version!
I hate how testing is something that defines every student at all times. I could not describe the amount of time I had to spend in SAT prep classes because I needed to learn how to take a test rather than learning a new concept in order to advance my academic future.
No such general “buy in” exists among educators or administrators, at least in the real world. Down to the last one, educators will tell anyone who is willing to listen that the rise in the number of assessments, the change in standardized testing as well as the adoption of Common Core is doing far more harm then good. In addition, the one question no one seems to want to answer is this: Just WHO is benefitting from these strictly political changes? When school systems (not the educators, but the policy makers) adopted the business model as well as the language of “students as clients,” learning went straight to the bottom of the list of priorties.
If there is no general “buy in” by educators, then there’s a general “cop out” by them. How else do we explain the emphasis schools place on the ACT and PSAT and SAT tests (try asking guidance counselors about them)? How do we explain the explosive growth of AP courses (manufactured by the College Board)?
There simply isn’t any good research to support this nonsense.
Students at my university has an unusually wide range of ACT scores, anything from 16 to 36. Perhaps because of this wide range, ACT scores do a good job predicting both retention and graduation at my institution.
AP classes do earn students college credit while in high school, and that can be valuable to households and students by allowing students to avoid large introductory classes, save some money on tuition, and in some cases graduate early. My middle son, for example, would have been able to graduate from college in three years, though he will end up taking an extra quarter to add a second major.
@TE: My guess is that if and when demographic factors are controlled for, the ACT doesn’t predict much at your school just as it and the SAT don’t predict much anywhere at other schools. They are, largely, worthless.
As to AP courses and college credit, there’s a decline in the number of schools who grant credit, mainly because what they find is the AP courses do not prepare students well for the next level courses (see my comments above). Moreover, VERY few students end up using AP college credits to graduate early and save tuition money. Very few.
We have such a wide range of scores (for you coasters a 36 is the same as 800 800 on the SAT and a 16 is the equivalent of 395 395 on the SAT) that ACT scores have explanatory value. No doubt the relatively small range of scores at highly selective schools like Miami of Ohio which I think was one of the schools in a study you cited, make it difficult to detect a difference.
It would be interesting to see a list of schools that do not give credit for AP. I know that Dartmouth made a bit of a splash dropping it for some classes I think, but it made the splash because it was so unusual.
All of the high end schools my son applied to do accept AP (though frustraightingly they generally did not accept transfer credit from incoming freshman). My institution gives it liberally and many of our highest achieving students walk on campus with at least sophomore status in terms of credit hours.
“No such general “buy in” exists among educators or administrators, at least in the real world. Down to the last one, educators will tell . . . ”
It may not be a general buy in but there are definitely way too many who do and allow these educational malpractices to continue. They aide and abet the standards regime.
And those of us who do try to fight it are the “negative” ones, the troublemakers who need to be weeded out.
In my town, parents obsess over test scores before they enter lotteries. Once they are in, they don’t seem too concerned about scores – especially if their school scored low.
The confession of a good test-taker: I never read an assigned book in entirety when I was in high school. It was part of my secret, quiet, rebellion against the obsession of acquiring points and the focus on testing. I remember thinking to myself that these authors, who poured their souls into creating what was now considered a classic, didn’t write their novels so they would one day be plopped into the lap of a teenager whose only focus was getting the most points on a test. However, I still got good grades because I am good at taking tests.
I learned early on that tests, especially standardized tests, are a game of strategy. When I took these tests, I didn’t do them from the point-of-view of the test-taker but that of the test-maker. My grades and scores were not a reflection of what I knew or how hard I worked but of how well I could play the game of collecting points. I gained enough points to do well at a large suburban school district, be accepted into a good university, and receive a degree.
I think that today’s reformers loved collecting points and valued it as students. Maybe it helped them understand their world and gave them a sense of order and meaning. The way I perceive my world and what I value is very different; for me, acquiring knowledge and understanding is very personal and private because it becomes part of who I am.
So here is yet another message to all of the people in power who are “reforming” our nation’s schools; we do not all think like you or value what you do and that is how it should be. Along with the power that you possess, you also have the responsibility to have empathy, humility, understanding, and respect in order to be good leaders.
I think your last paragraph is a good message for teachers as well.
Maybe she is a teacher. Maybe that is a message teachers believe in. Maybe you shouldn’t assume teachers do not have varied experiences and beliefs. The message is by teachers not for us. Why are you always so demeaning?
Where did I make any assumption about how varied a background teachers have?
I think poster Giannis had a good message in the last paragraph and it is something I try to think about when I teach.
“For teachers”…I wasn’t thinking of you as a teacher. Just your usual put downs of the k-12 kind…as though we don’t already believe it her message.
What put down of teachers? I do entertain the possibility that a teacher does not follow this advice all the time, but that is because I do not want to make unwarranted generalizations.
In our district, and I have said this before, teachers are not ALLOWED to disuss anything directly with parents concerning testing or CC …unless it is in support of it. So we say nothing , but we smile and nod in agreement with them. We simply can’t complain. It is considered insubordinate. They have us blocked from “complaining”.
The union complains at its meetings but never actually gets anything accomplished that pulls back the insanity. Esp in these times, holding on to a job beats unemployment, since no one is hiring.
So only those of is who were able to retire are able to speak out. Even then it isn’t appreciated.
@Deb: That’s a very sad commentary. I’m sure there are plenty of teachers who are afraid to speak out, because they’ll bet labeled as “troublemakers.” Only those with “nothing to lose” are those who speak out…or only those with strong convictions. Or those well-acquainted with research. Or those possessed of professional responsibility.
I agree with you that too often union “leadership” is sorely lacking. Inadequate.
Test scores, oh my! See Alfie Kohn’s thoughts on this. http://youtu.be/enQjXfauQV4
Notice that Kohn puts “awards assemblies” in with the competitive goofiness, and that he talks about social and moral growth as desired outcomes of schooling.
Welcome Students to the new age of Testing Mania
I will be your Tester for the 2013-2014 year.
I would like to see 100% of you make an A on all of the State Tests so the superintendent will visit our school….
All of you will be expected to score 100% on every test or you will not pass this class.
You need to see your guidance counselor today..
1. Drop your Art Class…(Useless class)
2. Drop your Music Class.(Useless Class)
3. Do not participate in Sports(Useless)
4. When you go home..there is no time for you to go out and play..
You should study 2 hours after school at home…then EAT your dinner..
then study 4 more hours..
5. I expect to see all of you in my class for before and after school tutoring on Monday-
Friday…every session….only excuse will be a Doctor’s Note..
6. I expect to see all of you on Saturday school….4 hours….every Saturday…extra credit
for Holiday Saturdays..
7. We have arranged for the library to be open on Sunday and you will be asked to join
a Test Study Group…..every Sunday unless the weather is dangerous..
8. Your parents need to buy a computer or an ipad for you ..You can get a .000005%
discount if you make a 100 on the next test..
9. There is a new calculator that costs $239 at the local office company which you must have in order to compute probability and recursive functions though we will ask you to compute by hand and justify first…..
10. There is a $1 coupon in yesterday’s paper for this calculator…The newspaper only costs $2.50..if you can find it..
11. We expect all of you to grow this year…You do not have time to play …You need only to eat and study…Doughnuts and Soda are a good waker upper so that you can stay awake and study for 4 hours or more each and every night…..We adults know what is best for you….
12..Please remember there will be no extra-curricular activities offered in this school during our Testing which is everyday and on the weekends..so get used to it.
You need to be able to wear the same size as every other student in this country and the world…..so let us get to work…
FYI – Here is one example of what alternative methods of assessment look like. On my conference report, I evaluate these qualities. The students do their own self evaluation using the same form. Both reports are given to the parents. Here is the list:
Moves with Purpose
Able to Work Independently
Attention Span
Completes the Work Cycle
Sensitive to the Feelings of Others
Attentive in Group Lessons
Participates in Class Meetings
Positive Response to Requests
Works without Disturbing Others
Self-Motivation
Chooses Challenging Works
Works for Own Enjoyment
Respects Property of Others
Neatness and Order
Conflict Resolution
I assess and record the academic progress of each student for the sole purpose of making immediate, individual decisions about instruction. However, the assessments don’t interfere or distract the student from doing their classroom work. I assess their written work, observe them while at work, and have individual or group discussions.
Parents want to know how their children are spending their day at school. One method is to have the kids log their time at school and include a description of what they were doing. Did they spend their time learning about geography or did they take a standardized test or test-prep? Parents need more information, especially how “reforms” are affecting their children, in order to effectively question those in power.
[FYI TeachingEconomist and Linda – I am a 1st-6th grade Montessori teacher at private non-profit school who attended public schools through college and also taught in public schools (mostly self-contained ED/special ed), whose children attended private Montessori school and public school, who as a child lived in a single-parent household below the federal poverty line, and majored in music education. I am sure that my life experiences and view points are different than yours or maybe they are similar. I’m okay with that. Let’s support each other.]
Marianne,
TE and Linda have a go at it every now and again, kind of like a sibling spat or lovers’ quarrel. One of those things you just shrug your shoulders and basically ignore.
Thanks Duane! I do think that we should all try to support each other a bit more regardless of what subjects or levels we teach or where we teach. We need to encourage each other to speak up because sometimes it is really hard to do that, especially when you read some of the mean-spirited comments on some of the other blogs or hear so many of the anti-teacher comments. Or maybe we should all watch the “Cats Stealing Dog Beds” video a few more times. That is sooo funny!
MG,
“Let’s support each other”.
In general, I get you.
Put please be aware there are some trolls/”devils advocates”/testing industry advocates/ here.
They do not show any evidence of support for the public commons, public ed or public school teachers.
Disagreement (civil) with these types is good.
We need to sharpen out talking points, rhetoric, etc. and learn how to respond to their blather if we desire to continue to have a public anything.
Just my HO.
Very nice, Marianne.
But you will not find criteria like that in schools where the emphasis is testing.
According to at least one Federal judge, teachers are little more than “hired speech” which means that your speech about your job, especially in school or under the auspices of the school, is subject to control and censorship by the school board andor superintendent. This may be one reason why so many teachers are relying on an anonymous blog or faked name to put their views into circulation. See Evans-Marshall v. Board of Education of Tipp City Exempted Village School District, No. 09-3775 (6th Cir. Oct. 21, 2010). Copy the information about the case to read the full text.
Thanks, you all have just reinforced our family’s decision to have our daughter apply to all test-optional schools. She has a hearing impairment (deaf in one ear and hearing impaired in the other) and dyslexia and qualifies for double-time on all test (this means 4 hours on a Sat. and 4 hours on a Sun. for the SAT, just to be specific). She took the SAT and ACT once each and scored poorly on both. I said, “Enough”. Would you willingly go through 8 hours of testing on your weekend? Did I mention she attends a private, college preparatory school and is in the National Honor Society? She secured an internship at our local art museum and is in multiple clubs. She has not taken an AP class (I told her that’s what college is for). Her SAT and ACT scores DO NOT represent who she is as a student; curious, attentive, hard-working, and persistent. I know there is a school out there who will see that she is the student they want. I have no doubt that she will be successful, because she is motivated and believes in herself. And I try not to engage in conversation with those who have “drunk the Kool-Aid”. But, it’s hard.
Indeed it is hard to escape the mythologies that permeate public schooling…and I say that as a public school advocate. I believe in it. I think it’s important. But it has been compromised severely with the absurd emphasis on testing, and by the belief that more is better, and by reliance on products (ACT, PSAT, SAT, AP) that are not only badly flawed but also that are used for what can only be described as nefarious purposes.
I guess I’m coming late to this party, but, FWIW:
What is most disturbing to me, as a social scientist, is how the concept of “data” has been narrowed down so absurdly. And that is precisely where the problem lies.
There’s nothing wrong with trying to collect various kinds of information about something that is important to all of us. Certain kinds of information can help correct for the very human biases to see what we wish to see and to remember bad examples more than good ones. You can also use collected data to see large-scale patterns that are hard to see otherwise (the “forest for the trees” syndrome).
But EVERYTHING depends on the kinds and quality of “data” that you collect. Standardized test scores are one kind of data; a teacher’s one-on-one assessment of a student in some subject is another. Detailed family background is another kind of very important data (and very hard to collect). But standardized tests only measure one kind of thing very narrowly, and the quality of their data depends entirely on how well the tests are designed.
It seems to me that one of the biggest problems is that current education policy is focusing only on the “data” which is relatively cheap to collect – and where outside contractors tend to receive most of the money. In my state, annual state tests on writing were suspended because of the cost of human grading, and are now only given twice in the grade 3-9 span. The state math test used to have students show their work, allowing both for more subtle grading and more useful information for teachers (many months too late, but that’s something else). Again, dropped because of cost.
Do these instruments test the things that we really care about in our children’s’ education? I think we can say that reading and math tests do capture a little of what we care about, but they are very narrow and imperfect measures.
So, sure, being “data driven” is just dandy. But remember the old programmer’s maxim: “Garbage in, garbage out.” Your conclusions are only as good as the “data” you are using to make them.