I am reposting this because I forgot to include a link to the study, its title and the names and affiliations of the authors in the first posting. Pretty awful oversight. Actually inexcusable on my part. I apologize to my readers and to the authors of the study.
We have known for some years that the scoring of state tests is easily gamed. In fact, proficiency rates don’t tell us much, because state officials may raise or lower the passing score for political reasons. It happened in New York for years, when the proportion of students passing the state tests went up and up until it collapsed in 2010 as a result of an independent investigation. The state officials enjoyed their annual press conferences where they announced annual too-good-to-be-true gains. And they were too good to be true. They were fake. When the fraud was revealed, there was no accountability. No one admitted having done the dirty deeds. No heads rolled. Accountability is for “the little people,” as real estate queen Leona Helmsley once said about paying taxes. In education, the little people are teachers and principals. At the top–at state departments of education–heads don’t roll. They crown themselves and use their exalted position to blame those who are far, far below them. Think “Yertle, the Turtle.”
An important new study by Professors Adam Maltese of Indiana University and Craig Hochbein of the University of Louisville sheds new light on the validity of state scores. This study found that rising scores on the state tests did not correlate with improved performance on the ACT. In fact, students at “declining” schools did just as well and sometimes better than students where the scores were going up. The study was published in the Journal of Research in Science Teaching. Its title is “”The Consequences of ‘School Improvement’: Examining the Association Between Two Standardized Assessments Measuring School Improvement and Student Science Achievement.”
Consider the ACT an audit exam.
Consider the state tests an invalid way of measuring student achievement and an invalid way of judging students, teachers, and schools. Consider them an invalid way of closing schools and awarding bonuses and firing people.
When students are prepped and prepped and prepped to pass the state tests, they aren’t necessarily better educated, just prepared to take a specific test. Too much prepping distorts the value of the test.
When your measure is invalid, don’t use it for rewards and punishments.
Perhaps if we used these exams appropriately, just for information, they might begin to have some value. As high-stakes, their validity is corrupted, as Campbell’s Law predicts.
Not only are the scores manipulated, but the circumstances of their release:
http://jonathanpelto.com/2012/07/18/education-policy-is-politics-politics-is-education-policy/
“And they were too good to be true. They were fake. When the fraud was revealed, there was no accountability.’
The real fraud in all of this is the whole process of standards and standardized testing. Start with a falsehood, more likely than not end with a falsehood. Start with an invalid process end with invalidities or as Wilson puts it conclusions that are “vain and illusory”
“Consider the ACT an audit exam.” It suffers the same problems as stated above.
“Consider the state tests an invalid way of measuring student achievement and an invalid way of judging students, teachers, and schools. Consider them an invalid way of closing schools and awarding bonuses and firing people.”
You’re getting there Diane! Until all those involved, from parents, students, teachers, administrators, professors and the talking heads, in other words, society, understand the invalidity of sorting and separating students, grading them, using the “one size fits all” approach that is educational standards and standardized testing, the harm will continue to be inflicted upon the innocent-not only teachers but, and especially, the students.
“When your measure is invalid, don’t use it for rewards and punishments.”
When the process is based on a logical error, that of being able to “measure” (attempting to quantify) teaching and learning (a quality) said process itself is invalid. When a measure is invalid don’t use it!
“Perhaps if we used these exams appropriately, just for information, they might begin to have some value.”
When the exams and the process of making, giving and taking these exams contain so many errors as shown by Wilson then any information gleaned will be “vain and illusory”.
So much effort expended on vanity and illusion. Kind of like a Manx kitten chasing its tail. It won’t ever catch it because it’s not there. Except that what we do with educational standards and standardized testing is worse. The Manx kitten will just tire itself out while we destroy students’ desire to learn, sort and separate them so that there are “winners” and “losers”, and then blame teachers for supposed educational shortcomings.
I sound like a broken record, and I get tired of saying it, but everyone involved in education should read and understand what Wilson says in “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577 or at least read and understand his “A Little Less than Valid: An Essay Review” found at:
http://www.edrev.info/essays/v10n5index.html
Alfie Kohn has been telling us this for at least a decade now.
http://www.alfiekohn.org/articles.htm
What about the kids who take these tests (and lots of practice tests to prepare for these tests) and stand to gain little or nothing from the process since test questions and answers are embargoed?
ACT is increasingly selling k-12 testing, tracking and teaching packages to states and districts (PLAN and EXPLORE), using their own corrupt “college readiness” standards to create the demand. If the ACT was a legit audit exam, it won’t be for long.
Michigan now uses the ACT as part of its state-required high school test in 11th grade.
The state also provides free online ACT prep. http://mi.gov/documents/mde/ACT_online_Prep_Communication_383411_7.pdf.
By entering into the marketplace in this and other ways, it seems to me that ACT is compromising the value of its test as an independent measure of anything.
Diane, would you please provide a link or perhaps the names authors of about the Indiana study? I’m not disagreeing with it. I would like to read it.
Many young people who graduate from high schools have to take remedial courses on entering universities. We have sponsored several meetings bringing together high school and college faculty to work with and learn from each other. We think it will benefit students if they are better prepared for college/university expectations.
This is not to say colleges and universities are perfect. But we’ve had instances here of students provided with 2 years worth of scholarships, having to spend much of the money taking remedial courses. It’s frustrating for them and their families. So I do think closer alignment between graduation expectations and college/university expectations makes sense.
That’s part of why I’m intrigued by the study that Diane cited.
http://newsinfo.iu.edu/news/page/normal/22841.html
Joe, thanks for pointing out my oversight. I have corrected the post, with a link to the study and its title and authors.
Diane
Joe,
I cannot speak for your state, but I am so tired of “the schools are not
preparing the kids mantra” and all the blame is on the teacher and never the administrators.
We are told we MUST give a student at least a 50 for ALL four marking periods no matter what they do. And at the end of the year any final grade between a 62-64, must be overridden and made into a 65, which is considered passing.
If it begins to look like they may fail more than one class, they are placed in a self paced computer program where you just take
Pre -tests and then advance your way through a computerized curriculum…very, very few fail. There is no reading of books or writing, just choosing answer choices on the computer.
Those who fail more than two or more classes go to summer school with a 1:1 tutor for TEN hours of work….ten total hours for failing the entire school year and only if you failed two or more. Only one and you are good to go to the next grade.
All of the directives above are requirements…if we refused to give the 50 or bump up a 62-64, then the teacher is insubordinate, so what exactly would you like the teachers to do?
And the constant test prep and loss of real instructional time is only going to make it worse. Welcome to “reform”, USA style.
Does not sound like valid education. Would you care to share what district this is? I can understand you want to remain anonymous but I would like to know what district. Thanks.
Maybe the colleges ought to stop accepting students who are not prepared – those who need remedial courses. That seems to me to be the ultimate incentive for students, parents, communities, schools, teachers, states, to make some meaningful change. But then the colleges would shrink and lose lots of money. If the colleges accept those students, and their tuition money, then they shouldn’t whine about the lack of preparation.
An intriguing idea about not accepting students who are not prepared. You are right that adults are making money from failing students (in this case, community college faculty). We are helping bring high school and college faculty to work with and learn from eachother, to help reduce the # of remedial courses students must take. Are you a public school teacher?
whether yes or no, have you suggested to state legislators that all colleges in your state should be prohibited from enrolling students who don’t have the skills needed for entry level reading, writing & math classes? What do others think of Mary’s suggestion?
http://jonathanpelto.com/2012/03/21/steven-adamowski-governor-malloy-and-perfecting-the-art-of-inflating-test-scores/#comments
Both sides in the value-of-standardized-tests debate seem to love rushing to judgement without a full accounting for the facts. I call this Buffalo Springfield’s “Hooray For Our Side” Law.
First, I see a summary of the study at the link provided, not the study itself.. Of the people who have commented here, how many of you have read the study being discussed? Can you point all of us to a link where we might read the study.
I have not read the study but am trying to find a library or other institution that might have a copy so that I can read it and comment on it.
The summary seems to indicate that the study compared school-level results on Indiana state achivement tests to individual-level ACT scores. If that’s what was done, the study is horribly flawed.
In Indiana only 26% of high school students take the ACT. Those 26% of students are VERY different from the 74% who do not take the ACT in many, many ways. If the author compared individual student ACT scores (26% of Indiana students) to school scores (100% of students at each school) the study’s conclusions are invalid.
I have no idea if this is what happened. I’ll try to review the study as soon as I can find a full copy. I hope people will take heed and read studies before posting unfounded conclusions: It’s the difference between propaganda and information.
For those who have read the study, please point us to a link to the full study if such a link exists.
Ed,
Haven’t read the full study but at the very bottom of the article in the link is this: The full study is available at the Wiley Online Library.
It appears that you need a subscription to access it.
Duane
I have not read the full study but will post it when it becomes available Meanwhile I think it’s ok to accept the authors’ summary of their findings
Diane Ravitch
All testing is flawed. A student’s achievement level cannot be truly known based on one measure, such as a state test. Rather, a profile of their performance on multiple measures must be examined. I am not proposing that we test students more. Classroom tests, work samples, projects, etc. should also have weight when determining a student’s level of academic progress. The over-emphasis on state testing results is so obviously an unsound practice … It is malpractice, to say the least. When are parents going to protect their children from the false labels placed on them … Basic, Proficient, Advanced … based on some cut off score that mysteriously changes every year? Future generations will look back on this nonsense and think that our current educational leaders were insane!
Thanks for this, Diana. Delaware released its test scores today proclaiming big gains in both reading and math across the board. I’m no conspiracy theorist, but I question all of this. Your basic gist is dead-on, though. These tests really are irrelevant in showing what kids know.
The current metrics used to assess school “success” are certainly flawed. The best learning is hardest to quantify. Personally, I’m more of a proponent of project-based learning, and it pains me to see media and newspaper reports that only focus on End-of-Year exams to measure schools. Somehow, the discourse must change. But there seems to me millions of dollars invested in the test-industrial complex. A shame.
http://www.mindfulstew.wordpress.com