When I spoke to the Texas School Boards Association a few years ago, a member of the audience got up and identified himself as a school board member and an engineer. He said that he didn’t understand why the government tests every child every year. He said that in the industry where he works, it is customary to test the products periodically, on a sampling basis. I will never forget what he said: “If we tested every product, we would spend most of our time testing the product, and we wouldn’t have time left to manufacture or to improve the product.”
I was reminded of that statement when I received this comment from Doug Garnett, who is a specialist in marketing, advertising, branding, communications, and technology. Garnett wrote, just minutes ago:
Where I’m mystified is this belief that in order to have “accountability”, EVERY child has to be tested in the entire nation.
In business, we rely heavily on statistical sampling because it’s flat out too expensive to measure every item. Sampling in manufacturing, sampling in store satisfaction, sampling in purchasing, sampling in advertising impact, sampling, sampling sampling.
The NAEP relies on sampling…because it’s EFFECTIVE!
Imagine this: IF we shifted to a sampling test approach an amazing array of issues would be mitigated. The tests would lose their intensely punitive nature – and evolve toward being instructive and enlightening. They would lose the “high stakes” and become simply learning that informs. And, WE could use their reduced presence to focus on the totality of education instead of creating testing farms.
So…why don’t these so-called “business people” behind reform endorse smart business approach like sampling? Mind boggling…unless we embrace the conspiracy to redirect all of that government spending into the profits of private corporations.
Diane Ravitch's blog 
Of course if business people behind “education reform” acknowledged the wisdom of this useful, cost effective tool–sampling–they would wipe out a large segment of the education market. We already have a sampling mechanism in the NAEP, and it makes no sense to spend millions on curriculum or test prep for NAEP.
LikeLike
The absurdity of the idea that you should measure every child’s so-called progress becomes reasonable when you recognize that the point of testing is to generate profit not produce a better product, I mean student, or human being. No one gets any useful information from the test. And yet testing is succeeding beyond any profit-center’s wildest dreams. Every single kid in every single grade every single year has a test — with a price tag on each product. Taxpayers buy the product. Teachers do the labor of delivering the product, at great expense to their own sanity and ethical well-being. And children are the silenced participants who concede their right to learn — or learn anything about themselves from this awful product. It’s a total scam. You can’t fight it on logic. It’s about scam artists and the theft of public property and children’s futures.
LikeLike
Wow. In a nutshell there’s the whole thing. The mendacity and fraudulence of corporate education reform laid out for all to see.
Thank you Doug Garnett!
LikeLike
The reason they don’t like sampling is because then “accountability” doesn’t work and you get into the old chestnut about teachers being the most important thing in a child’s life (at least in the classroom) and how can we go on with not knowing if the teachers in front of kids are effective.
The funny thing is, a lot of this was all premised on that belief on a couple of effective teachers closing achievement gaps (theoretically). It has never been done. It has been legislated in a few states and continues to be legislated that children not receive multiple ineffective teachers in a row (and schools have a crazy time already trying to schedule students based merely on time periods, credit requirements, class size, and room availability) – imagine then having the wrench thrown in of having to assign them (and invariably you will have impossible situations where there is no possible way to avoid 2 ineffective teachers in a row for every student as ineffective teachers are defined today.
So, we test kids as a proxy for the adults, each and every one of them, to achieve something that’s never been done, and try to bend time and space to make it come about by legality even if practicality gets in the way. And there is a thin line that if a teacher crosses it, they become wholly ineffective and fail, and the proxies for that are so rock solid, you can bet that teacher’s career on them not having done enough.
AFT won’t get what it wants because so long as the testing persists, there are those who will balk and not using the data.
I would bet my home that if the AFT gets what it’s politely asking for, within months to a year of the announcement, the government would back track on its position, or the next administration will choose to employ the levers the AFT allow to exist but ask not be used.
Tests used as intended would make sense to provide information we didn’t have about the overall system. They want micro management though and you can’t micro manage teachers from the statehouse without annual assessment of every student of every teacher (even if those teachers teach untested subjects).
Grade span testing would achieve telling us how individual students are doing even if we did it for every student. The only conclusion that can be drawn, is that testing every student every year is about testing the teacher, not the student. If the strongest interest is in continuing the testing, then all that’s connected to the test will remain even if they promise with pixie dust and unicorn horns that they won’t use it in some ways.
LikeLike
It also just occurred to me, that prior to this, administrators also only sampled lessons.
It only now occurs to me that the next level below annual testing, is daily testing, and judging the effectiveness of each and every lesson delivered to students and whether each and every student was reached and to what degree.
Want to take 1 guess of the only way that can be carried out once they figure out that judging a teacher’s entire year based on one test isn’t effective in determining which educators are effective?
LikeLike
Isn’t that what exit tickets are in a way? A brief assessment (test) if the lesson taught that day has been effective. Problem is that learning is not a discrete process and the sum of the parts had better be more than the whole or we are sunk.
LikeLike
My daughter had a long string of “retired” teachers, student teachers, long term subs, and coaches that viewed teaching as a second job. There were a few excellent teachers along the way, but it hurt her terribly in math. But testing and VAM would not have addressed these issues because we live in a district with much parent support. These teachers continue to teach. We compensated as best as possible at home. Instead, testing and VAM protects the poor teachers and penalizes teachers in the poorest districts or working with the most disadvantaged students.
LikeLike
Great points – great article! The engineer/board member’s words make so much sense. I guess the strategy is to figure out how to get the public at large REALLY UNDERSTANDING THIS as opposed “understanding” all the spin put out by the “ed reformers”. Because, in the end, the “ed reformers” are not interested in what benefits public education – their mission is to privatize public education to support the profits of related corporate industries centered around education and to do this AT ALL COSTS. Trying to convince “ed reformers” is a dead end. And unfortunately the COST is our nation’s public school students and the education profession!
But our nation has to begin with campaign finance reform so that the people with the “largest billfold” aka Gates et al cannot buy national policy and so that politicians once again are beholden to “We the People”. Once this happens, then maybe people will feel as if their vote matters because candidates will actually be nominated based on “We the People” and not based on which uber rich business or individual “buys” them. We the People will no longer be “sold” ideas but will be allowed to make up our own opinions based on an array of viewpoints communicated THROUGH THE MEDIA – yes the media once again will be a medium for revealing differing opinions instead of a medium to “sell an idea”. In this time period an SOS Rally like the one Ravitch and Carlsson-Paige led in DC in 2011 would be front in center in the press – it would not be ignored!
LikeLike
I’m with you, artseagal, in longing for things to be the way they should be – honest people doing things for the right reasons. I just don’t see that ever happening.
The deformers and profiteers just can’t understand what it is we educators do. They don’t understand personally caring about a bunch of kids every day and caring what happens to them and doing whatever you can to teach them the skills you know will help them live good lives.
These things are so far off their radars. What they understand is dog eat dog profit and “achievement.” Because in their business model that’s all that matters. Business folks define achievement as profit margin and salary. Teachers define achievement as looking into the eyes of each child every day and seeing the spark of light and joy that comes from learning something new and knowing that, at least during the 7 or 8 hours of a school day, that someone cares about them.
The business model requires number crunching and charts that show upward profits.
When is the last time you heard anyone asking who their favorite business person is and how that person has affected their lives?
LikeLike
Robert Tellman… your lines bear repeating…
“When is the last time you heard anyone asking who their favorite business person is and how that person has affected their lives?…”
LikeLike
“When is the last time you heard anyone asking who their favorite business person is and how that person has affected their lives?”
I am passing this on to a friend who is on the verge of losing her job at the hands of a vindictive principal. She doesn’t even know what she supposedly did that justified being put on administrative leave pending an investigation.
LikeLike
Campaign finance reform is unlikely with Citizens United alive and well, and ALEC owning state legislatures. These vermin play the pop-a-mole game very well.
LikeLike
Every kid is tested because they are not testing to measure the product (students) they are testing to measure the efficacy of the workers (teachers). If we trust our teachers we do not need the tests…but then again Pearson and Gates would not be making money
LikeLike
The difficulty of the actual tests (Pearson, PARCC, SBAC) turns an absurd idea into an abusive one. The minimum competency demanded by Common Core is not just developmentally inappropriate – it’s inhuman.
LikeLike
Yep. ANY test that “fails” 70% of students is abusive and ridiculous.
LikeLike
ANY standardized test is abusive and ridiculous not to mentions illogical, invalid and COMPLETELY UNETHICAL to use the supposed results for anything.
LikeLike
One problem is that education is not a business, and kids are not products. So why use any kind of business practices, smart or unsmart, on them?
Do tests, especially the way they are presently designed, measure anything crucial a kid will use and enjoy as an adult? Is whipping out answers quickly involve, say, creative or logical thinking?
What is this obsession with speed in education, anyways? Are the kids in some kind of race?
LikeLike
Great questions…to which Arne Duncan et al have no answers. Their reluctance to engage in honest debate on such issues is one of the surest signs that federally mandated “accountability” through testing is not really about education.
Why won’t our education secretary sit down in a public forum with people like Diane Ravitch, Yong Zhao and Alfie Kohn, and argue for his policies? Very simple: because they are indefensible. On the one side, you have reasoned refutations of the practice of high-stakes, timed, standardized tests an the uses to which they are now put; on the other you don’ have counter-arguments — you have slogans and propaganda.
LikeLike
“What is this obsession with speed in education, anyways? Are the kids in some kind of race?”
I don’t remember the research, probably because I thought it was stupid at the time, but someone(s) ran a study(ies) showing the correlation between speed of response and mastery of math facts. I’m sure others have done the same thing with literacy (Aimsweb). The idea is that speed is a good surrogate measure of fluency. That’s why generations of school kids have been subjected to math fact speed tests and timed reading tests. If you were a methodical worker, these exercises could be very demoralizing. If you had learning struggles,…
LikeLike
The question is if the faster kids recall math facts, the faster, more accurately they calculate, the better they are at math.
Is recollection of facts, quick calculations important? So they seem to some; people with quick mind are so darn impressive. But is developing quick minds the purpose of education?
Instead of quoting Socrates on the purpose of education, here’s an honest conversation among parents about their kids, using the correct vocabulary, the one that describes reality.
“The school principal told me yesterday that my Johnny is probably going to be among the top 1% of most accurate kids in his school!”
“Wow, that’s impressive. I cannot complain, though: My Jenny got the ‘freshman regurgitating the most scientific data in 10 minutes’ award.”
“This is great guys. You children probably won’t have any problems scoring high on the newly mandated College Precision Data Recollection test.”
“While I’m happy for y’all, I am also embarrassed. My Kylie just doesn’t do well on any of these tests; her teacher tells me that she is thinking too much.”
“Listen to the teacher! Thinking slows your Kylie down—it slows everybody down. The best way to make sure this doesn’t happen, is signing her up to the online version of the Pearson college prep program, and your Kylie will learn to whip out answers, in no time.”
“Thanks a lot, guys. I can’t wait to turn my depressingly smart and thoughtful Kylie into a precision test-taking machine.”
“You are welcome. Let’s drink a decaff grande latte to our kids’ future to be rich in tests and meticulous information gathering.”
LikeLike
“Do tests, . . . , measure anything. . . ?
Educational tests, teacher made or standadized, measure absolutely nothing as they aren’t measuring devices. One can count how many correct and incorrect answers there are on a test but that does not mean that anything was measured.
LikeLike
As I hope everyone who reads this blog knows by now, I’m 100% against all standardized testing all the time, and most certainly against “accountability” based on standardized tests.
BUT, for those who do believe in such things but just believe that we have too much testing, I don’t understand how sampling would work. There’s already a lot of outcry (quite justified) because teachers are getting rated based on kids and/or subjects they’ve never taught. If the point of “accountability” is to rate every teacher, how could that be done on a sampling basis? Would we just sample 10% of all students and base all teacher evaluations on that 10%? Or would only 10% of teachers get evaluated each year? Or what? I can’t imagine any sampling scenario that teachers would find any more fair than testing each kid each year.
Again, to clarify, my point isn’t that we need to keep things as they are with all kids being tested every year. My point is to be very wary of what sounds like a “reasonable” compromise. If “accountability” based on student test scores is bad (and it is), then there is no better way to implement such a system – it must be eliminated. There’s no right way to do the wrong thing.
LikeLike
For all those too ignorant to understand why test based teacher accountability is really bad (invalid, divisive, ineffective, misleading) policy, what alternative would you present? Until we have an answer that they will agree to, this will continue to be a lose-lose battle.
LikeLike
No answer will be acceptable unless it somehow promises to “objectively” fire bad teachers. Due to the fact that there is always a point of subjectivity in evaluation, the question is, whose subjectivity gets to decide even if it’s built into a mathematical formula.
What current policy does is transfer powers from schools to state houses and I don’t see them accepting a policy giving them less power since they believe schools won’t police themselves or else we wouldn’t be where we are.
If we took away unionized bargaining, gave away a ton of money that ends up in charter schools’ hands tomorrow, and promised to fire a percentage of teachers every year, are the only type of compromises I see being allowable.
We have viable answers for how to assess education and hold teachers accountable – there is no trust though at this point – all of the policy points I enumerated above point to a need to fire teachers by force, destroy schools by force, and do so until the market produces the magical solutions that we haven’t implemented yet because that would be too hard and take too long.
The real answers they’ll agree to are the ones that give educators less power unless they trade away “rights”, freedoms, and money.
LikeLike
I disagree. I think it’s a lose-lose battle so long as we continue to buy into the rephormers oft-repeated lie that we need “accountability” (with the implication that there isn’t any without standardized testing). There are multiple ways for parents to know how their children are doing – report cards, conferences with the teacher, science fairs, open houses, heck, just talking with their kids. How anyone else’s kid is doing is not anyone else’s business.
There are also ways to know how teachers are doing – that’s the principal’s job. Again, it’s not anyone else’s business, just like my performance review at my job is between me and my superiors.
The notion that we need some sort of nationally published stack-ranking system for schools or teachers is ludicrous and we need to say so.
LikeLike
My school district is very good at screening the employer/employee relationship from the public. They do not tar and feather their teachers in the village square. What happens behind closed doors has been on occasion very destructive, so I won’t give them a pass. This idea, though, that public employees should have their entire work career open to public scrutiny has gone much too far. I really like your comparison, Dienne, of teacher evaluations to private sector employment reviews.
LikeLike
Dienne, I agree 100%. So much of the accountability nonsense is a simple matter of improving the management skills of principals. It should be like an annual review in the private sector. However the reformers (and many parents) do not trust such a system.
Your response though proper will never be accepted in the current climate. So now what?
LikeLike
It’s not my impression that teachers would trust a system that worked like an annual review in the private sector. A common complaint i see from teachers (at least on this Web site and similar ones) is that principals are often not qualified to review teachers.
LikeLike
I think Garnett was responding specifically to the claim that annual testing is needed not to rank teachers but to compare schools across states so resources can be directed to under-served students.
If that’s the reason they need annual testing (and that is the reason DFER gave) then a sample should work for that purpose.
LikeLike
“There’s no right way to do the wrong thing.”
Quite correct Dienne, a big TAGO! on that statement. Amazing how few people understand the concept involved with those nine words.
LikeLike
Before we get too hysterical with this debate on when/if to test.
1. It is sometimes dangerous to blame a teacher on the outcomes of testing if the problems were established in earlier grades. Let me give you a very common example.
In a study of grade 2 children in three schools we concluded (by testing) that 41% of these children had high VAS levels (i.e. they were able to use whole word guessing strategies from a very early age) but that success was combined with low phonic skills.
These children are not surprisingly heading towards guess-dependency and phonic deficiency. The consequence is that, about the age of 10, most of these children will not have received phonic remediation because their high guessing capacity places them among the better readers. The guess-dependency makes them base their guesses mainly on the end letters with consequent inattention to mid-word letters. The problem then is that each year, the number of words that share the same end letters (infect, infest, inspect, insect, intact, inept, inflect, inject etc) increases producing ‘the 4th year slump’ (videos at http://www.vasreading echo.com). Inaccurate guessing then undermines enjoyment and meaning and these former high fliers sink back to be merely ‘average’ readers.
Two lessons arise from this: Firstly: the dangers of guess-dependency would have been identified possibly as early as grade 1, certainly by grade 2 and hopefully before grade 3 when guess dependency kicks in IF WE HAD TESTED. Secondly these children’s struggle didn’t become obvious until the age of 10 and it is that grade 4 teacher who gets the blame for the failure of hitherto ‘good readers’ and that would be quite wrong. However had we tested at regular intervals we could have identified that we have a teaching problem in grades 1 and 2 and saved our kids and teachers a mountain of unnecessary grief in grade 4. The tests have to be diagnostic to be effective.
Byron Harrison
VAS Research
Australia
LikeLike
“It is sometimes dangerous to blame a teacher on the outcomes of testing….”
Sometimes?!
LikeLike
Am I correct in saying that you are talking about deficiency testing?
Imo, It’s very different to tell a child “Please answer these questions so that I can see if I need to help you out in your reading” from “Please answer these 100 questions in one minute so that our State’s Grading Committee can evaluate how well you learnt what you were taught, and then assign you a grade.”
LikeLike
Byron,
What is VAS?
Have you read your fellow Aussie’s, Noel Wilson’s never refuted nor rebutted “Educational Standards and the Problem of Error”? If so your thoughts please.
Thanks,
Duane
LikeLike
We agree – but if we need to reach “agreement” with the other side – what I’m repeating are the points where they would veto any plan as being a “return to the status quo” and we all know Diane’s talking points on that. They have no incentive to want to do things that educators say are best – look at NY – Cuomo and Tisch got the evaluation system they wanted, and when it didn’t fire enough teachers, they took aim again almost immediately – even though it COULD fire enough teachers or at least declare them ineffective.
LikeLike
M, my main talking point: we have been living with the status quo for 12 years, and have not gotten the miraculous results that would allegedly flow from annual testing and accountability. Doing the same thing for another decade will degrade teaching and learning.
LikeLike
But one of the main arguments of “Reign,” buttressed by 20 or so pages’ worth of charts and graphs, is that the current state of America’s schools is strong, and all the trendlines are positive. There’s a disconnect here.
LikeLike
Well, another difference between public schools and manufacturing is, manufacturing has a whole specialty dedicated to “quality”. They have quality engineers and certified techs who work not to identify individuals who are “bad” but to improve process and avoid quality problems in the first place. They didn’t just pull people out of the jobs they do now and add “testing and quality assurance and control” to their work day. They paid a whole additional set of people to work that out and monitor it.
This is from the Ohio Dept of Ed:
“1/12/2015
Get ready to practice your test technology
A good step in preparing for the new state tests in English language arts, mathematics, science and social studies in February is for your district and schools to participate in a statewide practice test the week of Jan. 26. This involves using Monday through Wednesday of this week to get ready for the trial test day on Thursday, Jan. 29. A group of your students will take a practice test on Jan. 29 to simulate one test day in your district or schools. On Jan. 30, the department invites you to provide feedback on your experience.”
We don’t have a quality/testing team in my son’s public school. The people who are going to be doing all this work are the same people who are supposed to be teaching classes and running his school. Parents are told the tests take “9 hours” out of the school, that’s the time investment we’re making, but that’s obviously not true. The people working on this can’t be doing two things at once. If they’re putting their “testing and quality assurance” hat on that means they aren’t doing their other jobs.
LikeLike
The engineering of products is different in another way. If the engineers find a faulty product, they throw it away and try to find out where the fault originated. In public schools, children who don’t meet the “standard” can’t be thrown away and it is mighty hard to find out and correct the cause of their differences, which may be environmental or hereditary.
LikeLike
The problem with the engineering approach to statistical sampling is that it seeks to eliminate variances compared a rigid standard. It only works on the premise all products are the same.
LikeLike
Also, the whole premise of quality in engineering is it’s quality as defined by the customer. It’s also quality as defined by the use of the product. If it’s a disposable product quality means something different than if it’s a durable product.
You have a lot of “customers”. My definition of “quality” may be completely different than that of another parent, let alone all your other “customers” – students, employers, politicians, principals, the general public.
LikeLike
Well, it does seem as though schools are adopting a “business” approach that businesses reject. Those who are now controlling public education want to standardize children (the “product”) and to do so by giving everyone the same test, every year. They assume that the test is the remedy, not just a measure. They think that measurement will raise test scores, and everyone will somehow be proficient because they were measured. This is nonsensical. The children who live in almost every other state have lower scores than the students of Massachusetts. We could test them hourly, and that would not “close the gaps.”
LikeLike
Love this discussion. You should note that my comment that Diane graciously republished listed many areas of vagaries (not engineering) where we rely on sampling in business. Like consumer satisfaction (very difficult to measure or and best not to use rigid standards).
My own specialty is in advertising and communication. In my work, very much like education, we are attempting to estimate opinion or the impact of messages. And, we use sampling methods all the time.
We aren’t working to rigid standards – we are attempting to understand soft realities like attitudes, preferences, general understanding of a brand, etc…
LikeLike
Yes, and “soft realities” are the heart of the matter in education too!
As Yong Zhao has pointed out for example, South Korean 8th graders do better than Americans on standardized math tests; but if you ask them questions like “Do you like math” or “Do you think you are good at math” they do worse. So what is the important measure? Could our kids’ greater self-confidence have something to do with the fact that Americans generate more patents and Nobel prizes?
Zhao makes a fabulous analogy to wine: judging the quality of an education by test scores is like judging the quality of wine by measuring its alcohol content. The alcohol content is a real quantity, but it is not the right measure of value.
LikeLike
Jeff,
Zhao still has it wrong with the wine comparison because, yes it is possible to fairly accurately measure the volume content of alcohol. In the teaching and learning process we have no comparable measuring devices that have much of any accuracy whatsoever. There are no logical/true “standards” in the teaching and learning process against which one can attempt to measure a quality and/or quantity of “learning”.
LikeLike
Sampling like the NAEP could be used to develop testing profiles of schools and districts. The sample sizes would almost certainly preclude the availability of teacher level data for evaluation systems. However, sampling is not without its challenges. Ask any school that is selected for the NAEP about the work involved in collaborating with the NAEP representatives in selecting students for the sample and the problems of testing a portion of your students one day for NAEP while trying to have a normal instructional day for the others. In small schools, sampling might not produce a large enough sample of students with certain demographics and monitoring achievement across all populations is the common reason given by elected leaders and civil rights groups for the current testing emphasis.
LikeLike
I really want to know how these schools are “randomly” selected. My school has been selected to do the NAEP for the last three times in a row. I am in a very large district, and I cannot understand how we keep being “randomly” selected.
LikeLike
That’s easy.
They throw your school’s name in a bin containing no others and then randomly pull it out.
LikeLike
It’s a good question. Our middle school seems to almost always be selected, but our other schools are less frequently selected.
NAEP states that schools can be selected more frequently, even every time, if they are relatively unique to the state. For example, if most students identifying as Pacific Islanders in your state are located in your attendance zone your school will be repeatedly selected for that demographic representation.
http://nces.ed.gov/nationsreportcard/about/nathow.asp
LikeLike
“Chairman Alexander’s #NCLB reform bill would give states flexibility, control over testing”
You really have to hand it to him. He’s running rings around the Democrats.
By the end of this Democrats will be wholly defined as The Party of Standardized Testing. Duncan and Murray are not even pretending to offer some benefit to public schools in exchange for the testing. Their one and only demand is testing. They’re offering nothing to public education voters.
I’m actually surprised at how bad it is. I thought Democrats would move a little towards public schools and compromise and away from test and punish, but they’re doubling down. Duncan hasn’t moved at all since 2009 and it looks like Murray is following him.
LikeLike
EXACTLY…why can’t business learn from business. Pearson isn’t testing out every question just a sampling of them before they hand out all those tests…so why wouldn’t our education system benefit from the same random sampling?…oh right because that wouldn’t get rid of public education fast enough for them to sell it all off…it’s not really all about the students is really all about the $$$s.
LikeLike
Maybe…maybe…it’s because testing proponents recognize that every child is an individual, with his/her own strengths and challenges, and would benefit most from quality individualized instruction that is planned, delivered and evaluated by a qualified classroom teacher with the autonomy, funding and administrative support to do what’s right for each student. Yeah, that’s it! (Insert eyeroll here.)
LikeLike
Teacher, so the assumption is that the best way to recognize every child as a unique individual with his/her own strengths and challenges is to give all children a standardized test written by a committee who never met or taught that child?
LikeLike
Sorry, lousy attempt at sarcasm. I am so not being serious! I was implying that testing proponents may figure that the best way to address every child’s need is to test every child with a standardized test that holds them to standardized goals and measures. Which we all know is the worst way to help ensure individualized instruction. But I don’t think they know it.
LikeLike
“Check the assumptions”
Like all good scams,
The testing sham
Has elements of truth
With stats as tool,
Enough to fool
The parents and the youth
LikeLike
Sp & Fr Freelancer here
Many of the comments on this thread seek arguments to change the minds of of voters following blindly along with those pushing all things “accountability”.
We already have facts, research, & rationale on our side 😉 But our position is easily assailed by conversation-closers such as “teachers don’t want to be evaluated” and “teachers unions are [fill in blank: ‘bad’, ‘had their chance and look at the mess they made’, ‘care about themselves not students’ etc]. Could we simplify things for the sound-
bite voters?
1. Appeal to the wallet. Have your school taxes gone down since annual testing began in 2005? How about your per-pupil spending? The posted industry/engrg rationale works well here, explaining why costs can only go up with annual testing of all children.
2. Appeal to ideology. American kids are coddled by the accountability movement. We are discouraging students from setting high goals for themselves. Instead of ‘reach higher’, the accountability movement encourages kids to give up, blame every setback on teacher and school.
that’s all I got… Just looking for universal arguments that don’t raise flags and trigger knee-jerks.
LikeLike
I wonder about the “hold THEM accountable!” tone of the ed reform argument. I don’t find it appealing because it assumes I’m somehow on their team and schools are on the opposing team. There’s US and then here’s THEM- schools and teachers.
I never approached my children’s teachers or schools like that. I always started with the assumption we were basically on the same “team” unless I learned otherwise.
I saw this yesterday, when Duncan said he would “tell” parents if schools were administering too many tests. I don’t really see him as my team leader, versus my child’s public school.
I think it’s bizarre. You’re not my adversary, nor are you my subordinate.I don’t really need anyone to “tell” on you. I don’t see that as his role at all, frankly.
LikeLike
Good idea. Instead of using complicated terminology, let’s show parents what good teaching is and make them suspect that what they see is not easy to evaluate.
So here is this video about pretty amazing teaching of math
Now ask the parents: How would you rate the teacher’s performance in the video? How would you rate the kids performance in the video?
What kind of test would you design to measure the teacher’s performance and compare it to other teachers’ performance?
LikeLike
“RidiculousTesting”?
Ridiculous to persons
But surely not to Pearsons
If business is the game
Then testing’s really sane
LikeLike
“Teacher Accountability”
The teachers are to blame
In every single case
Where test score is to VAM
As extant nose to face
“Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores” — American Statistical Association
LikeLike
The reason testing is so prevalent is because it is business and a moneymaking enterprise for freemarket corporations that specialize in putting products into school to create a consumer market for goods. Pearson makes money on every step of this policy: publishing the texts, publishing the test prep materials, producing and grading the tests, etc. They make billions while putting school districts into the red and jeopardizing the education quality of ALL children. It is alarming and a misplacement of the moneys needed to educate our future citizens. I left education because I could no longer continue to do this! It is not what I signed up for and it does not show any indication of changing soon. Pearson and its like will not let go of this revenue. Nor will the hedge funds that invest in the charter school movement.
LikeLike
How very wearisome. Hitler was not the first nor the last to know that if you shout a lie loud enough and long enough people will believe it.
If people would stop and think for just a moment they would realize the counter productiveness of excessive testing by the state. Sadly that is happening way too seldom by way too many people.
LikeLike
Gordon,
“. . . nor the last to know that if you shout a lie loud enough and long enough people will believe it.”
Well if it can be done with a lie it can most certainly be done with the truth. The regular folks here may see me as a one pony horse show with my constant posting and reiteration of Noel Wilson’s never refuted nor rebutted dissertation. I’m just attempting to prove that if you shout a truth out loud and long enough, then people will start to believe the truth instead of the lies.
LikeLike
Oh, by the way that study is: “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
LikeLike
In addition to believing lies because they are repeated so often, don’t you find that data somehow becomes gospel when it gets written down? No matter how faulty the validity, administration and recording of assessments and their results and the collection and HONESTY of the data, the numbers somehow become magical and written in stone when put in a chart.
If people would stop and think of all the things that could go wrong accidentally or intentionally that affect the “numbers” they would realize that numbers should not be the end-all of judging the world. You CANNOT quantify what is in a person’s head and heart!
Life is not just about testing, ranking, competing, and winning. Neither is education!
LikeLike
“Life is not just about testing, ranking, competing, and winning. Neither is education!”
Robert, I am going to have to start adding your closing remarks to my little book of quotes worth living by. Keep ’em coming!
LikeLike
The narrative of students as “product” does none of us activists any good, because truly it’s a reformer red herring–whether offensive or not–and it seems to me we should ditch it fast. Children are not the “products” in reformers efforts to change education. Children are the consumers. Reformers aren’t working to “improve” children, their brains, or their prospects. They’re working to SELL them stuff. If reformers cared about the quality of learning American children receive, standardized testing would be the last thing they’d subject them to, because it’s the last thing they subject their own children to. They know it’s a colossal waste of their own kids’ valuable learning time and it does nothing to help them or their teachers.
Reformers do care about whether–actually how many–children will form their latest target audience in the Race to the Top of the education “market,” and standardization is the key to quantity in that biz. The testing, charter, and tech industries live and die by test scores. Without scores, specifically standardized scores in quantity, they’d have a much harder time justifying their existence or creating a market worth the investment. Every industry has its labor issues–must cut costs!–and how can you fire teachers in bulk if you don’t have a single digit number by which to “evaluate” a year’s worth of work? What if you have to rely on messy realities, such as what goes on in real classrooms, to understand the nuanced relationship between mentor and mentee? Forget the extenuations of family, nutrition, opportunity, oh never mind. What’s more, reformers simply cannot reduce overhead by firing the small percent of teachers who are phoning it in. That’s why 2/3rd of New York children HAD to fail the state’s standardized tests and why Cuomo and Tisch aren’t satisfied with the junk VAM they originally okayed that returned only 1% of teachers as ineffective. How can you take over neighborhood schools with charters, and raise millions from financial services execs, if you can’t brag about “higher” test scores in the Wall Street Journal? How can you replace entire urban school districts with a warren of administratively redundant and cookie-cutter charters, if you can’t scream “failing” while whacking at a colorful bar graph? How can you sell booklets and applications and assessments on a big enough scale if the whole school year isn’t building up to a single test that the entire nation of children takes, preferably on a computer? Worse, what if teachers and kids actually read good books together, took field trips, created performances, conducted hands-on experiments in classrooms–using old stuff like recycled soda bottles, eggs, and baking soda of course? Invest in Arm & Hammer stock now!
Reformers will trot out every argument–any argument–to keep standardized testing to vindicate the “business” of education, rooted fundamentally in the need for change on a grand scale. The latest, I see, is the “civil rights issue of our time” argument again–that without annual, universal standardized tests, we wouldn’t “know” that children in high-poverty neighborhood (therefore schools) do not score as well on standardized tests as children in middle and upper class neighborhoods. REALLY? What rock do they live under? Seems to me, folks with a shred of sympathy have understood for decades–centuries?–not only this disparity but the far more serious one that poor children have too slim a chance of moving out of poverty. Anti-poverty organizations have been working to change things all along, but with precious little support from the government OR party-going philanthropists. Sampling and the NAEP would provide, has provided, more than enough data for reformers to glean this nugget. Anyway, NOW THEY KNOW. And what happened? This testing revelation has resulted in the worst atrocities of curriculum-trimming, test-prep, and educational “disruption” being visited upon only the poorest schools and districts. The dawning revelation of social inequity makes a convenient defense when what you’re really trying to do is transforms schools into the next strip malls of America.
Standardized testing has nothing to do with improving education–not for wealthy suburbanites in Westchester and not for needy children in the Bronx. It’s all about scale, and propping up a vast and growing “education industry” that’s only worth the trouble (money) of the likes of Gates, Murdoch, the Waltons, and the Bushes, and, sadly, Obama and Duncan, if it’s standardized and millions of customers–I mean children–are buying.
LikeLike
Accountability is a red herring idea which doesn’t even work well in manufacturing, because that, too, is a team effort. Quality circles – collaboration at the ground level works. Singling out some part of the team to bear accountability for results ignores the collaborative nature of the manufacturing process. Assessing sanctions on that piece of the whole, or on the whole itself, does not improve the product– in fact, is likely to decrease quality by ignoring innovative inputs by other parts of the team.
This ‘education’ thing we do is eons less measurable as a ‘result’ than a manufactured product. It has no hard edges; it contracts and expands like the universe: bodies of knowledge are available and accessible as never before, and their value to any ‘market’ changes daily. What possible way to tackle such an effort than with a collaborative ground-level team? Efforts to snapshot a body of information & its appropriate pedagogy/ assessment etc at the national level are doomed to almost immediate obsolescence.
However I still like my ‘don’t coddle the students’ sound-bite for the low-informed voter. In my comment above, I suggest planks in a simplified platform aimed at the low-info voter: 1. have your school taxes or per-pupil expenditure gone down since annual ‘accountability’ testing began?– see this engr’s post for why the cost can only grow higher yet produce lower results… & 2. ‘accountability’ coddles students by encouraging them not to aim high but to blame any setbacks on teacher or school).
Students now, as ever, earn grades and earn diplomas. This is a type of accountability system which has ever been with us and doesn’t need changing. IMHO, complicating this simple formula with the bizarre political correctness implied by shifting the ‘blame’ for less-than-desirable ‘results’ onto teaching, curriculum, principals, entire schools, is a gross error which undercuts any individual’s desire to aim high. This is why civil-rights groups lean toward Common Core [looking for high standards] and VAM/ NCLB [looking for better teachers, better schools]. They are already beginning to get the drift that ‘ed reform’ translates to multiple standard tests which translates to closing schools and firing veteran teachers in poor areas, to be replaced with potluck mediocre charters with low-paid teachers… Perhaps we can win them over by first putting the responsibility for student results back on the students.
I think this is a first step to putting the onus back where it belongs: on the gov for policies guaranteed to minimize opportunities for the poor, working poor, working, & middle classes.
LikeLike
This is an excellent article. Our students are tested by their teacher created tests. All of this other testing is wasting precious instructional time. Why is it that educators, who spend years and year in college, more time spent on professional development and are continually learning are deemed unable to assess the children that they work with on a daily basis? What other highly educated professional is treated the same way?
LikeLike
So what’s professor Ravitch’s current view on testing? Does she support grade-span standardized testing? Or she doesn’t support any standardized testing with any frequency and any level (state, district). How about non-standardized testing?
LikeLike