Policymakers are mostly agreed that they can measure teacher quality by whether scores go up or down.
Research does not support this simplistic notion.
Mostly, researchers have found that teachers in affluent districts get bigger score gains on standardized tests than teachers who are in poor districts, who have many children who don’t read English or have special needs.
And there is what I think of as the cardinal rule: Tests should be used only for the purpose for which they were designed.
A test of fifth grade math tests whether students have learned fifth grade math, not whether their teacher is effective.
Matt Di Carlo here discusses a research study whose author suggests that there should be two different tests: one for students, another for teachers.
I have trouble visualizing what the teacher test would look like, or what it would measure that would be an accurate gauge of “teacher quality.”
But then I have grown increasingly weary of our public infatuation with standardized testing as the answer to our educational needs.
Diane Ravitch's blog 
“And there is what I think of as the cardinal rule: Tests should be used only for the purpose for which they were designed.”
Yes, I’ve been shouting about these UNETHICAL educational malpractices for quite awhile now on this blog and many others.
True that
Effective teaching cannot be quantified.
Try quantifiying effective marriage, effective parenting, or effective morality.
What we do have by now is vast quantitative evidence to prove that privatization is a sham. Let’s focus on those numbers.
M Schneider:
I strongly disagree with your categorical assertion that effective teaching cannot be quantified. You may well have a more valid point about marriage and parenting but that is because you are dealing with a generally small number of discrete outcomes.
Imagine you have to chose a driving school for your child from the 10 or so listed in the phonebook. How do you make the choice? What data would you collect? Would that data tell you something about the effectiveness of the driving instructor?
Are such quantitative measures perfect? Obviously not. Would more qualitative measures help? Certainly. I would argue that neither type of measure is sufficient for making a defensible choice.
The efficient search for a such a school should start with the most readily available pieces of data: successful driving test completion and variants thereof. Once you have found the best 2 or 3 schools on that metric then you can inspect the establishment or go for a ride-along. To start with the latter “qualitative” assessment does not make sense as an efficient search strategy.
Bernie,
Let’s quantify . . . Which teacher would you evaluate higher based on your quantification of teaching?
a. A U.S. History teacher who has been given all AP and Honors students whose students achieve an overall average of 82%, with an average of one or two failing students out of eighteen for the year in each class.
b. A U.S. History teacher who has been given all Academic and General students who earn a class average of 78%, with an average of five failing students out of twenty seven for the year in each class.
C. A U.S. History teacher who has been given the 9R classes of 18 year old Freshmen who have earned no credits, with a class average of 68% with one failure in each class.
All other factors are the same; it is in the same school, same year, same faculty. By the way, we will assume that teacher A’s students all achieved a proficiency rate of 100% on the state standardized test (first attempt), teacher B’s students had an 80 % proficiency rate (85% first attempt), and Teacher C’s students had a 50% proficiency rate (100% taking it for the fourth time.)
Bill:
Excellent and realistic question. I am going to have to think about it – although my quick answer is that no ordinal ranking of your three distinct cases is feasible and therefore no choice can be logically made. (see Kenneth Arrow’s Impossibility Theorem)
I will think on it further but it really does not respond to my immediate comment in which I pointed out that M Schneider cannot categorically say that effective teaching cannot be quantified. Proof: Simply take one of your distinct cases and reduce one of the outcome measures while holding the other the same. You may say that that is unrealistic but looking at Connecticut you can find comparators and, of course, statistically one could find even more. My general point is that the type of measurement system we are talking about can be treated as indicators or suggestive but not determinative. Who would chose a driving school where few drivers pass their driving test or where the school’s only student has passed the test or where the driving instructor was extremely friendly and gave lots of positive feedback but had not yet had anyone pass the test?
I am sorry to end abruptly but I have to run to a meeting.
Bernie,
Again, you fail to include the human equation in you reasoning. You cannot quantify effective teaching because each student is a separate independent variable that cannot be factored out. They start with different strengths and weaknesses, different backgrounds, in many cases, with different languages. In Hartford, our student population is highly transient, often transferring out mid-year and returning late in the year. Our graduation rate includes those who have been kicked out of charter schools (who are not accountable for losing those students to us). There are so many independent variables that no quantification is possible. For example, how many students go home in West Hartford and get beaten up on the way with brass knuckles, then experience their home being burned to the ground by an arsonist the next day? It happened to more than one of my students, many of whom live in gang-infested neighborhoods with little or no police protection.
You MUST include this unpredictable and undeniable human element instead of simply discussing statistics.
That question is not unrealistic; it happens in every school in every district throughout the United States. It is reality for teachers.
Bernie,
I believe that you just made our arguments. You said, . . .although my quick answer is that no ordinal ranking of your three distinct cases is feasible and therefore no choice can be logically made.”
Sorry to disappoint you, Bernie, but effective teaching cannot be quantified. There are too many variables outside of the teachers’ control. For as many “discrete outcomes” that one might connect to me and that label me a good teacher, there are as many “discrete outcomes” that one might connect to me and use to label me a poor teacher. The first problem is in the lack of purity of the connection between me (independent variable) and the student-manifested outcome (dependent variable). Then there is the quality of the measure (psychometric reliability). Then there is the subjective setting of some score that determines my :goodness.” Then there is the valid application of that score (psychometric validity).
As for choosing a driving school, I would as for a recommendation from a trusted friend. See, no numbers.
How about something like value above replacement teacher?
Do teachers of AP and honors students typically have a 90% overall achievement rate with no failures? In that case, teacher “a” would appear to be below average, and assigning a different teacher to that class may well be expected to be an improvement for the student. If the typical teacher assigned to a 9R class as a class average of 50% with numerous failures, your “c” teacher would be very difficult to replace and I would argue that you should do whatever it takes, including paying the teacher more, to keep that person in the classroom.
“I strongly disagree . . . effective teaching cannot be quantified. . . ”
Bernie, read Noel Wilson’s “Educational Standards and the Problem of Error” to understand why (see #1 below also) found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society
Amen!
I’m sorry but the quantification you suggest is no substitute for the way parents evaluate teachers in a neighborhood schools or choose a driving school. An ordinary suburban or small-city parent with his ear to the ground (or who puts several kids through the same school) knows that there are virtually no teachers who are ‘good’ or ‘bad’ for all; one seeks a good match. When you’re paying a significant chunk of income in property tax in a community school district with local control, the system takes care of itself.
Quantifying the effectiveness of certified teachers via some additional exam should be considered a method of last resort, perhaps the best one can do in an unstable school environment where students and staff are constantly changing, and parents have little opportunity to compare notes. As long as our country continues to fund school budgets based on property taxes, there will be many such unfortunate schools whose parents may need to insist on such a measure to ensure a bare minimum of teacher quality. But don’t mistake this sort of half-assed pseudo-science as some sort of universal panacea to be applied to schools which don’t need fixing.
Bernie,
How does student motivation factor into your equation? Nearly all students entering driver’s training are highly motivated and willingly complete the work needed to pass a test to get their license. The same cannot be said in your average classroom.
The College of Ed that I attended administered the Praxis as a measure of my content and pedagogical knowledge. Throughout each semester’s practica, I was observed many times (6 per semester) and received extensive written feedback from my Clinical Faculty member. The yearlong professional semester brought weekly observations with written and oral feedback along with analyzing my teaching videos. Collaborative cohort meetings–two per week with four of us in a group– were held every other week. All this to say, is there 1 test that genuinely captures the ongoing evolving skills, dispositions, and knowledge that this COE modelled for me and my colleagues?
Has anyone read the disclosure on the SAT that says, “This test is not a measure of what the student learned in high school. It is a predictor of college success.”
The statement cited by Diane has never been acknowledged by the reformers, who constantly violate that cardinal rule of assessment.
With so much test prep being marketed now, the ACT and SAT are not even that. The test score no longer is an indicator of college success; it is becoming an indicator of the effectiveness of ACT and SAT test prep.
Well stated!
So TRUE!
I agree that small differences in SAT and ACT scores do not do a great job predicting success in college, but large ones, say an 800 vs a 400 in math on the SAT do predict a difference in college performance.
The value of the point distinctions on these tests is becoming clouded, and it is due to the gaming provided by professional test prep.
Diane:
I think you are creating a straw man argument. When the only significant difference between two cohorts taking a test is the teacher, then the students performance on the test suggests that one teacher is better preparing the students for the test. Obviously the ceteris paribus assumption seldom is totally true and the size of the cohort always places a limit on the confidence of any inference. But by the same token we can statistically isolate many of these differences and make reasonable assessments. Note I am not arguing that such tests are a good way to identify weak teachers but they may be the only available way that those without a chance to observe actual classroom performance can estimate the quality of teachers. No teacher should be fired based primarily upon the test performance of their students.
A separate issue: If a teacher cannot score close to 100% on a test they give their students, should they continue to be the teacher?
Not my argument. Read the link.
My view is that teachers cannot be evaluated by test scores. Period.
Those in affluent districts get good scores. Those teaching needy students do not.
Why are we the only nation in the world doing it?
Arguing this one out with a numbers-obsessed uncle right now. I too see it as a “period” statement. Glad to have the link here and Mercedes’ points to help support my hunch.
Wait a minute . . . high school students typically have more than one teacher. Which teacher should be held accountable for a cohort’s bad performance, especially if the one-size-fits-all test does not test the curricula under which the student has been taught? And, I’m not sure about your reference to a teacher scoring 100%; no teacher designs the standardized test.
Also, since the tests do not reflect any given curriculum, shouldn’t the teacher teach the required course, or teach to the test? I’d rather teach the course; my students deserve a real education!
Bill:
Two points. First, my comment was a criticism of the assertion that a test of student achievement cannot be used to evaluate the performance of instructors. Mine is a statistical point and is fundamental to any quality measurement system that I am aware of. As I actually said, I do not believe that in general a teacher should be fired based on the performance of his/her students on a test. Note that this is different from identifying teachers who should be examined more qualitatively based on the performance of their students.
Second, as to my 100% comment – I find it hard that anyone would dispute that an 8th grade math teacher needs score close to 100% on a standardized 8th grade math test in order to remain teaching 8th grade math.
As to your notion of teaching to the test, that really depends on the nature of the test. Think about teaching somebody to drive. What is covered by the driving test is certainly not sufficient for safe driving, but it is necessary and therefore should be taught to.
Bernie,
I would agree that an eighth grade math teacher should score 100% on any test he/she give to his/her students. But, we are not discussing classroom tests here; we are discussing high-stakes standardized testing being fostered on us by our current political leaders who seem to be in thrall of the testing corporations and corporate reformers (tests that they themselves have never had to pass).
Your assertion that teachers can be measured based upon student performance on these tests ignores the far too many variables between students. Please see M. Schneider’s comments posted at 4:03. What she doesn’t mention is the fact that administrators often use student assignments as a way of “punishing” teachers; giving a favored teacher the easier students while giving an unfavored teacher the more difficult students.
Could a teacher whose students earned a 50% proficiency rating on the state standardized tests be considered more successful that another teacher whose students earned over 90% proficiency? Of course. It can depend on the student population assigned to the given teachers. You cannot quantify effective teaching. You cite statistical interpretation; I cite the human equation.
Let me be naiive here for a minute or two. Why aren’t those who design tests speaking out against the misuse of their product? I know that money has something to do with it, but are we so amoral and as crooked as a dog’s hind legs that morality and ethics have gone by the way-side?
Because they get millions of dollars from districts and states buying them. Follow the money!
And yes, the country IS that amoral and crooked!
Less than follow the money, I think it’s more that once people have been paid they consider it finished. It’s like the post-closing work after a real estate deal—it is not a high priority once the money has changed hands–it’s just paperwork after that. But you have a good point that maybe the test makers care more about the “deal” than the “product.” Plus I think the deals are so huge there is not much competition or incentive to care.
But like you, I put myself in the naive category. However, I am becoming wiser by reading this blog.
Also, do many companies care about “misuse of product?” Far more noble than most business ethics, yes? The retailers might care that Sudafed can be used to make crystal meth, or do they? And does the drug company? You have hit on an excellent point.
Actually, to be fair to those at the College Board (SAT and AP) and those at the ACT, both groups have said publicly that their student tests should not be used, and were not designed to be used, to evaluate the teacher. The ACT has been less vocal about this because they are also involved in writing several high school tests to be used extensively because they will also be used to evaluate teachers. However, both have admitted to the basic premise of the article.
I just love this comment at the NY Daily News!!!!!
ARTHUR GOLDSTEIN8 hours ago
I certainly have great respect for Kentucky’s legacy in bluegrass music. Bill Monroe was a genius, an American icon. When Kentucky people talk bluegrass, I’m all ears. They know what they’re talking about.
All due respect, when we’re talking education, I’d just as soon listen to Diane Ravitch.
I tend to agree. I can’t find anyone else who is as clear about what she stands for per education (not to blow smoke up ya know), but just giving praise where praise seems due.
I was discussing charters with a friend yesterday who said, ” well maybe our leadership knows something we don’t; like maybe we can’t afford public schools anymore.” I just can’t get behind that. I think there has to be a way. That’s why I follow Dianne’s blog.
Joanne, charters and vouchers are not cost savers and they don’t supply better education than public schools.
Diane– I will memorize that. Thank you.
(Sorry for putting two “n”s in your name before. My name ends with an “A,” btw. )
I think people are trying to rationalize charters and vouchers now because it seems to be taking hold and even if their gut says no, they feel powerless to stop it.
Again, I will memorize what you just wrote me.
I wonder if this broadened sentence makes sense: “charters and vouchers are not cost savers and they don’t supply better education than public schools . . .because it is the still the public being educated.”
??
Because. . .
Maybe the emphasis of public schools has been shifted from who they are for??
Charters and vouchers are not for the public. They are for some of the public.
Or maybe there is no “because”needed. ??
“Well, maybe our leadership knows something we don’t”…
My mom (now 85) told me back in the ’60’s that most Americans of her generation felt that way until Eisenhower’s lie about the U-2 incident was exposed. Those of my generation who had not picked up on multiple leadership lies over involvement in the Viet Nam War surely became cynical with the leaking of the Pentagon Papers in 1971. It may be that each generation needs its clear & provable evidence of leadership lies (over something more substantive than sexual escapades) before it gains the shrewdness necessary to protect its own interests.
Now that 3rd-8th grade teachers all over New York state have received the 20% of their Annual Professional Performance Review that comes from the state test, is the state trying to figure out how to balance the scores of those teachers who DO NOT have state test with those who do? How will that work?
I’m using this post as a starting point for comments related to several topics that we have been discussing on Diane’s blog. Finding this blog has been a welcomed breath of fresh air for a once disheartened and tired teacher. Thanks for the forum Diane!
Diane brings up a point that seems to escape those involved in education who have no understanding of the metrics that should be used to analyze test scores. Since most states who hire companies like Pearson Ed. have some involvement in the development of the questions and the scoring of examinations, we should be able to assume that they meet the basic requirements of acceptable standardized tests. I worked with Pearson, among other companies, when I was a math/science teacher representative on WA and AZ state’s standardized testing oversight committees. You can Google “what are the basic requirements of a fair standardized test” for some information on what a test should at least look like.
With that said, Diane’s pointed comment “Tests should be used only for the purpose for which they were designed”, is the crux of the whole issue. Is the horse pulling the cart, or is it the other way around?
This violation of common sense and a total misunderstanding of the process has cost/is costing/will cost, our country untold billions of dollars, which are desperately needed to effectively educate our students. Instead of these funds being spent on training, hiring and then assisting teachers in their classroom, districts/states are hiring “educational consulting” companies to “help” them understand the test scores and develop remediation which includes further training, testing, analysis, time, etc.
What does one day of this type of “remedial” training cost a school district? Does it often improve test scores? Or is it just another nail in the coffin preventing us from getting our educational system back on track?
Using the “data” and its misinterpretation, states and districts are then goaded into bludgeoning their teachers and administrators over “poor test score performance”. Obviously, if students are not learning, as evidenced by a pencil and paper examination (or computerized version), it must be the fault of the teachers, right? By the NCLB scenario for low performing schools, we then line up all the teachers and administrators and shoot them, put in a fresh batch of teachers and admin, or close the school and send the kids to other schools to re-kindle the ignorance.
So we’re testing, and then analyzing, misusing the data, and then retesting, reanalyzing, training and beating teachers, and on, and on……. it’s like using a wrench to drive a nail, or a hammer to open a can of corn. Remember a hard lesson I learned in a graduate stats class, “statistics can lie, and liars use statistics”.
We use the phrase “holding schools (teachers) accountable” to make educators the scapegoats of a broken system. Looking up and down the chain, who is really responsible for education in our country?
NCLB was obviously flawed, we finally figure out, when we get the epiphany that the goal for 100% of our students to be proficient was improbable from the beginning. We aligned and realigned our state and district standards with national, world and even Martian standards. Still it was a no go.
And then oh, my good Lord….coming to the rescue we have the Common Core Standards, or the Common Core Cannon as I call it, and the associated testing, paralysis of analysis, etc. Another seemingly well-intentioned and misused/misguided process.
Note that I don’t see a problem with, and in fact strongly support, examining students with unbiased and realistic tests aligned to well conceived and appropriate standards. This also assumes the examinations are properly scored and provide REAL useful data. This data is then interpreted in a way that assists the student and their teacher(s) in developing a plan of action that will result in moving the student toward mastery of the standards.
I also whole heartedly support standards for educators that are fairly adjudicated, with the aim being to assist teachers in learning, practicing and improving the use of sound pedagogical processes. Yes, in partial answer to my “who is responsible” question, teachers and administrators certainly have a vital part in this endeavor. But you don’t shoot the messenger because you don’t like the message.
If parents and the general public cannot understand this process, let alone teachers and administrators, how can we ever be successful and maintain that success? The target continues to move, while picking up speed and changing form. We cannot hit it no matter how hard we try.
The fueling of money from state and federal coffers along with the grant blood money from Gates, Murdoch and the other power merchants, has developed a whole cadre of educational minions, including groups like TFA, who are among the duped, believing that they are doing good. Their core statement says “TFA is a national teacher corps of college graduates and professionals who commit to teach for two years and raise student achievement in public schools”. I believe that it correctly states the false premises and basis for their failure.
These college graduates and “professionals” are brainwashed into believing that with the “right stuff”, motivation, inspiration and 5 weeks of training, they can somehow overcome 1-3 years or more of student learning deficits. This is while the real best of the best, experienced, credentialed, hard-working teachers, can barely keep their noses above water, hoping to help students to make some moderate gains, or to at least not slip further behind.
Believing that a poorly equipped teacher can routinely “turn around” students who are unmotivated, disheartened, bored and just plain fed up with how adults have screwed up the educational system, is pure rubbish. It belongs with other idioms of improbability, such as “when pigs fly”.
Any statistics coming from within any educational system must be considered as suspect until they have been properly analyzed. Ask what the data shows on its own; don’t manipulate the data to tell you what you want to hear. Oh, that’s one practice that cost me a district administration job by the way.
Anyone who goes into teaching believing that they will be able to become God’s gift to education after a typical indoctrination and brainwashing from TFA (or similar companies) and two years of experience, is a slice short of a full pizza.
Students come to teachers each year, as a composite of their previous educational experiences, both good and bad. Parents send their students to public schools, believing that they should be able to receive a quality education, without paying thousands of dollars a year, which they can’t afford, to private schools to do the same job. A wise principal told me once, “Jim. You must remember that parents don’t keep their good kids at home and send us the rest. They send us their best and ask us to give them our best effort in teaching them.” Very wise words indeed.
Certainly, some private and charter schools do a pretty fair job of providing sound education to their students. However, most of these institutions have the pick of the litter, and leave the disadvantaged and undisciplined students to the public schools. There are some good inner city and urban programs that work well in helping these students and teachers but since these processes are working, they’ll most certainly be disbanded, blown up or run over by the new Common Core tank.
Education is like any other industry that goes through sound planning processes, implements policies and procedures, trains its managers and employees, monitors results and then tweaks the system to gain the desired results. Oh, wait a minute…maybe we don’t do that do we? Maybe it’s closer to the donkey chasing the carrot held out in front of him on a string and a stick in an effort to get him moving ahead. If the carrot isn’t enough motivation then resort to the stick.
Teachers are faced with a new set of requirements and threats every school term. Their pay will be affected by their student’s test performance and the beatings will continue until everyone’s morale improves. Their supplemental pay will be cut or revoked entirely, or it will be attached to some stringent administrative process that is a waste of time and punitive in itself.
Their administrators are under the gun from the superintendent to ensure that their teachers are jumping through all the right hoops at the right time. Work on the “standardized test practice booklets”; drill, practice, and drill some more on the test questions, including the released items. Teach the students to “master” the questions. Just like Pavlov’s dogs, they will start salivating when they get the “real” test in hand.
But often, students face the anxiety of the weeks of these practice tests, only to be caught off guard by the high stakes examination asking questions that they don’t even vaguely recognize. Could it be due to their teacher being forced to take time to teach the test rather than the curriculum aligned to the standards? When you add up the preparation days for the various tests, state, national, district, etc., is this time being spent all value added? Are we getting results proportional to the effort we are putting in to the process?
As experienced teachers know, this is not a matter of our public or private school systems being unable to do their jobs. It is a matter of tying our teachers hands behind their backs no less, and then telling them to write on the board one hundred times “My students need to achieve; My students need to achieve”.
I could go on, and on, and am in fact working on the outline of a book. I don’t know if it will get written, but I have personally experienced all of the frustrations and processes I’ve discussed here, and more.
I spent 30 years as a nuclear engineer onboard US Navy nuclear submarines and working as a civilian engineer. Wanting to give something more back to my community, I spent 2 years getting my MEd in teaching, and another 12 years as a classroom math/science teacher K-12, mostly grades 5-12, and at the college level. I taught in both public and private schools, including conventional charters and corporation charters. Along the way I got another master’s in educational technology and curriculum and was about ½ through my PhD before my body gave out and I had to medically retire.
Oh, I must mention that I had the good sense to NOT get involved with a TFA chapter that recruits new teachers fresh from the military, with the promise of getting a job within “x” months of completing their training program. It just smelled funny. No, I did it the old fashioned way.
I came into teaching at the age of 46, with several advanced degrees, a pretty keen intellect, the desire to teach and most importantly, the willingness to learn. And learn I did, from day one. I learned from my colleagues, administrators, but most importantly, from my students. Although I improved as a classroom instructor I was never satisfied with either my or my student’s performance. I could always do something better to help them learn and likewise they could sometimes apply a bit more effort.
Just as I felt that part of my responsibility as a leader in the military and government service was to stand up for my charges and act as a buffer between them and the powers above, thus I did for my students. Regardless of how I felt about what we were doing, we did it in as positive and fun way as possible. Oh, and we used the standards to show that what we were covering is what we should be able to successfully demonstrate on any type of examination.
I feel this way now; I felt that way when I was forced to retire.
I’ll leave you with one of the greatest lessons that I ever learned. It was when I was a raw 18 year old Navy recruit, who entered the Navy’s Nuclear Power Program and became a successful and decorated nuclear propulsion plant watch officer and supervisor, and was taught to lead my men in the operation and maintenance of the propulsion system on one of the most advanced weapon platforms on the planet.
This lesson took me to the point I am today. I’ve always believed it, but unfortunately it is a lesson that seems to no longer be taught, or is not understood and acted upon. It applies to any group of humans, any age, any nationality, any color, any religion and any political or social view.
Here it is from HG Rickover, who is termed “the father of nuclear power”:
“Responsibility is a unique concept… You may share it with others, but your portion is not diminished. You may delegate it, but it is still with you… If responsibility is rightfully yours, no evasion, or ignorance or passing the blame can shift the burden to someone else. Unless you can point your finger at the man who is responsible when something goes wrong, then you have never had anyone really responsible.”
Who is stepping forward, raising their hand, and saying “I am responsible”? Or is it more like the two bears in the forest, who when finding themselves in the hunter’s gun sights, point at one another with a “shoot him” look on their faces.
Sir Winston Churchill once said, “an appeaser is one who feeds a crocodile in hope that it will eat him last ”.
Teachers and administrators are the last line of defense in this war on education. I know you’re tired, and weary of the fight, but all that is keeping the education war mongers at bay are you career educators that haven’t gone over to the dark side. My prayer is that you find refreshment and a new vision for now and the future, as you continue to lead your students forward.
If you’d like to send me direct comments, please email me at poohcornerpens@hotmail.com. Thanks for listening.
Jim Sanders
Tucson, AZ
Regarding all of “this”… my favorite quote:
Eisner (2005) wrote:
The uniqueness of the particular is considered “noise” in the search for general tendencies and main effects. This, in turn, leads to the oversimplification of the particular through a process of reduction aimed at the characterization of complexity by a single set of scores. Quality becomes converted to quantity and then summed and averaged as a way of standing for the particular quality from which the quantities were initially derived. For the evaluation of educational practice and its consequences, the single numerical test score is used to symbolize a universe of particulars, in spite of the fact that the number symbol itself possesses no inherent quality that expresses the quality of the particular it is intended to represent.
But what if “… the purpose for which they (these exams and their scoring) were designed” is to scapegoat teachers and undermine universal, locally-run public education, and use the resulting planned disorder to consolidate institutional and economic power?
It’s possible King and Tisch have overreached, and risked setting back their ultimate agenda. This episode has the potential to begin the awakening of the Sleeping Giant: parents. If the parents awaken and begin to resist the criminal misuse of these exams, then King and Tisch will fall.
But if they succeed in controlling the narrative about the test scores, reinforcing the Big Lie that public schools are failing and teachers are at fault, then the tests will have been an infernal success.
You cannot judge qualitative issues using quantitative analysis. Try judging The Mona Lisa or any other painting using a rubric or matrix. Teaching is much more an art form than any other profession. We are not making students via a production line. I can’t believe some of these people who come up with these B.S. evaluations schemes get paid for their work. I feel the same way about much of the media as well.
Do teachers construct an ordering of their students learning and comportment?
What are you getting at? I’m not playing the twenty questions game. If you have a point to make, then make it.
It must be that student learning, unlike teacher performance, is quantitative. Otherwise student grades would be as meaningless as you think evaluation of teaching.
There is a lot that follows from this position. It would be foolish to assume the quality of teaching depends on things that can be measured quantitatively, like years of experience or number of graduate credit hours.
I would make the argument that qualitative observations are just as important in observing student learning and academic progress as quantitative data is. In my opinion qualitative data is more important. No two students learn in exactly the same way. Not all students are good at test taking. There are many forms of assessment.
I don’t remember stating anything about using years of experience or graduate credit hours as a basis for the quality of teaching. I will say that as no two painters paint the same way, no two teachers do either. Ed-reformers try to fit that square peg in that round hole though. Quantitative evaluation, meaning tying tests scores to teacher effectiveness is an asinine and clueless proposition.
Exactly which ed-reformer group are you associated with and who are you? Come out from behind the shadows.
They are both quantitative measures of teachers that are used to determine salary and, if there layoffs, determines which teacher will keep their jobs.
The qualitative observations of students are transformed into quantitative observations of students when a grade is assigned. We map students performances into the number line.
I am just a teacher, an economist, and a public school parent. That is all.
I’m in my state we don’t use those methods to determine a teacher’s salary or seniority in case of layoffs. We don’t have unions, it’s against the law here. That’s a flawed system your state uses.
I will agree with some of your second paragraph, although which data gets more emphasis is the difference in assessing the students overall grade. It’s my belief that there are so many different types of learners that judging or emphasizing quantitative data or high stakes tests scores over qualitative data is just plain wrong and unfair. There many types of intelligence, high stakes testing only shows one of those. It’s like asking an elephant to climb a tree to judge it’s athletic ability vs. a monkey’s.
Finally, you are not “just a teacher” there is no such thing. You are a TEACHER and part of a group that is the most important part of our democratic ideals. I respect your opinions as a parent as well. I’m one too. Be well.
teaching economist:
I think we share a common perspective.
I find this discussion fascinating and frustrating. On the one hand it is difficult to disagree with Diane’s general points that some reforms are too focused on standardized tests or that quantitative measures are applied too blindly or simplistically. On the other hand the aversion to any kind of standardized testing or quantification is baffling. It seems as though many commenters simply do not trust those who are charged with having to make decisions.
I know of no way to improve an existing process or system except through some form of albeit crude quantification and measurement.
I think it may be the nature of Internet groups to pull people to take more extreme positions than they would otherwise argue for. There is common ground here, but trying to find it is not a popular endeavor.
Rick: If you are saying that a quantitative metric never captures all the nuances of teaching, I agree. If you are saying that a well designed metric cannot help identify strong and weak teachers or provide a useful measure of teaching effectiveness then you are simply wrong. We use measurement systems all the time to evaluate multifaceted and complex human performance. Teachers do it every time they give a student a grade. Quantitative (ordinal or ratio, it doesn’t really matter) measurement systems are developed to address particular needs or problems that involve comparisons across time or people or against a baseline. For example, how do you determine the winner of an ice skating competition? How do you decide who to select, to promote or to let go. If you want to decide which painting is more likely to increase in value overtime then I would bet that someone can help you make that decision through the use of a quantitative measurement system. Every NFL team does it when assessing who to draft. Even selecting a person for elective office fundamentally involves a measurement system. No one is or should be arguing that these systems are perfect. Nor in many cases should they be the final arbiter. But, despite their limitations, many are useful and that is the primary reason why we have them.
Bernie,
Subject vs objective? Russian judge or American judge?
Is someone a good coach because their teams win the most games or does that coach have the better players? Sometimes the best coaches (or teachers) have teams that are mediocre or bad. That doesn’t however mean that the coach isn’t a good one, it is possible they just don’t have very talented players.
I can go into a class room and evaluate a teacher without needing to jot anything down, it’s just that obvious. Teacher evaluation is subjective. Using high stakes tests are the same. Students in each and every class room start at different baseline of prior knowledge when they enter that teacher’s room. So we should grade a teacher’s effectiveness on a test that has bias built in because of the person(s) who wrote the test? I don’t thinks so. Why must we put numbers and use rubrics or matrices on everything, especially when it is truly unnecessary and counter productive?
The premise of Diane’s article is that we can’t use testing to judge both students and teachers. It is also my belief we should do neither. What we have is people trying to use a business model for a non-business industry.
As for selecting who I should vote for in an election. I don’t use anything but my gut and listening skills to determine if they believe in the same things I do. I sure don’t test them first.
Rick:
First, if there is no need for a measurement system there is no need for a measurement system. So I for one am not in favor of doing things for which there is no need. As I tried to point out, the need for a measurement system comes from the type of issue that you are trying to address.
When you are trying to evaluate someone’s performance you have potentially 3 related problems: (a) how to persuade the person that you are making reasonable judgments; (b) how to provide feedback to the person that they might find helpful in improving their performance and (c) how to compare one individual with others. In some situations not all these problems will be relevant in others all three will be relevant. These issues stand whether you are making subjective or objective judgments. A good measurement system should address all three issues as well as having technical properties associated with what you are measuring and why, e.g., level of granularity, inter-rater reliability, item reliability…
The interesting meta question is the one you implicitly pose: Should measurement systems be used in education. In general I would say sparingly. However, if some aspect of the system is broken or needs improvement then I see no way to improve the system except by introducing measurement systems of various types. Let me cite an example. The Finnish Education system is working extremely well. It has no need of standardized measurement systems to improve its processes. Even so, in order to gain entry into university students have to take standardized exams that are essentially the equivalent of AP exams without as much emphasis on multiple choice questions. Those not going to University do not need to take the exam. Notice how the need drives the measurement system.
You argue that those pushing measurement systems are inappropriately applying a business model. I disagree. The need for and nature of measurement systems is dictated by the type of problem you are trying to address. Improving the performance of teachers is no different from improving the performance of an athlete or salesman or supervisor in a call center.
Bernie, most teachers aren’t the problem. It how the evaluations are being used to scapegoat them when combined with test scores.
Also, colleges use remedial classes to make more money. Many of those students don’t need to extra background help, it’s a cash flow issue. If you are talking college entrance exams, I’m fine with it, because other criteria are used, besides test scores, for entrance. Some of these same ed-reform B.S. at the K-12 level is happening in the collegiate level.
Bernie, what do you do for a living?
Rick:
I am retired. I was a management consultant. I designed, implemented and evaluated performance assessment, selection and promotion systems for large organizations and I developed surveys and other types of measurement systems. 25 plus years ago I helped design and build some of the first on-line multi-rater survey and assessment tools. I have helped companies grow, merge, turn-around and down-size. I worked with the National Science Foundation on the Education and Utilization of Engineers. I am very much aware of the technical and conceptual limitations of these systems and the dangers of managing strictly or simplistically by the numbers. (My doctoral research involved issues of making decisions when you cannot measure differences among people.)
My wife was a foreign language HS teacher and now teaches English to graduate students from over 20 different countries at a local university and prepares them to take college courses in English and to take the TOEFL.
I have a deep and abiding interest in education. I have read Diane’s excellent if somewhat repetitive Education Reform book and I agree with 80-90% of it. In general I believe HS teachers need to be smarter than 90% of their students and know more than 100% of their students; all teachers need to work harder than 90% of the parents; and that the best teachers need to work with the least motivated students. Principals need to be in and from the classroom and have the respect and admiration of their faculty. They also need to have the wisdom of Solomon, the patience of Job and the courage of David.
Bernie,
I think you understand this better than most then. As an administrator myself I agree with being in classrooms during the day. I would do my paper work after school was over or at home. I believe we have very hard working teachers. Yes, there are a few lazy ones, but that can be fixed. I enjoyed the conversation.
Rick
I just read and listened to something by Charlotte Danielson. Does anyone have anything to say about her? Opinions? Thank you.