The American Statistical Association released a brief report on value-added assessment that was devastating to its advocates.
ASA said it was not taking sides, but then set out some caveats that left VAM with no credibility.
Can a school district judge teacher quality by the test scores of his or her students?
ASA wrote this:
“VAMs are generally based on standardized test scores, and do not directly measure potential teacher contributions toward other student outcomes.
o VAMs typically measure correlation, not causation: Effects – positive or negative – attributed to a teacher may actually be caused by other factors that are not captured in the model.
o Under some conditions, VAM scores and rankings can change substantially when a different model or test is used, and a thorough analysis should be undertaken to evaluate the sensitivity of estimates to different models.
• VAMs should be viewed within the context of quality improvement, which distinguishes aspects of quality that can be attributed to the system from those that can be attributed to individual teachers, teacher preparation programs, or schools. Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. Ranking teachers by their VAM scores can have unintended consequences that reduce quality.”
Now, if teachers account for only1%-14% of the variability in test scores; and if the majority of opportunities for qualit improvemt are found in the system, not individuals, and if VAM ranking “can have unintended consequences that reduce quality,” then it is hard to read this statement as anything other than a warning about the danger of relying on VAM to rank teachers.
But our intrepid team of Harvard economists is unfazed!
What do Chetty, Friedman, and Rockoff say about the ASA statement? Do they modify their conclusions? No. Did it weaken their arguments in favor of VAM? Apparently not. They agree with all of the ASA cautions but remain stubbornly attached to their original conclusion that one “high-value added (top 5%) rather than an average teacher for a single grade raises a student’s lifetime earnings by more than $50,000.” How is that teacher identified? By the ability to raise test scores. So, again, we are offered the speculation that one tippy-top fourth-grade teacher boosts a student’s lifetime earnings, even though the ASA says that teachers account for “about 1% to 14% of the variability in test scores…”
Diane Ravitch's blog 
Let me get this straight: we’re having a knock-down, drag-out fight…over about $1,100 per year over the course of a lifetime of work? This struggle gets stranger and stranger as it evolves. And this effect–even if it does in fact exist–applies to just one in twenty teachers!!! My God, what a waste of time. We’ve spent how much time and money in the last decade to arrive at this discovery about education reform? College and Career Ready means being ready to earn $1,100 more a year for life. That’s about 60 cents an hour.
Are we headed back to the fighting over the number of angels on the head of a pin?
Stupid doesn’t even come close.
As we both know, while the entire argument is about money, it is NOT really about the lifetime earnings of any public school students.
It’s about a LOT more than that, Steve. Think about it.
Reblogged this on David R. Taylor-Thoughts on Texas Education.
These are a bunch of business & economic people who know how to evaluate business models- not teaching & learning. Their unwillingness to concede to different theories mirrors the blind faith and hubris of the economists who failed to predict 2008 economic crash.
The more I read, the more angry I become with the preposterous use of VAM to rate teacher effectiveness. I am irate because this type of bogus assumption is made simply because they “can” collect data.
What angers me most is the manner in which this takes the responsibility of learning AWAY from the student. How insulting that is to the students! It is their work and efforts that teach them the life lessons. The teacher is there for guidance, for inspiration, and for support. The teacher at his/her best helps the student reach his/her own goals. Furthermore, all students should not have the same goals, ortberwise, we would produce drones. I’d that what we really want?
Sure, there are poor teachers, and doctors, and lawyers, etc. But, in the end, students who work to earn their place in the class, in college admissions, etc. can not be disrespected for their own efforts to achieve. A teacher is only as “effective” as her/his students desire. Provide the inspiration and the ability to believe in themselves, and students can soar.
The same is true for teachers. Provide them with the tools and inspiration, and the ability to believe they are contributing to the LIVES of students, and they, too, will soar.
Until testing returns to being a diagnostic instrument instead of a punitive tool to demoralize the entire profession, we are doomed to be chasing our tails for the rest if this initiative.
Our students’ lives are at stake. Teachers’ lives are at stake. Education is not meant to be a gamble, a lottery, or a one size fits all marketing plan.
Investment should be made in students and educators not in software platforms and ipads.
Most of all, give students the credit deserved for the efforts they make?
With apologies to Tammy Wynette
“Sometimes it’s hard to be a teacher
Giving all your love to just one school
You’ll have bad scores, and you’ll have good scores
Teachin’ to the test made up by fools.
But if you love them, you’ll forgive them
For statistics that you don’t understand.
And if you love them, oh be proud of them
‘Cause after all it’s just a VAM.
Stand by your VAM, give it a year or two more
Of classroom observations
And boring students to the Core.
Stand by your VAM, and show the world you love it
Keep teaching lessons from the can.
Stand by your VAM.
Stand by your VAM, and show the world you love it
Stop cursing Arne the Dun-can,
and . . .
Stand . . . by . . . your . . . VAM!”
Love it! If you see this message, I’d enjoy getting to know you. I love a good song parody. It would be great to record somebody singing this!
If you’d like to contact me, my email is joe.nashville.eduoke@gmail.com
The other thing that truly bothers me is no one has trouble figuring it out when they or their child or their school has an outstanding teacher. It’s not like looking for a something hidden in the noise – say like B-mode Polarization at Degree Angular Scales from the big bang. We all know when our kid has a great teacher and we all know when we take a class from a great teacher.
Sounds like you’re in favor of having the students rate the teachers as a major part of their evaluations!
I think it would be reasonable for older students to play a role in evaluating teachers.
…especially the opinions of students who have no desire to be in school…
Those students who have no desire to be in school might be the most important ones to ask about their impressions about what goes on in the classroom.
TE: My state already has students (from K on up) rate their teachers as part of teachers’ evaluations. Parents also rate. It is an enormous waste of time–we had to give up a class period to run our kids through the computer lab to do all of the surveys, because each kid has to survey each teacher. Most of the time, parents didn’t even complete the evaluation. Also, if a class is small, even though the kids did the evaluation, we didn’t get the scores, because they were statistically insignificant. Even our teacher for kids with major special needs–many of whom cannot read–had to be evaluated by his students. For the younger kids, instead of ranking using words (superior, excellent, etc.), they chose colors: red, yellow and green. Some kids got tired of one color and changed to another! Plus, what about the kids who are color blind?
I teach 8th and 9th grade and heard students discussing who they were going to rate what. The teachers that they didn’t like, for whatever reason, were going to get rated lower. Even if the teacher was an excellent teacher but gave a lot of homework or really hard tests. It’s not fair to teachers to give our life’s work to moody, unpredictable teenagers, who may not really realize what impact a teacher is having on them until later.
Those moody, unpredictable teenagers are your life’s work.
My preference would be to ask the students about previous year’s teachers so the students have a chance to see how those teachers prepared them for the downstream classes.
TE, Teens are still developing socially and morally. If you have ever dealt with 12-18 year olds, you would know they have not fully matured as adults. They are looking FOR guidance, not GIVING guidance. To put them into the position of judging an adult normally seen as a leader, mentor, authority figure, and “in loco parentis” is uncomfortable and unfair to the child. We do not normally let people vote or serve till 18, consume alcohol till 21, etc.
BTW, 360 evaluations in business were a very short lived phenomenon. And that was with adults. One of my supervisors laughed and threw them all in the trash.
I have tried anonymous surveys. The response may surprise you. Students are, in fact, very reluctant to criticize their teacher. Often, I only get neutral, general, positive comnents. The occasional negative is often very specific over a grade or unrelated personal issues (i.e. student is angry in general). What I look for is ways to improve the classroom. It is like a doctor asking the patient “do you think I performed your surgery correctly?” Talk about uneasiness.
The best teachers I had were not the most popular. In fact, they would never have won teacher of the month or scored high on student surveys. I didn’t realize that till later in life as an adult when I found their tough, no nonsense, experienced teaching better prepared me for life.
Surveys are not the answer. An personal, trusting, open teaching relationship is much more effective.
I am not surprised that students are reluctant to criticize teachers. I would not ask students about the efficiency of the teacher (your example about the precision of the operation), I would ask about the relationship between student and teacher, the daily atmosphere in the class. Other aspects of the evaluation system will look at the appropriateness of a teachers performance.
I ran across this series of blog entries concerning surveys of high school students. I found the answers enlightening. Perhaps you would be interested as well.
http://grantwiggins.wordpress.com/2014/05/21/fixing-the-high-school/
Relationship-based teaching is important from early childhood onwards.
The lack of children’s maturity in rating teachers aside, look at adults. How many marriages would survive a required annual survey of spousal efficacy? Relationships do not lend themselves very well to intense scrutiny for the sake of compiling quantitative data, ranking and yanking. I think we’d be likely to see a self-fulfilling prophesy.
And “if it ain’t broke…”
Why would you try to survey about a student/teacher relationship in stark, quantitative terms? If you are, as you say, trying to understand the personal relationship, a survey is a blunt instrument and silly. No, I have found an open, honest approachable teaching relationship is highly effective and establishes long term trust. Surveys undermine trust.
I am glad you agree surveys cannot measure teacher effectiveness. Now, maybe the Reformers will have the the same enlightenment.
I read the link grantwiggins. Not sure what that means other than a voluntary response pool of students at a wealthy suburban school. Plus this is a general response, not individual teacher rating. Big difference. I can find links on Google to support just about any position.
I think it illustrates one of the useful things that you can learn from students: The survey asked do the students feel that 1) the teachers really know me, 2) give me encouraging and helpful feedback at least once a week, 3) provide work that lets me play to my talents and interests, and 4) always make the work interesting and relevant to us.
In the second entry of the blog, the survey reported that a large majority of A students disagreed or strongly disagreed with 2-4.
And, TE, I ADORE my unpredictable, moody teenagers. I love teaching this age group, and I think I’m pretty good at it, after 13 years. But reducing my teaching and their learning to 10 questions, given in NOVEMBER of that school year, is ridiculous and insulting to me and to them. And BTW, I got great evaluations, so this isn’t sour grapes to me. I just think it’s unfair to give the power of someone’s job to teenagers. Most of my students agree with me, and think that these surveys are dumb and unfair (their words).
No – I am not sure why you thought that. I think teacher evaluations are basically a complete waste of time and energy. It makes sense to me to have a principal or other teacher observe and provide feedback or for teachers to ask their students for feedback that they don’t necessarily share.
It’s like Mao’s Red Guard. Next thing the students will be eating their professors as they did during the cultural revolution in the 1970s.
More than two million educated Chinese, including teachers and professors, killed themselves due to the persecution of the teenage Red Guard.
Yes, the Red Guard Obama voters.
“We all know when our kid has a great teacher and we all know when we take a class from a great teacher.”
We certainly do not all know when our kid has a great teacher. I sometimes see my kids’ teachers before and after school, but I don’t interact with them enough on those occasions to tell much more than whether I like them personally, which is a very different standard than whether they’re a “great teacher.” I also go to parent-teacher conferences, which gives me more information. But I have no way of knowing what actually goes on during class.
The most popular teacher at the high school where I taught never failed students, told lots of jokes and showed many films and there was never any homework in his class. The kids loved him. The students voted him the most popular teacher on campus annually. He taught history. The staff called him “Mr. Hollywood.” We had another one just like him in the English department, but his jokes weren’t as good so he always came in second for most popular teacher.
Unpopular teachers were usually demanding, gave homework, failed students and didn’t show films or tell lots of jokes. Those so-called unpopular teachers were also under lots of pressure from parents and administration to stop giving kids failing grades, because that would hurt the child’s self esteem.
The only teachers who were verbally attacked by parents were the unpopular teachers and many of the class transfers demanded by parents were from classes where kids had to work or fail, and into classes like Mr. Hollywood’s. But there weren’t’ enough Mr. Hollywoods to meet the demand.
What did I learn from this? Don’t trust what kids bring home and share with parents about who the good and bad teachers are.
In thirty years, I only had one parent who wanted their child put into the toughest teacher’s class. That child was home taught to ninth grade and then the child wanted to be among kids his own age instead of stuck at home with mommy as his teacher. His parents met with the ninth grade counselor concerned about the Hollywood teachers. Their child ended up in my English class and four years later as a senior he was the editor of the high school newspaper. Today, (or at least the last time I had an e-mail from him) he was the news anchor in a mid sized city for one of the major networks. But during those years, there were many parents who wanted their children out of my classes because of the quality of work I demanded and the nubmer of failing grades that kid’s earned for not working. I wasn’t alone. The majority of teachers back then were just as demanding as I was. The Mr. Hollywood teacher was in the minority.
I talk to my kids and other parents. I ask the teacher for a syllabus or curriculum at the beginning of the year. I have my teachers email and communicate if issues are suspected. I look at my kids’ assignments and ask for tests. I establish a cooperative, not confrontational, relationship with the teacher and offer to augment the classroom at home. I check classroom web sites and online gradebooks. As a parent, for 99% of the teachers, this works well, including my special needs kid. As a teacher, I greatly welcome a parent who wishes to work with me towards the success of our student.
But I put no stock in surveys, rumors, or popularity contests. We have parents in my kids’ schools that are petty and vindictive. We lost a great teacher due to a small group of parents angry their kid was not given a lead extracurricular position.
For the last five years, all students at my school have participated in online evaluations of their teachers. Some parents even participate. Here is what I have discovered:
1) The most popular teachers get positive ratings (even though they may not be “good” teachers).
2) The teachers who had more positive ratings had more questionable but fun projects that barely or slightly bordered our state standards. These teachers, however, seemed to connect truly more with students. There is a trade off with all things, I suppose.
3) The teachers who had more negative ratings were the teachers who worked their students. While this wasn’t always the case–some were truly bad–the Honors, AP and math teachers usually take a beating on these evaluations.
4) Parents tended to praise the more lenient teachers and condemn the rule-follower teachers.
5) The more detentions, suspensions, or referrals a student received, the more likely these students rated all teachers poorly.
6) The more detentions, suspensions, or referrals a student received, the more likely the parents didn’t participate in the survey and when they did, they rated the school poorly overall.
7) Most students do not have fully developed evaluation skills to rate their teachers. A student who was constantly reminded to complete his work was likely to rate that teacher poorly. These students also think The Hangover and Transformers were good movies. Despite how we worded the survey questions, it usually boiled down to like or dislike.
8) There were exceptions to these generalities, of course. And that is one reason why we continue to give out this survey. However, we never tied these surveys to teacher evaluations because they are so weak in determining teacher/school efficacy. They were good at giving parents and students a voice. And some of them were good at giving teachers some valuable feedback in the comment sections. But that was about it. And the worse (random) set of surveys always came from the 12th graders. Many of them were just nasty and dirty.
Chetty and other economists would be wise to stop seeing themselves as the ordained bestowers of unquestionable truth and start gaining some insights into “Ethics in Economic Theory”
http://www.paecon.net/PAEReview/issue20/Wilber20.htm
“Ethics in Economic Theory”
Isn’t that a contradiction in and of itself?
From W. Edwards Deming’s The New Economics for Industry, Government, Education (Second Edition), page 33:
“In my experience, most troubles and most possibilities for improvement add up to proportions something like this:
94% belong to the system (the responsibility of management)
6% are attributable to special causes
“We shall understand these proportions after we do the experiment with the Red Beads.”
Then from pages 24-25:
“Differences there will always be between any two people, any two salesmen, etc. The question is, what do the differences mean? Maybe nothing. Some knowledge about variation (statistical theory) is required to answer these questions.
“Ranking is a farce. Apparent performance is actually attributable mostly to the system that the individual works in, not to the individual himself.
“A simple equation will help to understand the futility of attempts to rank people. Let x be the contribution of some individual, (yx) the effect of the system on his performance. Then suppose that we have some number for his apparent performance, such as eight mistakes during the year, or sales of $8,000,000. Then
x + (yx) = 8
“We need x. Unfortunately, there are two unknowns and only one equation. Johnny is the sixth grade knows that no one can solve this equation for x. Yet people that use the merit system think that they are solving it for x. They ignore the other term (yx), which is predominant. …
“Ranking comes from failure to understand variation from common causes.”
I remember in the late 1970s that my dad came home furious with the manner in which he had to evaluate his workers. He was told to put all his workers into quartiles. They had various jobs with different tasks assigned. He had developed his team over several years. He was close to them and trusted all of them almost equally for the skills they brought to their jobs. He could not believe that upper management was forcing this unfair evaluation upon the workers. I know he talked straight to all of them, and I believe he protected their jobs cash best he could. But they had the right to know what was being forced upon him and that he couldn’t avoid it completely. Management experiments have always occurred. Fairness is seldom the rule.
It is really frustrating that this VAM experiment continues to harm educators and students. It will cause even more damage. More lives will be ruined. Ridding education of teachers’ unions takes away our power to do anything about this.
At our school, we have been working under a hostile work environment for over 10 years. Thank goodness the local union reps have decided to grieve without fear. Bullying of great teachers has been the rule. Test scores have taken over the mentality of the state and, therefore, the administrators, as teachers work so hard and do all they can to maintain a high morale in the building. They do this for the kids, as they are walking in personal sadness every single day. Maybe this will end this summer.
My mom, who teaches elementary music at a Title I school, is being told that her hours may be cut again. She has been teaching music to each class for a one-hour block a week. She is now being told that her blocks may be cut to 45 minutes once a week, so that the students have more time for the tested subjects. The principal wants to have a TWO AND A HALF HOUR block of ELA every day. Of course, since test scores are the big thing, that will mostly come down to work sheets and drill, and not literature. I can’t even imagine what that’s going to do to kids.
Here is what Reynoldsburg Schools in Ohio are planning to do for teacher evaluations. Frightening.
http://www.plunderbund.com/2014/06/01/reynoldsburg-school-districts-uninformed-contract-offer-to-teachers/
And they said they studied the research for over a year.
Plus, they are eliminating health care coverage.
At least in our district, the administrators are told not to give 4’s (the highest level). So no one would get that biggest pay raise. A great way to save money! The whole thing is disgusting.
Stats 101 – Extrapolation carries great risk and variability. Correlation is not causation. These guys need a refresher course.
Perhaps they could also examine chaotic systems and the butterfly effect.
Hubris, ego, and a sense of superiority must go with the Harvard degree.
A hundred thousand curses on the people who devised this crap for the enormous damage that they have done. Morale among teachers across the country is at an all-time low.
And VAM is one of the reasons why.
This stuff is insidious. It robs administrators and teachers alike of their autonomy. Everyone knows that it’s invalid, and so the system itself becomes a joke, as it did in the old Soviet Union.
The philosopher Paul Grice once made the extraordinarily important observation that a fundamental condition of communication is that people have to be able to be able to depend upon the other person in the communicating actually believing what he or she says.
Well, NO ONE except a the creators of this garbage and a few clueless plutocrats and politicians and educrats believes it. They know that VAM is a sham, and yet they mouth the language that the account-o-mati-crats have forced upon them, and so their speech becomes riddled with falsehoods, and it all becomes suspect, and it breaks down.
And that’s very, very serious indeed.
And one has to be a damned fool not to recognize this.
Or completely out of touch with what is actually happening, every day, on the ground, in schools around the country.
Or both.
cx: The philosopher Paul Grice once made the extraordinarily important observation that a fundamental condition of communication is that people have to be justified in believing that the other people with whom they are communicating actually believe what they are saying.
In other words, if I can’t trust that you believe what you are saying, then our communication will become totally FUBAR.
That’s what is happening, right now, in schools because people have to go around pretending that they believe that VAM and the standardized tests are valid and that David Coleman is the most profound learning theorist in history. Communication itself is being compromised. Teachers and administrators are becoming dinosaurs caught in the tar pit that is ed deform.
Sorry about the garbled version, above. Tired here after a long night of writing.
Quip from the old Soviet days: “We pretend to work and the government pretends to pay us.”
exactly
And, now VAM is on it’s way to a college and university near you. University professors will soon be ranked on how the students of THEIR students perform. The next step will be to evaluate the grandchildren of the children. Monte Python…we’re ready for the next act.
This is exactly the case. What is being orchestrated here is absolutely unprecedented, and it’s very, very dangerous. I consider the attack on our schools being carried out by the deformers to be every bit as serious as would be a coup in our federal government. I am quite serious about that.
Substitute VAM for Spam:
When we look for factors effecting future success, how can we ignore these? Disruption , caused by implementing poorly designed tests? Constant churn? Removing subjects that students enjoy (music, art, sports) for more test prep? Destroying any future pleasure in reading by using selections that are way beyond students’ reading levels? I’m sure others can add to this list.
The reveal is this word: “economists”
There’s also a team of “economists” who still believe in President Reagan’s “trickle down theory” that doesn’t trickle because the wealth pours to the top of the pyramid defying gravity.
It seems once an “economist” has a theory, they will die believing in that theory no matter the evidence, and in this case, the theory leads to profits for Hedge Fund billionaires—-who cares about teachers, parents and kids when it comes to $$$$$$
These Harvard sell-outs are more interested in billions of dollars than millions of people.
There was also a school of economics at the University of Chicago that gave us the neo-liberal movement that money should move from the poor and middle class to the wealthiest people who know what’s best for everyone else.
The equation looks like this: 99% > $$$ > 1%, or put another way: children living in poverty > $$$$ > Bill Gates and Hedge Funds
Lloyd,
You might be interested in some of the other papers by Chetty. Here is a link to the executive summary (alas, the paper is behind a paywall) for one: http://www.equality-of-opportunity.org/files/Trends%20Executive%20Summary%20January%202014.pdf
I can’t remember the author’s name but he wrote a book about his theory of the importance of a child’s self-esteem, and it lit a fire that revolutionized parenting in America.
Then on his deathbed, this author said he was misunderstood and the movement that spread from his writing to the pulpit of churches to the home about the importance of self-esteem was all wrong. His book even led to textbooks in Catholic parochial schools having chapters on the importance of instilling self esteem in children. There was even a study out of California’s legislature and the elected representative who chaired that study wanted self-esteem taught in the public schools—-fortunately on that one, the legislature didn’t let his proposed bill to legislate the teaching of self-esteem didn’t get out of committee but some public school districts took his proposal seriously and the pressure on teachers to stop failing students grew enormously leading to grade inflation and dumbing down the curriculum.
And decades later, when the damage was severe and test scores were dropping like bombs, who was blamed as the politicians, parents, priests, pastors and administrators slipped into the shadows? teachers
Economics, as the premier pseudo-science of our time, all too often uses the trappings of math and science to mask the voracious appetites of the Overclass, whose greed is so insatiable that it now seeks to ingest a foundational institution of the Republic.
This bogus research has about as much to do with the lives of students and teachers as phrenology does to character and personality development.
And, ooh, he’s from Harvard !!!!!
Be careful, Diane: Chetty may ask to measure the contours of your skull, to see why you have such deviant opinions…
TAGO!
Michael Fiorillo: what Duane Swacker said!
😄
And the defense on this blog of the massaging and torturing of logic, numbers and facts by the accountabully underlings of the leaders of the “new civil rights movement of our time”? I would demand a better class of shills and trolls but “no se le piden peras al olmo” [you can’t ask for pears from an oak tree, i.e., don’t expect the impossible].
VAM is a sham, resting on the very shaky foundation of the scores generated by standardized tests.
Since Señor Swacker has referenced Noel Wilson in this thread, let me channel a little Banesh Hoffman (THE TYRANNY OF TESTING, 2003 paperback of the 1964 edition of the 1962 original, p. 143):
[start quote]
The most important thing to understand about reliance on statistics in a field such as testing is that such reliance warps perspective. The person who holds that subjective judgment and opinion are suspect and decides that only statistics can provide the objectivity and relative certainty that he seeks, begins by unconsciously ignoring, and ends by consciously deriding, whatever can not be given a numerical measure or label. His sense of values becomes distorted. He comes to believe that whatever is non-numerical is inconsequential. He can not serve two masters. If he worships statistics he will simplify, fractionalize, distort, and cheapen in order to force things into a numerical mold.
[end quote]
Wilson and Hoffman by clear superiority in the first minute of the first round, mercy rule invoked to avoid permanent debilitating injury to Chetty and Friends.
😎
Nasty little can of whoop-ass you dumped on them Mr. Fiorillo, nicely done!
How much Gates Foundation money does Chetty get? How much is floating around Harvard? Why don’t we have “teach-ins” on VAM research like we had “teach-ins” on funding for the invention of napalm? The neo-liberals have declared war on public education and VAM is their defoliant.
Pablo 2063
You can check his Vita to see the grants that he has been awarded: http://obs.rc.fas.harvard.edu/chetty/cv.pdf
Even more telling is the fact that Chetty is being funded for three years by the Laura and John Arnold Foundation for “Where is the Land of Opportunity? The Geography of Intergenerational Mobility in the U.S.” –as if this right-wing hedge fund billionaire who aims to destroy people’s pensions gives a hoot about Americans who struggle with upward mobility.
One can see what kind of money Harvard has been getting from Gates here: http://www.gatesfoundation.org/How-We-Work/Quick-Links/Grants-Database#q/k=harvard
Here is a link to that paper: http://obs.rc.fas.harvard.edu/chetty/mobility_geo.pdf
Here is the abstract:
We use administrative records on the incomes of more than 40 million children and their parents to describe three features of intergenerational mobility in the United States. First, we characterize the joint distribution of parent and child income at the national level. The conditional expectation of child income given parent income is linear in percentile ranks. On average, a 10 percentile increase in parent income is associated with a 3.4 percentile increase in a child’s income. Second, intergenerational mobility varies substantially across areas within the U.S. For example, the probability that a child reaches the top quintile of the national income distribution starting from a family in the bottom quintile is 4.4% in Charlotte but 12.9% in San Jose. Third, we explore the factors correlated with upward mobility. High mobility areas have (1) less residential segregation, (2) less income inequality, (3) better primary schools, (4) greater social capital, and (5) greater family stability. While our descriptive analysis does not identify the causal mechanisms that determine upward mobility, the new publicly available statistics on intergenerational mobility by area developed here can facilitate future research on such mechanisms.
The Arnolds are awful. But don’t miss the Richardsons. They boast an impeccably neoliberal (even neoconservative) repertoire of convictions and philanthropic pursuits. Why of course we should find them patronizing the good professor:
http://en.wikipedia.org/wiki/Smith_Richardson_Foundation
TE,
If you are suggesting that our concern for these funding dynamics can be dismissed based on the seemingly tame or even “progressive” conclusions of individual papers, then I think you’re misunderstanding (and underestimating) how philanthropy functions in (and distorts) the academic and think tank worlds.
For more on this, see:
http://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CCcQFjAA&url=http%3A%2F%2Fwww.education-inquiry.net%2Findex.php%2Fedui%2Farticle%2Fdownload%2F22618%2F30133&ei=Pq6MU4C5BJHLsAT61oLIBA&usg=AFQjCNGzlBwREvsA8udKQBqzPMWwaq-3tg&sig2=ZlypqH4OSSBFEwC8cNRXHw&bvm=bv.67720277,d.cWc
https://www.academia.edu/5100201/The_Rise_of_Venture_Philanthropy_and_the_Ongoing_Neoliberal_Assault_on_Public_Education_The_Eli_and_Edythe_Broad_Foundation
P.S. The Arnolds are awful. But don’t forget the Richardsons, sort of the textbook billionaire neocons abroad, billionaire neoliberals at home, and billionaire “centrists” in name. Funny how that works:
http://en.wikipedia.org/wiki/Smith_Richardson_Foundation
Whatever the outcome of the study, I would not believe for one minute that, as the Arnolds take aim at stripping seniors of their means for basic survival, they give a damn about upward mobility for the poor, especially since according to some, “This means some of the children of the affluent must fail.”
http://opinionator.blogs.nytimes.com/2013/09/29/the-glass-floor-problem/
I have no idea what the Arnolds think about anything. In any case, this thread concerns the integrity of the forthcoming AER articles.
Wake up. It matters who is funding the research and John Arnold is the Wolf of Sesame Street:
What really matters is the validity of the research. I do understand that judging research by beliefs about funding sources is much easier however.
What really matters is how the money of corporate billionaires influences policy decisions that are based on interpretations of research to their advantage –although I understand your naivité in believing that politicians and wealthy elites actually care about the validity of research when formulating policies.
In this thread we are discussing the validity of basic research . The best way to understand the validity is to look at the research itself. I believe Chetty has made both the data set and the code they used to do the econometrics publicly available. Feel free to test it for programming errors and to see if alternative specifications make a difference in the results.
TE,
First you pretend that Chetty’s funding isn’t questionable. Then when we call you on it you pretend that funding details are irrelevant. For the record, the study in question is an academic outlier, and now even Gates’ own funded research is having a hard time concluding otherwise:
http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/05/13/the-irony-in-new-study-that-bashes-popular-teacher-evaluation-method/
Here’s a direct critique of Chetty et al:
http://www.washingtonpost.com/blogs/answer-sheet/wp/2014/05/13/the-irony-in-new-study-that-bashes-popular-teacher-evaluation-method/
Add to these the latest ASA release, along with the expanding heap of like-minded research, and it’s hard to understand why anyone other than a naif or hack would persist in defending VAM as a useful policy tool. Given this context, one doesn’t need an expertise in Chetty’s econometric algorithms in order to suspect his conclusions:
Click to access PICANG14.pdf
Actually, this thread has absolutely nothing to do with validity and you are the one who opened the door to a discussion of the grants that are funding Chetty’s research. Diversionary tactics won’t work with me so back off.
Victrorino,
First you might note that poster Paul AFT 2063 asked “How much Gates Foundation money does Chetty get?” I googled that for him, and that is why I linked to the vita. I was just answering another poster’s question. Second, in the original post the authors are chastised for not modifying their conclusions. Why modify conclusions that are valid? The only reason to chastise the authors of the paper about their conclusion is if the are not valid, hence my suggestion that folks do the hard work of actually reading the paper and replicating the results.
Many researchers have already demonstrated that VAM is voodoo science.
As a poor person who was not paid enough money to pay my rent this month, I can tell you that the estimated $21 more per week over the course of my 45 year career would not pay even one month’s rent in a year’s time, or help to get me out of poverty, so even if the results were true, they are insignificant to the lives of people in poverty.
And I know personally that the reason why I didn’t have kids is because of my upbringing at home, not because of my 4th grade teacher’s impact on my standardized test scores. But, if that’s what you are looking for, I would not be surprised if confirmation bias played a role in your interpretations. Unlike reformers, I value qualitative research and examining the voiced histories of actual people over number crunching.
Right, TE, it was your response that precipitated this and when a light is shining on the matter of research funding by specious billionaires, people are not going to ignore it.
Try as you might, you are not the thread police here.
The answer to Paul AFT 2063’s question was that Chetty received no grant money from the Gates foundation and apparently no grant funding at all for the paper we are discussing here.
The best way to determine the validity of an empirical research paper is to actually read the paper and duplicate the results. The authors have made this easy for you by providing their data set and STATA code to analyze the data. Here is a link to the STATA code (http://obs.rc.fas.harvard.edu/chetty/vam.ado). Have at it.
TE,
Stop with the nonsense. According to your bar, only a specialist in German or Germany could arrive at a reasonable or credible conclusion about Hitler’s Mein Kampf; only a trained physicist could have anything worthwhile to say about the laws of physics; only a PhD in economics adept at econometrics could comment intelligently and usefully on the larger subject matter in which the econometric method is being deployed. This is balderdash. While no one is an expert in everything, we are all capable of surveying contested landscapes and discerning accordingly. As the folks on this thread have insisted repeatedly – and as you have ignored repeatedly – there are a good many contextual reasons to be wary of Chetty’s study, none of which require us to attempt to “duplicate results.” The ASA report, along with a number of other critiques to which we have referred above, already pinpoint serious (and empirical) problems with relying on the very STATA code you so casually fetishize.
Anyway, explaining the basic contours of rational discourse with a dyed-in-the-wool number cruncher gets boring. When the number cruncher in question displays a distinct inclination toward numbers being sponsored by the likes of the Arnolds and Richardsons, the exercise becomes downright exhausting.
I do think that understanding Mein Kampf would require a reading knowledge of German or having an expert translate it for you. I do think that you need a trained physicist to criticize various aspects of quantum theory. I do think that if you are going to argue an econometric analysis is incorrect you need should actually look at the econometrics and not use an ad homonym argument.
Have you read Chetty’s response to the ASA criticism? How about his response to Alder (Should Chetty have ignored Alder’s concerns because National Education Policy Center is funded by the NEA? I don’t think so, but you might). Here is a link to both responses: http://obs.rc.fas.harvard.edu/chetty/value_added.html
As far as I can tell, the Arnold foundation and the Richardson foundation have nothing to do with this work. Why do you bring them up?
TE,
That’s some interesting wording. And I’m not just talking about “ad homonym.”
I’m referring to “…or having an expert translate it for you”; “you need a trained physicist…”; “…if you are going to argue that econometric analysis is incorrect…” You seem have to conceded my point at all three junctures. For one, I agree that one must rely (at some point) on an expert translator in order to understand the relevance of Mein Kampf. My contention is that one need not translate Mein Kampf itself. I agree that, to some degree, dependence on the work of a trained physicist is necessary in order to discuss the relevance of various laws of physics. My contention is that one need not be a trained physicist to discuss and understand their relevance. I agree that if I was set on deeming a certain econometric method “incorrect” I should go directly to the method in question. Thankfully, my contention regarding said method has little to do with whether it is correct or not (at a technical level) and everything to do with the method’s larger relevance to VAM-based policy. Your apparent failure to wrestle with the larger contextual problems regarding Chetty’s paper(s), and your single-minded focus on technique, is what’s at issue in this thread — and both inclinations reflect, to some extent, what’s at issue with the VAM debate in general.
If the only two studies in play were one sponsored by NEA and one sponsored by the Arnolds and Richardsons, then your latter paragraphs might* stand on surer footing. But the point I’ve been making isn’t that Arnold/Richardson-backed research should be dismissed out of hand, but rather that its funding is possibly relevant when contextualizing its (outlier) confidence in VAM-based policy, especially at a time when virtually every other major study, paper, or association release — extending the gamut from Gates-funded to NEA-funded to unaffiliated — is concluding otherwise.
*For what it’s worth, you presumption about NEA’s relationship to VAM-based policy and corporate reform is also misconceived.
The research on this paper by Chetty did not involve a grant from either the Richardson or Arnold foundation. The Rchardson foundation grants went into research concerning tax policy and the Arnold grant was used for a paper about inter-generational income mobility and geography in the US. You are correct though that the NEA paid for the publication of Alder’s criticisms of the Chetty study.
The criticism of the original post is that the authors stand by the result of their research. That is a criticism of the technique. It would have been helpful if Dr. Ravitch had linked to the public response to the ASA comments given by the authors of the paper.
TE (or rather to my non-troll audience),
Chetty is swimming in corporate cash, and not just from the Arnolds and Richardsons. The JPAL Governance Initiative, and the Poverty Action Lab in which it is housed, is just one of many billionaire-backed programs forever diverting discussion from structural questions of political economy and capitalism to “micro” questions about, say, the behaviors of poor people (see Esther Duflo) or the test scores of students (see Chetty). In the Governance Initiative’s case, one of its major sponsors is none other than…the Hewlett Foundation. Also, mapping grant influence is notoriously difficult. While Chetty’s CV suggests no Arnolds/Richardson cash went to publishing promos for VAM, it also suggests no cash went to publishing promos for VAM in general. This is curious, seeing that Chetty has devoted considerable time and energy to this subject, and he’s received considerable press for doing so. What we do know, however, is that at precisely the same moment Chetty was pushing promos for VAM, he was receiving sizable chunks of his financial support from corporate entities that are quite vocal about their support for VAM.
Anyhow, based on what I’ve read about this TE fella, I’ll leave it at that. No more back-and-forth from me. Cheers.
Devoting serious time and effort to academic studies is what faculty members do. I am sure there were an army of graduate students creating and cleaning the huge data set that was used in the study. That is what graduate students research assistants do.
You think trying to find the best way to get parents to vaccinate their children is a waste of time? Finding how to get the most bed nets distributed and used a waste of time? We have managed to reduce malaria mortality rates by 25% globally in the last 10 years. Seems to me that saving those lives is a worthy endeavor, but I am just an economist.
TE – I have no problem with any of the above. I do have a problem with billionaires sponsoring think tanks and manufacturing celebrity intellectuals who are intent on pushing (faux) “radical” and “progressive” research so as to drown out genuinely radical and progressive study. To “find the best way to get parents to vaccinate their children” while refusing to ask how that question relates to larger distributions (and systems) of power and wealth is not only intellectually bankrupt but morally irresponsible. The same goes for someone who claims to be intent on closing the opportunity gap while refusing to confront political economic restraints that render such a closure impossible. The fact of the matter is, if the Michal Kaleckis of the world were receiving the big bucks and press treatment that your beloved Chetty receives, we’d be a long ways closer to closing the gap. But, of course, it’s that very gap that ensures the money and media treatment leans in the favor of corporate avatars like Chetty.
The above in my post is what Esther Duflo, and the Poverty Action Lab that you talk about in your post that I commented on, does.
Right, TE – and it’s all Duflo et al do. The moment they start asking bigger questions — questions that actually start risking profit losses on the part of their benefactors — they lose their financial support. Your refusal to acknowledge this simple point about the pernicious aspects of corporate philanthropy is quite stunning. My only guess is that you’re either one of the benefactors or one of the beneficiaries. Anyway:
http://www.bostonreview.net/forum/foundations-philanthropy-democracy
Actually I am nether a benefactor or a beneficiary beyond the fact like most who post here my retirement income depends, in part, on investments in capital markets.
Oop. I neglected to include the link re NEA’s relationship to VAM-based policy and corporate ed reform. Here ya go: http://deutsch29.wordpress.com/2013/11/09/nea-aft-common-core-and-vam/
Following your logic, TE, then experienced teachers/educators should be who we listen to about what works in the classroom—not an economist, or a Bill Gates billionaire oligarch who never taught a day in their life and never attended a public school or grew up near or lived in poverty.
If we really want to know about what works in education, we must trust and rely on experienced and successful, certified, public school teachers who have worked with children from all socioeconomic levels over a long period of time—at minimum a decade in the classroom. And I’m not talking about teachers who are labeled successful from the results of student scores on standardized tests.
After all, that’s what they do in Finland, and the results there prove that method works best.
Sometimes it become almost irate that TE sucks so many people into his form if “discussion”. It is a wasye if time. The discussion never resolves. The perpetual posing of question after question is distracting and wastes time on issues most of us aren’t interested in because they don’t carry the discussion forward. In fact they never satisfy TE. He changes every answer to another question and never stops pushing people to explore his links and views. He doesn’t even seem to take a hint or a direct hit from anyone. He just returns with another obsessive question or comment about catchments and total student choice. We get that, TE, and you can do whatever you want. Explore your own issues in your own districts or catchment or whatever you want to frame it. It isn’t relevant most of the time.
What you say is true and if you study the methods of flawed logic in debate/argument, you will discover that slipping in endless questions to divert the topic away from where one side of the argument can’t win is a major method used by people who don’t have legs to stand on. When they can’t win, they change the topic with these questions. When that happens, point out what they are doing and get back on track. If they refuse, ignore them for the fools that they probably are.
Lloyd,
The core question here is if the research in Chetty et al was well done or should they, in Dr. Ravitch’s words “modify their conclusions”. It is the endless ad homonym arguments that are presented against Dr. Chetty that are the distraction.
I think who we should ask depend on the kind of question we are asking. If we want to analyze data that is filled with endogeneity, an econometrician is a good person to ask because they have to deal with this kind of data all the time and have developed some good tools. If you want to ask about how people learn, I would go to cognitive scientists. If you want to ask about best practices in a classroom, I would go to classroom teachers.
It’s ad hominem, TE, not “ad homonym.” And placing one’s argument in the context of larger political economic and philanthropic forces does not an ad hominem make. It’s called critical thinking.
Thanks for the correction. It is not critical thinking to condemn a piece of empirical research because of funding sources at Harvard. Critical thinking is the hard work of examining the research on it’s own merits.
Oh dear, TE. You’re something.
And no, TE, the hard work of understanding the world is not same thing as opening and closing the cabinets. Do us all a favor and read a humanist or philosopher of knowledge for goodness’ sake. Maybe start with anything Habermas. In fact, I recommend all your economist buddies read some Habermas. If reading somebody outside your discipline is too much to ask, at least read Amartya Sen.
I have read a good deal of Sen’s work. Not so much Habermas is a little too continental for my taste. In political philosophy I find contractualism or contractarianism more appealing.
Once I, too, was in graduate school and could remember things that Jurgen Habermas wrote. Now I am a deeply depressed, middle-aged man who knows less with each new moment than I did the last. Behold my sight and tremble!
FLERP – I’ll be there soon enough. In solidarity.
I appreciate your input, rivercityblues.
Apparently, TE did not bother to read the link provided showing all the money Gates has contributed to Harvard to fund research in education. Just because Chetty didn’t list it on his CV doesn’t mean he didn’t get some of that.
TE is a “reform” troll who regularly monopolizes this blog with his holier-than-thou nonsense. The best thing to do is to ignore him.
How much Gates money does Dr. Ravitch get? How much money do right wing opponents to the CCSS give to NYU? Should we be suspicious of Dr. Ravitch’s scholarship because of the folks that donate to the institution? Should we be suspicious of it because of the labor practices of the folks working for NYU?
Disparaging Diane’s integrity is an over the top example of the kinds of questions TE will pose just to keep his inane chatter going.
Personally, I think he has really gone overboard this time and that this alone should warrant his being barred from this blog.
CT,
Did you mean that we should only be concerned about Dr. Chetty’s academic integrity because of grants given to Harvard University or is that a general rule that causes you to be concerned with Dr. Chetty’s academic integrity? I took it to be the latter and applied it in a general fashion.
No one wants to hear the ongoing ridiculous questions that signify your pretzel logic and your incessant need to be the center of attention, TE. Get a clue and STFU!
rivercityblues, I forgot to warn you that TE always likes to have the last word, so this could go on ad infinitum. Previously, Deb accurately pegged the TE ride as a möbius strip loop…
You do know what happens when you cut a möbius strip in half through the center length. Now, cut it a second time. There is a resulting thin strip looped with a circular strip, out there on its own. Sounds like a plan…
Got it, CT. My bad. Anyway…
I scanned his resume and did not see the name “Gates”. This, however, was right on top.
“National Fellow, Hoover Institution, Stanford University”
I would like to remind everyone of how Hoover sent troops against the WWI “Bonus Army”. Barbara Kingsolver recounted this in her novel, THE LACUNA.
(I like historical fiction because it doesn’t ignore empathy for the humans involved, unlike “history”) He sent cavalry, with swords and bayonets to “clear out” veterans.
I have the same problem reading the work/opinions of many economists (except Paul Krugman & Bruce Baker, for example) They seem to have no empathy.
He sent cavalry, with swords and bayonets, to “clear out” veterans.
It is time for teachers and students, including Harvard Education students, to occupy the Harvard Education Department to call attention to the complicity of much of their faculty. They are being paid to legitimate gentrification and the new segregation in many of our urban areas. They are training TFAs to take over charter schools and charter districts.
For a look at what Gates money is doing in Ethiopia and Uganda, check out the current issue of NYRB for the review of Easterly’s new book. Always technocratic solutions that flop.
I hope a Harvard faculty member who is not complicite in the Gates Boondoggle will be more outspoken.
You might be interested in the world that Esther Duflo is doing. Here is a TED talk she gave: http://www.youtube.com/watch?v=0zvrGiPkVcs
I often feel like our schools are being treated as Third World nations by Gates. Same well-intentioned but arrogant and ham-fisted meddling in a realm they think they understand but don’t.
Do Chetty and Co. say that a teacher can raise the earnings of a student they have never taught, but are in the same building? That’s where this whole thing falls apart. Chetty may be talking about when the tests correlate to the teachers, the students, and the class subject being taught. Is Chetty defending the implementation of VAM by the states? He loses all credibility as a learned man, or man of science, if that is the case.
“By the ability to raise test scores.”
Since those test scores (and the educational standards upon which they are based are COMPLETELY INVALID as proven by Noel Wilson in his classic never refuted not rebutted (just the opposite of Chetty, et al.’s work) ““Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 then any conclusions drawn are “vain and illusory”. To use such scores as Chetty, et. al. do is to perpetuate MASSIVE ERROR AND FALSEHOODS. To do so is not only invalid, but illogical and UNETHICAL (as Chetty himself admits when he says that some teachers will be fired because VAM is not valid). And the whole raised earning argument is pure economist’s wet dream ramblings.
Hey, Chetty I invite you to read and refute/rebut Wilson’s argument. What, no you can’t do that, you’re an economist not a rational thinker. You’re wrong Chetty!
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as one dimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
xxxxxxxxxxxxxxxxxxxxx I have 3 things to say about VAM
(1)Anyone privileged (as I was) to be an at-home Mom during their kids’ K-12 ed will have learned that their pre-conceived notions about ‘good’ and ‘bad’ teachers were mere bigotry. On the playground, waiting to pick up our kids, we swiftly learned that the ‘bad’ teacher who played favorites, the ‘good’ teacher who was strict, the dicey teacher who played up to the parents but manipulated the kids– each had students who thrived under their tutelage. Even the aging teacher who purportedly fell asleep during class: this was the gal who transformed my passive 1st-grader into an avid reader!
none of that translates via VAM, folks…
(2)There is no VAM which can properly assess the 8th grade math teacher who saw & understood my IEP-encumbered eldest’s grasp of advanced math concepts despite deplorable testing.. she simply made it her & his goal to trump the testing by hook or by crook– and succeeded.
(3)I’m long past these trials, but as a local chorister I am privy to conversations among today’s teachers. In the autumn, I was appalled at an overheard conversation between local math & science teachers regarding the time and angst required to put together SGO’s– for each student in every class — an assay which required as well some 30pp of backup–all of which had been visited upon them that August!
I taught for 34 years in a small city 9-12th ugh school. We had students fill out short, specific, anonymous evaluations of our courses and teachers. IT WAS NOT USED FOR TEACHER EVALUATIONS. In addition, I asked my students to write me letters about the class and my teaching. Some of them thought I had a tail and pitchfork and some thought I walked on water. Most of them, however, gave me honest, insightful comments that I found very useful. I found the key was letting them know I truly wanted to do a better job. I learned much from their insights, both positive and, ouch!, negative. Those evaluations were helpful to me, but would have been a complete waste as an evaluation tool.
WHOOPS! “high school.”
Why a complete waste as an evaluation tool? It seems to me that the honest students who provided you with valuable feedback could also provide the folks evaluating your performance with valuable feedback.
So am I to understand (based on Chetty et al.) that today’s teachers would have higher earnings if they had had more “effective” teachers themselves? I doubt it.
These Harvard VAMification lovers apparently leave out the amount of college debts students owe to calculate lifetime earnings.