I have often criticized economist Raj Chetty for his study claiming that teachers in elementary school who “produce” test score gains affect the lifetime earnings of their students. This study, which he conducted with two other economists, supported Arne Duncan’s obsession with measuring teacher effectiveness by test scores of their student, a practice that was repeatedly debunked by scholarly organizations and caused the unjust firing of thousands of teachers and principals.
But he has a new study that makes sense, as
Diane Ravitch's blog 
The idea that public colleges are engines of social mobility hardly seems profound, but after the Chetty picking in his VAM “study”, I would be skeptical of anything and everything Chetty produces.
“Chetty Pie”
To bake a Chetty pie
You pick the ripest cherries
Enticing folks to buy
The Chetty data queries
The problem with quoting Chetty’s work when it seems to support reasonable ideas is that it lends him undeserved credibility that he can cash in later on another of his goofy claims.
By all rights, Chetty should have been laughed right out of the academy for his VAM “study” and I suspect he would have been if his field of economics were not such an unscientific joke.
The same way, you don’t want to throw out somebody’s research just because he did junk in the past.
But yeah, we always need to be critical, and to avoid the problem you mention, it’s a good idea to mention the junk study when mentioning Chetty.
His VAM study showed what he is capable of
And what he is capable of is not an unbiased analysis.
Errors are excusable, but chetty-picking is not, in my opinion.
I don’t know why anyone would tolerate that in an undergrad lab write-up to say nothing of what is supposed to be scholarly work.
I know it happens all the time in disciplines like economics, which is one of the many reasons why economics isn’t a science.
Lest anyone think I exaggerate how much economists cherry pick, I suggest they read what William Black had to say
http://neweconomicperspectives.org/2013/10/economics-science-economists-scientists.html
Fraud is the massive, giant rotten cherry killing the tree but economists simply pick around it as if it were not even there
And lest anyone think I exaggerate Chetty’s own Chetty’s picking, read a summary of Moshe Adler’s critique.
http://nepc.colorado.edu/blog/moshe-adler-chetty
An incomplete post? Seems truncated, no link?
Try this: “Stanford research analyzes colleges as engines of upward mobility”
http://news.stanford.edu/2017/01/18/stanford-research-analyzes-colleges-engines-upward-mobility/
and also the link within that to the interactive tool in the NY Times:
Thanks!
Thanks for the alterntive route to the study. I do no subscribe to many of the publications that Diane includes in her posts.
Arrogant academicians may have noted a shift in the winds.
It’s it interesting how many of these people (academics included) have suddenly found a social conscience of late?
Chetty was Obama’s go to academic for VAM (Obama even mentioned Chetty’s work in a state of the union address) and provided critical testimony in the junk anti-tenure Vergara case.
As Chetty’s collaborator Friedman put it, better to fire those “ineffective” teachers sooner rather than later.
Why people pay any attention to these two is a mystery wrapped up in an enigma.
Get “attention”, for the same reason that selective papers, within the non-peer reviewed NBER, get noticed for policy implementation, if they legitimize wealth accumulation by the rich. If the research has value for the middle class and poor- not so much.
The problem with this study is that they start with the assumption that value add has real value, but it has been proven to have little authentic validity. Firing teachers over a bogus algorithm makes no sense, especially when fewer young people find education an appealing career. By the way when the presumed impact is broken down on a per student basis the presumed financial gain is about $250 per student per year. Raising wages from $10 per hour to $14, increases a worker’s income by about $9,000 per year. If employers are interested in raising test scores, they should pay workers more, because family income correlates to higher test scores. This will have more impact than charters or vouchers that provide meager gains at best while vouchers provide students with a decline in scores. This will provide families with greater stability, as instability is one of the more disturbing byproducts of poverty. An added bonus is that fewer students will arrive at school hungry, and more will be ready to learn.
Also, it just presents the gain as an average, which means that those working wage-stagnant, low paying jobs are not going to see any increase. To my knowledge, the identity and nature of the mechanism that actually produced the gains was not elaborated upon. Was it superior negotiating skills? A employer checklist of some kind where higher grades meant higher pay? Ouija board? My own BOE (back of the envelope) calculations came in at around $3.90 per week or about $200 per year. I would also take a hard look at any adjustments Chetty made for socioeconomic status, as from all I’ve seen, those are highly subjective.
And well funded public education is the magic economic engine of America. The benefits of an atmosphere of equity, where minds thrive, temporarily freed from the endless socio-economic pecking order. Exactly the magic vouchers will erase.
The Chetty. et. al., study uses a widely accepted Mobility Index, and “big data” statistical models – the results are stunning … maybe high schools feeding in public universities (especially in New York City!!) are educating far better than their critics claim. I blogged with links to the Study below:
According to a press release from Stanford, this Chetty study defines a college’s upward mobility rate as the fraction of its students who come from families in the bottom fifth of the income distribution and end up in the top fifth in their early 30s. Specifically, it takes into account both the number of low-income students attending a post secondary school and the “success” rate at which that cohort of students later moves into the top 20 percent of wage earners.
Those earning more than $58,000 in adjusted incomes for 2014 are in that top bracket.
The problem is that wages at age 30 are related to occupational choices and job availabilities, not just education. Average pay for the same occupation may also vary by geographic locale and demand (in addition to seniority in the position). In 2015, for example, the average salary for an elementary teacher was $57,730, for middle school $48,360, for high school $60,440. What are we to make of these differences?
The study also ignores the well-established fact that the studies undertaken for a college degree are not necessarily chosen because they promise a higher than average income (upper twenty percent). Nor is completion of a college degree the only path to economic and social mobility (both too easily equated with “success” in life).
I am especially aware of the role of occupations and earnings and sources of income among students who complete collegiate studies in the arts. Many enter college with no assurance they will earn a salary.Some have no interest in getting a job as a salaried artist.
Studies of occupations in the arts, by college completion, and average income have been produced from time to time by the National Endowment for the Arts. These studies show that very few artists who have college degrees also work full time in their field of study. Many work for less than a full year, and many hold two jobs. Among the lowest in pay are actors and writers. Here is the latest (and dated) NEA study that seems to be relevant. https://www.arts.gov/sites/default/files/105.pdf
Chetty’s study seems to assume that upward social mobility depends almost exclusively to education that will lead to occupations of a certain kind–those with some prospect of yielding a higher than average income before the age of 30. I could not find data in the study that looked at the influence of labor markets on job availability and average income.
Accord to the Stanford press release, Chetty seem to think that public colleges and universities “could provide the most scalable educational model for increasing upward mobility.”
I am not sure what that comment that means. Public colleges and universities were founded in the 1800s in the belief that accessible and affordable education could and should be provided to many, that is, “at scale.” Has Chetty no understanding of the history of land grant universities and public investments in higher education, including community colleges?
From my point of view there is no need to reinvent “a model” that seems to be working quite well.“ There is an urgent need to preserve and protect the existing models (plural) of affordable post-secondary public education. There is also an immediate need to stop the budgets cuts for public education, including post-secondary scholarships and mentoring programs for students from low-income families. It would help to have an administration where educators were not regarded as “flush with cash” and teaching nothing, to paraphrase Trump’s inaugural address. It would help to have an administration that valued scientific inquiry, the good works of people versed in the arts and humanities, and had some respect for the offices to which they were elected.
If you want to view the whole set of projects of which Chetty’s study is one part, go to http://www.equality-of-opportunity.org. The maps and graphs are really worth a look and if you like data, the details for this study are also there.
I think this is interesting because it represents a shift in focus on the purpose of education in the bigger picture, which is, in part, to promote social mobility from one generation to the next. It also acknowledges higher education as a part of that mobility.
California Department of Education has released a new set of accountability tools called the “California Dashboard.” It includes the usual standardized test scores on math and ELA proficiencies, but attempts to improve things by including other measures.
As I read through all of this, I can’t help but think, as a college-educated parent, that this is only superficially interesting to me in raising my own kids. A bigger goal for me is to help position my kids to pursue an education after HS graduation — like college. As such, I am supplementing and mediating public school programs with additional extra-curricular experiences and resources.
I find that local school administrators are obsessing over the new California dashboard. It makes me think of an analogy that is critical of this approach.
When you drive a car to get to your destination, you look to the road ahead. But you don’t obsessively stare at the dashboard to get there.
Social mobility is on the decline in this country. Overall, those of us in the 65+ group did better than our parents, but our children are doing worse due to the global economy, the demise of unions and expansion of technology. While there are exceptions to this, the general trend is downward.https://www.ft.com/content/7de9165e-c3d2-11e6-9bca-2b93a6856354
If Gates is allowed to digitalize education for the kids of the middle class and poor, unlike his own kids, then, the coerced technological replacement of education jobs, is a symptom of oligarchy and colonialism.
Charts in the studies at http://www.equality-of-opportunity.org. suggest that social mobility rates ( as defined by Chetty ) have not improved since the Great Depression.
“When you drive a car to get to your destination, you look to the road ahead. But you don’t obsessively stare at the dashboard to get there.”
Nor the rear view mirrors!
Some of these people have been looking at a rear view all their lives.
That’s how they get ahead.
And in that mirror are their ancestors who procured their get ahead wealth, eh!
Duane,
I was not talking about the “rear view” in the mirror.
Some objects are not closer than they appear, thank goodness.