Matthew Di Carlo of the Albert Shanker Institute of the American Federation of Teachers is neither pro-TFA nor anti-TFA.
Here he reviews the latest study of TFA by Mathematica.
It has been widely reported that the study found little or no difference between the test scores of the students taught by TFA and by regular teachers. TFA saw that as a victory, since it presumably showed that no training or experience was needed to achieve the same results. Others saw it as a repudiation of TFA’s oft-repeated claims that their recruits were superior to career teachers.
Di Carlo parses the results and reaches this conclusion:
Now, on the one hand, it’s absolutely fair to use the results of this and previous TFA evaluations to suggest that we may have something to learn from TFA training and recruitment (e.g., Dobbie 2011). Like all new teachers, TFA recruits struggle at first, but they do seem to perform as well as or better than other teachers, many of whom have had considerably more experience and formal training.
On the other hand, as I’ve discussed before, there is also, perhaps, an implication here regarding the “type” of person we are trying to recruit into teaching. Consider that TFA recruits are the very epitome of the hard-charging, high-achieving young folks that many advocates are desperate to attract to the profession. To be clear, it is a great thing any time talented, ambitous, service-oriented young people choose teaching, and I personally think TFA deserves credit for bringing them in. Yet, no matter how you cut it, they are, at best, only modestly more effective (in raising math and reading test scores) than non-TFA teachers.
This reflects the fact that identifying good teachers based on pre-service characteristics is extraordinarily difficult, and the best teachers are very often not those who attended the most selective colleges or scored highly on their SATs. And yet so much of our education reform debate is about overhauling long-standing human resource policies largely to attract these high-flying young people. It follows, then, that perhaps we should be very careful not to fixate too much on an unsupported idea of the “type” of person we want to attract and what they are looking for, and instead pay a little more attention to investigating alternative observable characteristics that may prove more useful, and identifying employment conditions and work environments that maximize retention of effective teachers who are already in the classroom.
For me, the problem with all such studies is the assumption that the best (perhaps the only) way to identify the best teachers is by comparing changes in test scores. Great teachers supposedly get higher scores than mediocre teachers. I think that places far too much faith in standardized testing and in the assumption that education is solely measured by those tests. It makes the tests the arbiters of all things, even though most teachers do not teach tested subjects. Test-based findings are even more suspect when the children are very young.
Diane Ravitch's blog 
I will consider believing that TfA teachers are equal to or better than traditionally trained teachers when the elite private schools that Gates et al. send their kids to start clamoring for TfA recruits.
LikeLike
This is why numbers that presume to describe education, no matter how much data was collected and processed to produce them, never tell enough of the story to be the primary or even secondary driver of policy. When that which is measured is what is managed excludes other critical information, the measurement/management system bears sole responsibility for it’s own failure to produce improvement. Exactly as Deming told us decades ago. His insights remain just as relevant today as they have always been.
LikeLike
I do well on standardized tests. But that doesn’t mean I’m a better teacher.
Being a good teacher to me means that every day I am thinking about ways to better my instruction. Finding out what works and what doesn’t work takes time and experience. And sometimes what works for one class doesn’t necessarily work for another.
As much as I am proud of my test scores and grades, I now realize that they are not a complete picture of who I am. And to rely on them as the sole measure of my students is foolish.
LikeLike
” Finding out what works and what doesn’t work takes time and experience. And sometimes what works for one class doesn’t necessarily work for another.” Exactly so.
TFA is based on the assumption that subject preparation is the most critical factor in teaching well, and that the rest is just easy–learning the tricks of the trade in teaching.
The rush to hire TFA people is based on the same theory that the best athletes in a sport make the best coaches, and that skills learned in playing as an adult, and in coaching adults, are easily modified or transferred to children. That is not the case. Some great coaches were mediocre players. Some great players have been mediocre coaches. And there is Arne Duncan, who was good enough to play some pro ball and earn a degree in sociology but remains clueless about what matters in education.
Teaching students is often like getting a caterpillar to figure out how to coordinate all those limbs to move in one direction, and at the right pace between the front and back to avoid being in a collision with your rear. Any skilled performance is extraordinarily difficult to deconstruct and parse and re-conceptualize into a “teachable form,” especially when the aim goes beyond training for a specific task.
LikeLike
Using student score changes to come up with a teacher rating is ‘GIGO (garbage in/garbage out)
LikeLike
When I was young, I wanted to teach both English and Biology.
I scored very well on my SAT (best in my class at a well-regarded Catholic high school where I attended with the current governor of my state). My grades were “good” — I usually had second honors.
I never even applied to Brown University here in RI. I assumed we didn’t have the money. Even after receiving a $20,000 scholarship in exchange for promising to teach for eight years after I graduated from college — which in those days was enough to pay for nearly all four years at the local state school — I figured we just didn’t have the money. In retrospect, this was almost certainly a mistake.
I attended Rhode Island College, where a guidance counselor told me early on that it was “impossible” to get certified in English and Biology and graduate in four years. I initially chose Biology, and later switched majors to English (in part because ALL of the science courses were only offered at what seemed to be ungodly hours to my seventeen year old self).
Years later, I learned that the RIC guidance counselor had been… technically correct. But students at Brown, who were able to create their own majors, were in fact able to get all the coursework in that was needed to graduate with teaching certificates in both English and Biology.
I’ve met a small number with those exact certifications.
Maybe if I had been able to take fewer education courses — which in my view were mostly worthless — I’d have been able to squeeze in enough science to get my dual certification.
Maybe I’d be better off now. Maybe my students would be too.
It’s hard for me to defend the bloat of ed. courses I was required to take all those years ago.
LikeLike
All teachers struggle at the beginning, including those who’ve gone through formal licensing programs. The difference between them and TFA temps is that they, at least initially, intend to stay in the classroom, rather than have a cup of coffee while padding their resumes and gushing about their awesomeness.
Even if the results of this study are accurate, it seems to avoid the fact that teachers keep growing and developing after their fledgling years, unless they are TFA, the overwhelming majority of whom leave. So, even granting the (questionable, in my mind) assertion that TFAers have results equal to those of formally trained teachers, there’s no mention of the costs, social and economic, of such intentional churning and high turnover.
Wendy Kopp has explicitly stated that the purpose of TFA Is not to supply teachers to classrooms, but to identify, groom and train “leaders” (aka privatization cadre) in education.
Given that the widely-lauded founder of this organization says that developing teachers is not the primary purpose of this organization, and that the overwhelming majority of them leave the classroom, isn’t it mostly beside the point to be arguing about their effectiveness?
LikeLike
These results are only valid if you believe that test scores accurately reflect what is learned in the classroom.
Has anyone ever established that this is true?
This could be answered if the test score distributions were analyzed while controlling for the classroom mark.
To my admittedly limited knowledge, this was done at LAUSD (but never published by anyone that I know of) when someone complained that their valedictorian child had not passed the CAHSEE (the high school “exit” exam). A simple analysis showed that the distribution for fifth graders was the same whether the student was getting a 4 (highest possible mark) or a 1 (lowest) for the 2009 administration of the ELA California Standards Test. The same phenomenon was observed in math scores. The total sample was over 50,000 kids distributed over 500 schools throughout LAUSD.
The same was observed to happen with students getting As in math and English for middle and high school students. This was not observed for students getting Bs, Cs, Ds, and Fs. Nevertheless, these distributions showed unexpected numbers of “advanced” students contrary to expectations. This could be because non-A students do not take the job as seriously and tend to simply bubble up instead of making a conscious effort “to do well” in the tests.
Any person dealing with data will tell you that if the distribution is similar for students who have radically different classroom achievement levels across a cohort this big and coming from so many different schools, then it is highly likely that what goes on in the classroom has not much to do with how the test is designed. Therefore, the standardized tests these students took are useless in determining the influence of the teacher effectiveness on the test scores. (Of course, if the teacher doesn’t teach at all, the scores will be theoretically close to zero, but those “come out in the wash.”)
Given that, the results of this study are not surprising to me at all and indicate that the test scores do not depend on the teacher’s effectiveness. At least, that is what was observed at LAUSD for the 2008-09 administration of the CSTs. But I suspect the same could be do elsewhere and for other tests.
LikeLike
If you torture your data long enough, it will confess to anything.
LikeLike
“Like all new teachers, TFA recruits struggle at first, but they do seem to perform as well as, or better
than other teachers, many of whom have had considerably more experience and formal training.”
Clearly suggests to me that M. Di Carlo is pro-TFA
LikeLike
I agree that his bias shows, particularly in the preceding statement
“it’s absolutely fair to use the results of this and previous TFA evaluations to suggest that we may have something to learn from TFA training and recruitment”
no, it’s not fair or even valid to simply assume with no proof that the quality of a teacher can be gauged by student scores — and on a 5 minute test in this case!
But it is “ridiculous” — which also happens to be what this entire “TFA study” is.
It’s a joke.
LikeLike
Jb2, since most are certain to leave, who needs them?
LikeLike
“For me, the problem with all such studies is the assumption that the best (perhaps the only) way to identify the best teachers is by comparing changes in test scores.”
This assumption is at the heart of this discussion. It is beyond insulting to students that their worth be determined by a high stakes test, especially one that has been designed to be difficult enough that a majority fail. If the authorities feel the need to spread scores through artificially high cut scores, then the purpose is obviously for ranking, not for examining knowledge or learning. It is beyond insulting to teachers that their profession is reduced to producing good test takers.
LikeLike