Aaron Pallas is a sociologist at Teachers College, Columbia University, who is one of our nation’s best scholars of education. He is quick to spot Bunkum.
He said this about the report on teacher preparation programs by NCTQ:
“To be sure, few of us relish being put under the microscope. But it’s another matter entirely to be seen via a funhouse mirror. My institution, Teachers College at Columbia University, didn’t receive a summary rating of zero to four stars in the report, but the NCTQ website does rate some features of our teacher-prep programs. I was very gratified to see that our undergraduate elementary and secondary teacher-education programs received four out of four stars for student selectivity. Those programs are really tough to get into—nobody gets admitted. And that’s not hyperbole; the programs don’t exist.
“That’s one of the dangers of rating academic programs based solely on documents such as websites and course syllabi. You might miss something important—like “Does this program exist?”
Pallas noted that the Washington Post published an editorial praising the report. He commented: “I look forward to the Post instructing their restaurant reviewer, Tom Sietsema, to rate restaurants based on their online menus rather than several in-person visits to taste the food.”
Diane Ravitch's blog 
I love the comparison concerning the Washington Post on food vs. education. Why not? Fair for one, fair for all.
Perhaps NCTQ could advertise the best, nonexistent teacher training programs as “everything we saw on our site visits was true.”
These guys are playing to win and using every trick in the book. As you know, we have few friends, and even fewer in high places. The profiteers/privatizers manipulate a compliant or complicit media with ease, get their headlines and move on. But I’m ready for a fight, and I thank you M Schneider for your leadership and courage
You’re welcome, Chuck. Happy to be actively engaged in the fight.
@M. Schneider, much respect for your comprehensive blogposts. I’ve re-shared yours, Pallas’ AACTE’s and many others’ concerns in a collection here: http://bit.ly/nctq_alert. Please connect and keep writing… much appreciated esp. for those just entering the conversation and in search of history/context.
If this is not the biggest pile of …..!
These same EdReformers folks are making life or death decisions for teachers and students. It is not even funny! Scheisters! Probably made lits of $$$! Of course.
You are correct…biggest pile of $&@!
This is indeed a flawed study with several weaknesses, but it is important in that it is taking a stab at being empirical–something that far too often gets chucked for much more loosey-goosey views in the field of education. Rather than throw the baby out with the bath water, perhaps it would be best to ask why Columbia University has the implied website or online syllabus dedicated to undergraduate elementary and secondary education programs. What caused NCTQ to make such a serious error as rating non-existent programs? Having a great syllabus and teacher preparation experiences does not necessarily mean that a program is phenomenal–it’s professors could still be a bit crap. However, a strong syllabus and multiple teacher preparation experiences provide a sort of insurance that the program could be quality (it seems reasonable to believe that skilled professors would be adept at developing syllabi and designing appropriate teacher prep experiences). However, if a program is unorganized–one with weak syllabi or lacking in other teacher preparation experiences–this sends up red flags about the quality of the experience gained at the institution. The professors may be incredible, but if they’re too unorganized in their course syllabus to present anything intelligibly and they don’t provide adequate teacher prep experiences, what quality can be claimed for the program? And if the professors are truly incredible, they should be able to make the adequate changes and be ranked more accurately next time.
If I may correct your first sentence: “This is indeed a FATALLY flawed study with MAJOR weaknesses THAT RENDERS IT COMPLETELY USELESS, ALTHOUGH IT IS EXCEEDINGLY SELF-important in that it is taking a stab at being empirical, IT UTTERLY FAILS–something, ATTEMPTING TO BE EMPIRICAL, that far too often, AND RIGHTLY SO gets chucked for much more SUPPOSEDLY loosey-goosey views in the field of education.”
Empirical studies, by definition have to have a basis in logical thought. The NCTQ “study”, or better said “opinion piece” lacks a basis in logical thought (as shown by many since the opinion piece was released..
For something to be empirical, it needs data taken through measurements and observations in the real world. This is an important definition because it does not necessitate logical thought–although it is genuinely hoped logical thought is involved! NCTQ is logical to analyze the problem of failing teachers and ask why they are failing. NCTQ is reasonable to ask how they are being trained for the field and look at the effectiveness of the training programs–that is, take empirical measurements of different aspects of the training programs. Many universities were very resistant to providing information to NCTQ, so they used the best information available. NCTQ is very straightforward about how they made their study, which prominently and honestly displays their weaknesses. They also say that they plan to publish annually, allowing them to improve their measuring system; allowing schools with poor ratings to contact NCTQ and point out the errors in the poor rating; and helping programs turn their focus inward to make some needed changes to earn a better rating next year. While flawed, I look forward to seeing the methodology improvements NCQT makes next year. I am also eager to see the improvements many universities make to better and more effectively organize their teacher training programs.
EG, (not quite sure how to interpret your moniker as it’s an interesting play on words and word usage)
In any endeavor using “empirical data” the question at the forefront should be “Who determines the data?” Now one can believe that the “data” is “out there” to be discovered or one might say that the data is determined by the human doing the “discovering” by the type and scope of questions being asked.
“. . . that is, take empirical measurements of different aspects of the training programs.”
What you consider to be “empirical measurements” are not and cannot be the “complete story” in evaluating the teacher prep programs as by definition teacher prep involves many aspects of human experience that are not amenable to anywhere near accurate “measurement”. What is left out of this so-called measurement says as much if not more about the intentions and/or assumptions of the promulgators than what is included as per your statement: “to make some needed changes to earn a better rating next year.” The assumption is that a better rating is good and desired, a better rating on an admittedly flawed process, eh, and by a group that admittedly has a very political agenda from its inception as there has been various processes in place to judge those programs through a more thorough vetting (NCATE for example).
Data, especially data stewed in ideology, is not the end all be all, in dealing with human interactions such as the teaching and learning process. Some do worship at the altar of data, but what they worship is a paucity of the information needed to more accurately assess teacher prep programs. For a quick read on the perils of data worship see: http://schoolfinance101.wordpress.com/2013/04/20/the-perils-of-economic-thinking-about-human-behavior/
BROADIES GONE WILD!
Reblogged this on David R. Taylor-Thoughts on Texas Education.
As I understand from one of Diane’s previous comments on another post, she is proud of the schools which refused to supply data to the NCTQ, a situation which – following the restaurant metaphor – would be like refusing to serve food to the restaurant reviewer because you have already decided that you are not going to like his review… and then complaining when he has to do the review based on the menu he finds online. If schools refused to supply data, fine, that is their choice – but it is hypocritical to then complain about the lack of data in the resulting report.
If I am not mistaken, schools refused because they disagreed with methodology behind the rating procedure. Would you want to help be rated by procedures that are flawed?
Vinh, I know this is a bit off-topic but, no, no one wants to “be rated
by procedures that are flawed”–that’s exactly what’s happening to teachers all over the country! (And now the teacher prep. programs.)
Vinh: if I may, let me flesh out a little what I presume you are talking about.
1), the NCTQ opted for an evaluation that required very little data, evaluation and verification. For example, with the resources they could have called on they would have been able to send teams of acknowledged experts in various fields to hundreds of physical locations, talk to thousands of individuals in the relevant institutions, and interview many thousands more graduates of those institutions. They opted instead for cheaper and faster. If you are serious about producing quality research that leads to trustworthy inferences, you don’t hollow out an alleged serious research effort by starving it of time, effort, expertise and funding.
2), Lacking the above, they should have—in the interests of fairness, accuracy and ethical conduct as researchers and advocates—simply have refused to publish their results.
IMHO, the only way to look at the report the NCTQ came up with was that they simply gathered together whatever low-hanging data fruit was easily at hand that bolstered their preconceived notions. Hence, they now find themselves in the position of praising an institution for non-existent programs.
There are “no excuses” to justify what can most charitably be called pseudo-research in support of edupreneurial $tudent $ucce$$.
In a comment on another of today’s postings, Robert D. Shepherd invoked a well-known American writer and satirist. He comes in handy when considering the NCTQ’s baseless report:
“Prejudice—a vagrant opinion without visible means of support.” [Ambrose Bierce]
🙂
It’s interesting to note that NCTQ rated teacher preparation universities in Texas based on the Common Core State Standards. However, Texas institutions, especially state funded institutions, don’t prepare teachers based on the Common Core, they prepare teachers based on the TEKS—that is, the Texas Essential Knowledge and Skills. Texas doesn’t even participate in the Common Core State Standards, and with that said, how can a report be considered credible when it fails to correctly measure content taught at teacher programs that do exist? — unlike evaluating a university’s program that doesn’t exist (i.e., Teachers College at Columbia University)
The real question is whether alternative teacher preparation programs like Teach for America and The New Teacher Project, that last for three or four weeks in the summer, are undermining efforts to improve teacher preparation programs in the United States? After all, the Advisory Board of NCTQ is comprised of TFA supporters… namely TFA’s CEO Wendy Kopp. Maybe the overarching question should be — why pay any attention at all to Washington based reform efforts toward school improvement?
Reblogged this on PHIGURITOWT and commented:
I think it’s interesting to note that NCTQ rated teacher preparation universities in Texas based on the Common Core State Standards. However, Texas institutions, especially state funded institutions, don’t prepare teachers based on the Common Core, they prepare teachers based on the TEKS—that is, the Texas Essential Knowledge and Skills. Texas doesn’t even participate in the Common Core State Standards, and with that said, how can a report be considered credible when it fails to correctly measure content taught at teacher programs that do exist? — unlike evaluating a university’s program that doesn’t even exist (i.e., Teachers College at Columbia University)
The real question is whether alternative teacher preparation programs like Teach for America and The New Teacher Project, that last for three or four weeks in the summer, are undermining efforts to improve teacher preparation programs in the United States? After all, the Advisory Board of NCTQ is comprised of TFA supporters… namely TFA’s CEO Wendy Kopp. Overall maybe the overarching question should be — why pay any attention at all to Washington based reform efforts toward school improvement?
Wouldn’t it be wonderful if schools and colleges of education had banded together…under the leadership of AACTE? CAEP? someone? to respond in a united way to NCTQ. That would require that all acknowledged the truth– that the NCTQ “study” design is ridiculously flawed, the rankings (whether 4 star or 0 star) are without merit for bragging, dividing, or defending. Of course, that did not happen and we continue to be divided as we are in defending K-12 public education’s value to our country. We have organizations that are doing the work we should do–designing curriculum, PD, and assessments, while they relay to their members the messages of DC power-brokers, corporate edu-philanthropists and tell us what is feasible rather than asking what is possible and good. While these organizations, abetted by our state commissioners, are busy relaying messages from the powerful instead of representing our voices and accumulated areas of solid research, the pseudo-science wins. Of course, the organizations protect their titles and turf and CAEP is after all an “accrediting” organization and only wants to protect its status while the programs it accredits are trashed. .
Teacher education programs and faculty as well as our “representatives” were largely missing when our partnership might have benefited K-12. We were largely silent about the gutting of K-12 that continues and by that silence we have lost our most potent allies. Following the lead of the more powerful unions in K-12, these appeasements will only lead to our increasing marginalization no matter how much we pay AACTE and CAEP, they have already ceded key organizations and the media war to NCTQ.