Linda Darling-Hammond of Stanford University wrote a critique of the National Council on Teacher Quality report on teacher preparation, and the NCTQ responded to her critique.

Here Linda Darling-Hammond responds to NCTQ:

In my blog about the recent NCTQ teacher preparation report, I identified errors in their program reviews — a few examples of the many dozens I have heard about.  NCTQ has responded with their rationale for the ratings, claiming that, in “fact,” they got it right.  Below are the real facts about these errors. 

I want to preface this reply, however, with two points on which I agree with NCTQ: First, while I have seen many strong teacher education programs, there are many others that are very weak and need major improvements.  Second, the areas that NCTQ rated — selection, content knowledge, pedagogical knowledge, and strong connections to clinical training  — are important areas of focus.  I am pleased that attention will be directed to these critical aspects of preparation.

Two things, however, are unfortunate:  The NCTQ indicators of these standards do not provide an accurate picture of the candidates actually prepared or the opportunities actually offered by these programs, and they provide no information about the outcomes of programs — what candidates actually learn and can do. In addition, based on the inaccuracies that are surfacing for most programs, the data collection was conducted very poorly.  And it is unfortunate that NCTQ and U.S. News and World Report would publish ratings without even checking the data. 

NCTQ did concede that they made an error in their ratings of the Stanford courses I mentioned and has noted that it invites programs to respond with corrections to their data.  But there is no published plan to correct the ratings, and I fear that programs may have too little confidence in the NCTQ methods to take steps to engage with them further. 

Fair warning:  Reading the details below could tire some readers who are not deeply interested in teacher preparation.  But to actually understand what has been rated and what the truth is requires a commitment to dive more deeply into program design than NCTQ apparently has been willing to do.

Columbia College’s Alleged Teacher Education Programs

The Claim as stated by NCTQ:  “The Review is so badly done that NCTQ asserts that there is an undergraduate teacher preparation program at Columbia when there is none.”

NCTQ statement of “Fact”: “Actually there is: the Urban Teaching Track Childhood Education program at Columbia College.”

The Reality:  There is indeed no such program at Columbia College.  Originally NCTQ rated TWO undergraduate programs at Teachers College, the independent graduate school at Columbia University — one in elementary education and one in secondary education — that do not exist.   (Teachers College offers no undergraduate programs and is not connected to Columbia College.)  The president and provost at Teachers College were mystified about this attribution and went on a hunt for this program.  They could not find one at Columbia College or at Teachers College. It turns out there is a program by this name at Barnard College, which students can take as an undergraduate minor.  Barnard College is not affiliated with Teachers College.  NCTQ appears not to know what institution it is even reviewing when it gives these ratings.  (For some unknown reason, NCTQ did not rate the program content, only its “selectivity.”)

UC – Santa Barbara’s Alleged Failure to Offer Critical Teaching Courses

The Claim as stated by NCTQ:  “We got U.C. Santa Barbara’s ratings wrong because we missed the elementary math courses, English Language Learners courses and a year-long student teaching program.” (In addition, I had noted that NCTQ entirely missed the UC-Santa Barbara secondary education program, which they do not address in their rebuttal.  The truth is that they did entirely miss that program.) 

NCTQ Statement of “Fact:” “We didn’t miss these courses or the student teaching program at all. We looked at each one and each one failed our standards. That explains their low scores, not sloppy errors on our part.”

The Reality: The evidence shows that NCTQ’s raters did in fact either miss the content of these courses or rate them erroneously.  Their ratings are not plausible when the details of the program are known.

1) On English Learners, NCTQ review said: The program fails to meet the standard because there is no required course that delivers instructional strategies addressing the specific early reading needs of English language learners and requires candidates to practice such strategies.  This is false.  There are multiple courses (6 in all) that treat these strategies.

• At UC-Santa Barbara, Candidates begin in August with a 2 unit course on “Foundations of Academic Language” that prepares them for the Reading/Language Arts (2 quarters) and English Language Development (ELD)/SDAIE (an approach to teaching English learners in content areas) course series (3 quarters).   In addition, they have a course in “Culture and Language in Teaching and Learning” that also addresses teaching reading for ELs. All course assignments are linked to student teaching experiences, and require some form of assessment, teaching, or other activity with the candidates’ K-12 students. Candidates are only placed in partner schools that serve a diverse student body that includes children with linguistic diversity. The program requires that Candidates must have opportunities to teach English Learners, and this requirement is stated in the application that schools use to apply as a partner.  Each reading/Language Arts assignment requires attention to learners in the classroom (which will include ELs) and the Literacy Assessment assignment requires a series of assessments with a student struggling with reading, generally an English Learner. Incorporation of Academic Language and ELD standards are a required component of the Lesson Design Template that all candidates in the program must use. All elementary reading/language arts lessons, and lesson plan assignments require consideration, assessment, and specific strategies for English Learners. The reading courses are integrated and articulated with the year-long three-course ELD/SDAIE series.

2) On Elementary Mathematics, NCTQ review said: The institution does not meet this standard because it requires that teacher candidates take little or no coursework designed to develop their conceptual understanding of elementary mathematics topics. It thus fails to ensure that all essential topics are adequately covered, regardless of the design of the instruction. This is false. There are 2 courses that do precisely this, plus another course in mathematics methods:

• Two math courses are required of elementary candidates prior to taking their elementary mathematics methods course. The syllabi for these courses shows that they are focused on concept attainment both for candidates and for understanding how children think about these topics.  The mathematics methods course builds on this conceptual understanding to enable candidates to learn to teach these concepts to children.

3) On Student Teaching, NCTQ review gave the program said: While the program provides student teachers with sufficient feedback it fails to meet this standard because it does not clearly communicate to school districts the desired characteristics of cooperating teachers, and fails to assert its critical role in the selection of cooperating teachers.  This is false. The partnership agreement between the university and school districts outlines roles and responsibilities of university and school-site personnel and the characteristics of cooperating teachers.  

• The agreement makes it clear that UCSB-funded on-site coordinators and supervisors are involved in the selection of cooperating teachers and that such teachers must be able to model and develop the instructional strategies reflected in the California Teaching Performance Expectations, as well as planning with the teaching candidates weekly, sharing curriculum materials, and allowing candidates to explore approaches to teaching and learning found in the Common Core Standards and the Next Generation Science Standards.   They must also teach diverse classrooms that include English learners. 

4)  On Struggling Readers,  the NCTQ review said: The program fails to meet the standard because there is no required reading course that delivers instructional strategies necessary for teaching struggling readers and requires candidates to practice such strategies. This is false.  There are two required courses in elementary reading/ language arts, both of which specifically treat the teaching of English learners, students with reading disabilities, and others who experience other difficulties in reading, and both of which are linked directly to clinical experiences that require candidates to practice these strategies. 

• The courses treat the Common Core State Standards in ELA, stages of reading development, and specific strategies to teach oral language development, word identification, phonological awareness, phonics, structural and contextual analysis of words, comprehension strategies, as well as strategies for reading different kinds of texts.  Candidates study the California Content Standards for CAPA (California Alternate Performance Assessment) used with special education students and they design accommodations and modifications for students in their classes who have special needs.  When they complete the Performance Assessment for California Teachers, candidates must also design and teach lessons suitable for students who are English learners as well as those with disabilities and be evaluated on their teaching.

Cal State – Chico’s alleged Failure to offer hands-on learning opportunities

The Claim as stated by NCTQ: “An implication is made that our rating of Cal State University at Chico is wrong because we missed their great ‘hands-on’ instruction at its learning lab.”   (What I said was that: California State University at Chico was rated poorly for presumably lacking “hands-on” instruction, even though it is well-known in the state for its hands-on learning lab and requires more than 500 hours of clinical training during its full year of graduate level preparation.)

NCTQ Statement of “Fact:” While CSU-Chico’s learning lab may be fabulous, it is immaterial. All we know is that Chico does not give student teachers adequate feedback or require that student teachers are assigned to classroom teachers who are effective.

The Reality: 

• With respect to feedback for student teachers: In its extensive student teaching program, Chico links feedback to candidates to California’s thirteen Teacher Performance Expectations (TPEs). Formative assessments include formal  classroom teaching observations over the course of both practicum experiences conducted by university supervisors. Feedback is guided by detailed rubrics.  Instructors and university supervisors guide and coach candidates in the completion of formative assessments that prepare them for the teaching performance assessment and provide them with timely feedback. At the midpoint and end of each practicum semester, the candidate, the cooperating teacher and the supervisor engage in a three-way discussion to evaluate the candidate’s progress in addressing the TPEs. This discussion results in the completion of a Teaching Practicum Evaluation Form. Candidates also self-evaluate, and all three individuals participate in a final evaluation. In the event that a candidate is not successfully demonstrating competency on one or more TPE at any given point during the semester, an Improvement Plan is implemented. The Improvement Plan details specific areas of concern and recommends specific actions that need to be successfully completed. With support and guidance, the candidate is given additional opportunities to demonstrate success. At the end of the student teaching semester candidates, their cooperating teachers, and their university supervisor participate in an evaluation of the candidate’s strengths, growth needs, and growth goals. They use this information to develop an implementation  plan that is then carried forward to their support provider in their induction program during their first two years of teaching.

• With respect to selecting cooperating teachers: To qualify, a cooperating teacher must hold the appropriate credential (including authorization to teach English learners), have three or more years of experience teaching in California, teach in a diverse school, and be deemed capable of effectively guiding a beginning teacher by both university and site personnel.  Experienced university supervisors provide input during the selection process, based on their own evaluation of teachers, along with site administrators who must recommend that a teacher is able to successfully guide the learning of a credential candidate. Both university supervisors and administrators make candid input about cooperating teachers in a data base that is maintained to guide placements. Cooperating teachers are removed from the data base when concerns are raised about their effectiveness. 

Conclusion:  NCTQ’s methods may have been incapable of finding these readily available data, but that does not mean they do not exist.  Even more important is evidence that candidates in fact are able to teach when they reach the classroom.  We need more accurate and comprehensive methods for evaluating programs so that we can properly guide the improvements that are necessary.  The National Research Council will soon issue a report on more productive methods for evaluating teacher education.  I hope their findings will be the focus of as much attention by the media and the field as these.

Some in the mainstream media have given credence to the report of the National Council on Teacher Quality about teacher preparation. They should have talked to some of the institutions that were graded before venturing an opinion.

In this post, Dean Michael Feuer of the George Washington University School of Education explains how far wrong the report was about his school, its programs, and its students.

Linda Darling-Hammond of Stanford University is one of the nation’s leading experts on teacher preparation. This is her commentary on the report released by the National Council on Teacher Quality, which attempts to rate the quality of the nation’s colleges of education by reviewing their catalogues and course syllabi.

What Can We Learn about Teacher Education Quality
from the NCTQ Report on Teacher Prep?

Linda Darling-Hammond

This week, the National Council on Teacher Quality (NCTQ) issued a report titled: NCTQ Teacher Prep Review. Billed as a consumer’s guide, the report rates programs on a list of criteria ranging from selection and content preparation to coursework and student teaching aimed at the development of teaching skills. While the report appropriately focuses on these aspects of teacher education, it does not, unfortunately, accurately reflect the work of teacher education programs in California or nationally.

NCTQ’s methodology is a paper review of published course requirements and course syllabi against a check list that does not consider the actual quality of instruction that the programs offer, evidence of what their students learn, or whether graduates can actually teach. Concerns about the organization’s methods led most schools of education nationally and in California to decline to participate in the data collection. (NCTQ’s website indicated that fewer than 1% of programs in the country “fully cooperated” with the study.) NCTQ collected documents through websites and public records requests. The ratings published in this report are, thus, based on partial and often inaccurate data, and fail to evaluate teacher education quality.

The field’s concerns were reinforced last month when NCTQ published ratings of states’ teacher education policies which bore no relationship to the quality of their training systems or to their outcomes as measured by student achievement. In this study, the highest-achieving states on the National Assessment of Educational Progress — including Massachusetts, Vermont, New Hampshire, Maine, New Jersey, and Minnesota — all got grades of C or D, while low-achieving Alabama got the top rating from NCTQ. It is difficult to trust ratings that are based on criteria showing no relationship to successful teaching and learning.

In this latest study of programs, the indicators used to measure the criteria often fail to identify the aspects of practice that are most important or the actual outcomes that programs achieve. A case in point: Graduate programs at highly-selective universities like Harvard, Columbia, and Stanford got low ratings for selectivity because they do not require a minimum grade point average or GRE score, although their students in fact rank far above national averages on these measures. NCTQ was uninterested in the actual grades or test scores earned by candidates.

In addition, the degree of inaccuracy in the data is shocking. Columbia was rated highly for the selectivity of an undergraduate program that does not even exist. Stanford received low scores for the reported absence of courses in secondary mathematics education that do in fact exist (indeed candidates must take three full courses in mathematics curriculum and instruction) and are prominently displayed, along with syllabi, on its website. UC-Santa Barbara’s three courses in elementary mathematics education, four courses in the teaching of English learners, and full year of student teaching were also entirely missed, along with its entire secondary credentialing program, all prominently displayed on the website. California State University at Chico was rated poorly for presumably lacking “hands-on” instruction, even though it is well-known in the state for its hands-on learning lab and requires more than 500 hours of clinical training during its full year of graduate level preparation.

It is clear as reports come in from programs that NCTQ staff made serious mistakes in its reviews of nearly every institution. Because they refused to check the data – or even share it – with institutions ahead of time, they published badly flawed information without the fundamental concerns for accuracy that any serious research enterprise would insist upon.

In addition to these shortcomings, NCTQ’s methods are especially out of synch with California’s approach to teacher education in two ways:

• First, while the NCTQ checklist is based largely on the design of undergraduate programs (tallying subject matter courses required during the program), California moved long ago to strengthen teacher education by requiring graduate level programs, which require subject matter competency BEFORE entering preparation. The means by which the state ascertains teachers’ competency — through college majors, approved subject matter programs, and rigorous state-developed tests — are ignored in the NCTQ ratings.

• Second, while NCTQ focuses on paper requirements for inputs, California has moved toward accountability based on stronger evidence of outcomes, including rigorous tests of basic skills, content knowledge, and pedagogy. These include California’s Teacher Performance Assessments, required under SB 2042, that have made the state the first in the nation to judge teachers’ skills and abilities in real K-12 classrooms with real students. At least one of these assessments has been shown to predict teachers’ later effectiveness in raising student achievement. These outcomes are also absent from the NCTQ framework. The candidates who have made their way through all of these assessments constitute only two-thirds of those who initially set out to seek teaching credentials.

Accurate, well-vetted information on course requirements and syllabi, plus extensive data on actual candidate qualifications, evaluations of program quality, employers’ assessments of candidates’ readiness, and graduates’ performance in classrooms are available through state and national accreditation records, as well as in-depth studies conducted by researchers. The California Commission on Teacher Credentialing (CCTC) is a ready source of such data, as is the national accrediting body (the Council for Accreditation of Educator Preparation). CCTC received no request from NCTQ for this information.

Unfortunately, the answer to the question of what we can learn about teacher education quality from the NCTQ report on Teacher Prep is “not much.” Without reliable data related to what programs and their candidates actually do, the study is not useful for driving improvement.

In contrast to the NCTQ approach, researchers and educators serious about improving preparation are focusing attention on developing accurate and reliable data about program outcomes and useful evidence of program quality. In California, the Commission on Teacher Credentialing, like others across the nation, is redesigning licensing and accreditation with these goals in mind. To secure ongoing improvement, teacher educators must pursue comprehensive accountability and increased transparency in data about the outcomes of our programs and the opportunities to learn they provide.

Linda Darling-Hammond is the chair of the California Commission on Teacher Credentialing and the Charles E. Ducommun Professor of Education at Stanford University.