A reader from Houston suggests that we watch the PBS documentary on Houston’s Apollo program and watch the faces of the students:
He writes:
To see how many kids react to an overemphasis on testing, watch Dropout Nation. PBS Frontline’s Dropout Nation series featured HISD and its Apollo Program in its September broadcast. While there are some good things about Apollo-individulized tutors, more support staff, etc., it’s data driven focus contains the seed of its own destruction. Talking about tests all the time, doing test prep all the time, making kids take tests that they are not relevant to them and that they are not prepared for is wrong.
Watching these kids tell their stories is painful. Watching what some staff are willing to do help kids is heroic. Seeing testing be a focus is exasperating. I was not surprised by the emotional and physical reactions of these kids as staff kept trying to get them and keep them in school. The kids keep saying that the learning is irrelevant. They keep saying that school is boring. They keep saying that no one understands them and their plight. Telling them, “No Excuses!” is disrespectful. Children are not responsible for the circumstances that they are born in and a pat phrase is offensive.
At one point a kid shows up after being gone for days and the staff try to get him to take an SAT test that is about to start.
The Apollo program is in its 3rd year and only the featured high school, Sharpstown, has shown slight improvement. Much of those gains may be to student attrition. Teacher attrition has been high as well. Perhaps that is why Frontline did not show one classroom teacher in the whole episode.
Superintendent Dr. Grier has asked for 17 million more from the Board. If only there were a way to make sure that money went to anything but testing. Social workers, psychologists, teachers, etc. but not a dime for testing.
Diane Ravitch's blog 
Why are they still buying into this “accountability” and “data-driven” model? The data room (around 24:00), reducing students into the set of courses they are taking and their grades is just plainly wrong, and a sharp contrast against the personal viewpoint of this episode.
Terry Grier is one of the staunchest supporters of the College Board and its products, from the PSAT to the SAT to the Advanced Placement program. He went around touting (for a long time) the College Board “study” that said PSAT scores predict AP test scores. But the “study” was a seriously flawed one. A seemingly innocuous statement undermined its validity. The authors noted that “the students included in this study are of somewhat higher ability than…test-takers” in the population to which they are generalized. That “somewhat higher ability” actually meant students in the sample were a full standard deviation above those 9th and 10th graders who took the PSAT. Even then, the basic conclusion was that students who scored well on the PSAT had about a 50-50 chance of getting a “3” on an AP test (the most common score).
Grier kept talking about the “study,” but he’d never actually read it.
Fairfax County, one of the biggest and “best” suburban school systems in the country recently hired Karen Garza as its new superintendent. Garza was most recently superintendent in Lubbock, Texas. But prior to that she was the Chief Academic Officer for Houston Independent School District.
When Garza left Houston to become superintendent in Lubbock, the Houston district’s news release noted this:
“Dr. Garza led the development of HISD’s ASPIRE program, the school improvement effort that paid more than 18,000 teachers and instructional staff more than $70 million in performance bonuses over the last three years based on the academic improvement of children.”
Houston’s ASPIRE program was developed through a “quick and relatively noncollaborative planning process” that was not transparent and that relied on “a complex formula” that teachers did not understand. Worse, teachers “were not allowed access to the data that formed the basis of their performance awards.” The ASPIRE merit pay plan was funded (in part) by the Broad, Gates and Dell foundations, some of the top supporters of corporate-style education “reform.”
See, for example: http://www.cecr.ed.gov/guides/summaries/HoustonCaseSummary.pdf
Researchers and test experts caution against the use of value-added models to evaluate teachers. They note that “value-add models of teacher effectiveness are highly unstable,” and that “Teachers’ value-added ratings are significantly affected by differences in the students who are assigned to them.” Testing expert Jim Popham says that the use of student test scores to evaluate teachers “runs counter to the most important commandment of educational testing — the need for sufficient validity evidence.”
But Garza called value-added models “proven methodology” that are both “valid and reliable.” She said they offer up “increased accountability for schools, teachers and students.” Researchers say that “Value-added ratings cannot disentangle the many influences on student progress.” The National Research Council concluded that value-added models “should not used to make operational decisions because such estimates are far too unstable to be considered fair or reliable.” Yet Karen Garza said they can be used for “recognizing excellence,” for “informing practice,” and for “improving teaching and learning.” And while value-added models “use complex mathematics” that most educators – most people – do not understand to arrive at teacher evaluation scores, Garza says that value-added “reports are easy to interpret.”
Apparently she knows something that the experts do not. The Lubbock school board president said, in reference to Garza, that “you can’t outthink her.”
News reports note that Garza “focused attention of student test scores” and had a “laser-focus” on those scores. The Post says that under her direction the number of students in Houston passing Advanced Placement tests “set new records.” But Houston had a system that paid both students and teachers for those scores, the most common of which is a “3,” the equivalent of a C-. Worse, research on Advanced Placement courses and tests shows that they are grossly over-hyped and “are frequently inconsistent with the results of the research on cognition and learning.”
Apparently, her work in Houston was a primary reason why Fairfax hired her. The Fairfax school board chair siad she wowed the board with “the things she has been able to achieve in various school districts, it’s her vision in leadership.” And Republican board member Elizabeth Schultz said Garza’s arrival represents “the beginning of a sea change in Fairfax.”
Maybe Garza will indeed lead a “sea change in Fairfax.” Or maybe she’ll just so more of what she did in Houston, with a renewed emphasis on College Board products (the PSAT, SAT and AP).
It might be better if she started thinking critically and reflectively, or that “sea change” is likely to be just a heavy dose of corporate-style “reform.”
[and if anyone wishes to wager, my bet is on the corporate “reform”….not that I endorse it, I do not….but it’s hard for people to abandon what they think they “know,” and far too many educators – not to mention students and parents and community members – are badly and sadly misinformed about the College Board]
oops…DO more of what she did….
Haven’t seen comments from you, democracy, on this blog before, but my hat’s off to you. You’re providing much valuable, cited information. Thanks, and looking forward to reading more of your commentary!
I saw this show. It was a strange look into the dynamics of school reform in a very difficult environment. -No teachers interviewed, sounds typical. What would teachers know anyway???? Pretty sad.
Grier just wants to be education secretary. Let him at it. The evaluation system is arbitrary and unique to Houston. It is easily gamed to allow admin to get rid of teachers they don’t like. And if you knew the intellectual capacity of the admin that would scare you.
Also there are 600 hundred teacher openings on the HISD website right now. They won’t fill them with people even remotely qualified. The word is out.