You may recall the story. Last Year year, NPR reported on a dramatic transformation at Ballou High Dchool in the District of Columbia. Despite a long record of poor performance, 100% of its students graduated and every one of them was accepted into college. Great things are happening in D.C., the reporter said. However, months later, NPR ran another story admitting the errors in the miracle story. The school was burdened by excessive absences, students graduated who had not met basic requirements, and some were admitted to community colleges that accept all applicants.
Enough questions were raised about the story to get the attention of the ombudsman, who explained what happened here.
The ombudsman, Elizabeth Jensen, writes:
”The original piece is somewhat startling to read, given what we know now. As the story points out, the graduation rate in the 2015-16 school year was 57 percent. That year, only 3 percent of students met citywide English standards and no one passed the math. The question in any reporter’s mind should then be: Was the 2017 progress simply too good to be true?”…
”On its face, the report in late June was accurate; as the headline said, the students did get accepted to college (some of them to a D.C. community college that accepts all who graduate, information which was only uncovered later by McGee). Still, a story can be accurate but incomplete, and therefore leave a misleading impression.
“The report also included this line: “But it was a strong support system within D.C. Public Schools that made it a reality.” There appears to be truth in that: The story documents the college application support, the pep rallies and the paid-for college tours the school system provided.
“But it’s also an ironic truth, in retrospect. The school may have provided lots of support but, as the second story showed, it also subsequently graduated students who did not meet graduation requirements. That included overlooking “high rates of unexcused absences.” How high were those rates? As NPR and WAMU reported last week, “Half of the graduates missed more than three months of school last year, unexcused. One in five students was absent more than present — missing more than 90 days of school.”
“So WAMU/NPR did not fall for a P.R. pitch. It invested time in reporting. Why did its report fail to turn up the other problems?”…
”My take? The first piece was far from perfect. It contained some overly enthusiastic language about Ballou’s accomplishments (I like feel-good stories as much as others, but unqualified statements are often what get reporters in trouble). The claims the school made were hard to believe — which should have made the newsroom more skeptical. If the reporting team needed more time, it should have been granted, despite deadlines.
“But this situation also underlines one of the challenges of reporting: when to stop. Often the reporter’s instinct is to keep reporting, but deadlines are a reality, after all. I don’t believe the reporting was slapdash (some reports I found in other news outlets simply took Ballou officials at their word and did not do any original reporting at all), but it is unfortunate that of all the people talked to, not one raised the problems that became apparent later. Perhaps one more interview would have been the one that unlocked the bigger picture.”
Ahem. My take? Journalists should always question stories that involve miraculous claims about test scores and graduation rates. Skepticism should be their default attitude towards claims that sound too good to be true. If at first they take the bait, they will tend to stop digging and become defensive. Those who take the bait will look foolish, and indeed they are. When a school makes outlandish claims about test scores, ask first who graded the tests. Then check the process for excluding and selecting students. Ask whether the school has the same proportion of students with disabilities and English learners as neighborhood schools. Dig deeper. Ask whether it accepts students with cognitive disabilities, or only those with mild learning disabilities. Keep digging. It has been my experience that behind every “miracle” school there is either fraud, dubious practices (e.g., credit recovery), or careful selection and exclusion of students.
