Bruce Baker has a great post about Matt Di Carlo’s April 1 post on the Maryland NAEP scores. Baker knows that Di Carlo was using his post to mock the misuse of NAEP data, but Baker shows how people like Chris Cerf frequently use misleading graphs to make a point.

When Di Carlo put up his April Fool’s Day post about a “miracle in Maryland,” he forgot to label it as a satire. He updated it.

I posted it  in the original version, and noted that neither Matt nor I believe in miracles. When Matt updated his post, I updated my post.

But the interesting point is that the graphs in Matt’s post about the gains posted by Maryland were not made-up. They were real. Maryland has made impressive gains in reading and math.

Matt’s point was that NAEP data cannot be used to explain causation. He extrapolated possible causes as a way of satirizing the way certain big-name reformers like Jeb Bush use NAEP data and claim credit for their policy preferences. Doing this is wrong. NAEP shows trend lines, but it doesn’t say why the scores went up or down. In most states and cities, there are many things happening at the same time: demographic changes, state law changes, federal mandate changes, and unexplained changes. The trend lines can’t tell you which of those changes, if any, caused the scores to change.