Matt Di Carlo is a cautious social scientist who looks at education policy from every possible angle.
Like me, he does not believe in miracles in education. Education is a steady, slow, incremental process of development that is hard work. Changes comes slowly.
Yet here is Di Carlo on the dramatic gains that Maryland has made on NAEP, outpacing the nation.
How did they do it?
Maryland has charters, but not very many, and they are unionized.
Maryland has high per pupil spending.
Maryland did not implement any of the market-based teacher reforms demanded by corporate reformers during the time of its greatest gains.
Maryland has strong teachers’ unions.
Maryland teachers are well paid.
Don’t keep it a secret.
Update: Matt Di Carlo added an update to his post to say it is satirical. But the NAEP charts are factual.
His point is that you can use NAEP data to make any argument you want and to be careful about using them for anything other than what they are: trend lines. Why the lines go up or down requires more analysis.
Diane Ravitch's blog 
Unfortunately Maryland signed on to Race to the Top and is now imposing the market-based reforms, specifically, tying teacher compensation to student test scores. The State Department of Education has rejected the teacher evaluation plans of several counties, including Montgomery and Prince George’s (disclosure, I was part of the group that designed the plan in Prince George’s) because they did not make a sufficient portion of the teacher’s evaluation dependent upon student test scores. That does NOT bode well for the future. It is interesting that the two largest districts in the state got hit by this. Montgomery and Prince George’s are the 17th and 20th largest districts in the nation, according to this accounting. If you add the figures from that chart, you get almost 267,000 students, while according to the Census Maryland has a total student population of about 848,000. Just those two school authorities (out of the 24 in Maryland) have almost 1/3 of the public school students in the state.
And yet Maryland is about to implement RttT policies, the CCSS, and a new teacher evaluation system with a 20% weight on standardized test results and a 30% weight for student growth measures. Why?
In case some readers miss it, let me include two key parts of Matt’s excellent post:
1. “There’s too much at stake for policy makers to waste time worrying about trivial details, such as policy analysis. Whatever laws and practices were in place in Maryland during this time period are clearly responsible for the Miracle. They should serve as a model for the nation.”
2. “(Update) Note to Readers: This post is satirical.”
Matt Di Carlo is my favorite blogger with respect to demonstrating the confusion between correlation and causation when evaluating data.
Maryland is supposedly one of the top school system in the country. Can anyone tell me why they are changing their curriculum to the untested, unpiloted Common Core. Is it the $250 million bribe from the Fed. Gov. for Race To The Top funds. That can’t be it. Our State School Board surely would not sell out our children for only $250 million which according to this yrs. budget would only operate our schools for 15 days. My county (Worcester County)is only getting $1.1 million over 4 yrs. and publishing the cost to implement RTTT is estimated to be $5.1 million. Not a good return on investment for an untested program.
Could ALEC be rearing its ugly little head in Maryland?
Diane, you list several characteristics such as strong teachers unions and high PPE, and appear to be citing those as examples for WHY Maryland is doing well. A few thoughts:
1) Are there no other state level variables that could be at play, such as reading initiatives or Response to Intervention (RtI)? How have you come to the conclusion that strong teachers unions, for example, are the active ingredient? Aren’t there other states with strong unions that are NOT performing well? How do you explain the difference?
2) You seem to be using state test/NAEP data as proof that teachers unions, high PPE, etc. are effective. In other words, you are considering NAEP as a valid method of evaluating the educational climate of a state. This seems to be different from some of your other posts in which you say that using high-stakes standardized tests to measure educational effectiveness is bad practice because it doesn’t capture the quality of education, but also is incorporating variables such as poverty. How do you explain the difference in your approach, and why are you choosing to use high-stakes tests to evaluate education when it yields positive results for what you believe in, but want to dismiss those results when not favorable? If we are consistent with your interpretation above, are we to conclude that – if Maryland had LOW NAEP scores – teachers unions, high PPE, etc. would be BAD ideas?
Matt Di Carlo wrote that post as a satire.
He added that as an update; I updated to reflect his update.
The NAEP trend lines are accurate and factual.
Maryland has made significant gains on NAEP.
Di Carlo wanted to show that it is chancy to make interpretations of NAEP trend lines.
They are trend lines.
I think he meant the post as an implicit rebuke to Jeb Bush, who goes around the country touting “the Florida miracle,” and pointing to NAEP trend lines.
He doesn’t think that mandatory class size reduction, which the voters demanded, might have had a role in the gains in the early grades.
Di Carlo would warn not to jump to conclusions.
Thanks Diane – so it sounds like I may have misread your original post. I’m now hearing that you’re saying that NAEP scores should NOT be used to measure the effectiveness of educational conditions/programs such as high PPE, teachers unions, charters, etc.?
I will say it again.
The NAEP data for Maryland are completely accurate.
NAEP does not explain causation.
Look at the NAEP trend lines in Florida, and juxtapose them with the slow phasing in of class-size, and the correlation simply doesn’t hold up. Plus, many inner city schools where scores were lowest already were under the class-size limits before the referendum was held because of student population declines. Why do you so proudly wear the same ideological blinders you accuse your former comrades and current foes of wearing. You are both two sides of the same coin, equally intransigent and therefore incapable of finding real solutions..
While satire has its place, I think there is so much satire that many people are confused by it. They read something, try to validate it or just repeat it without validation. Then there are all kinds of comments slung at the “bad reporting”. I think this is too important to pepper with satire. We need direct, factual, meaningful confrontation with these corporate take-overs of public education. Some have taken “free market” way too far. I am very tired of the “freedom to be stupid and uninformed”. That is what we need to be bold enough to push back against.
EdEdEd, NAEP provides trend lines. The lines show whether scores on the subjects tested went up or down. The NAEP trend lines don’t say why they went up or down. Di Carlo was warning implicitly about the risk of attributing causation, when there are many factors at play. For example, Jeb Bush goes around the nation showing off the Florida NAEP scores and claiming credit. I doubt that he ever mentions the mandatory class size reductions that occurred during the period of improvement. But the one takeaway is that the trend lines do not reveal causation. Period.
The satire thing is confusing since the data is real, but I think I get his point about drawing conclusions. My conclusion would be this: Maryland is heavily dependent on government employment (next to DC) and thus was less affected by the recession which began just when its scores started to head upward. The conclusion: kids whose families did not lose jobs, go into poverty, etc. did better than those who did. Which reinforces the link between income and education outcomes.