Blogger Michael Deshotels (Louisiana Educator) compares state test scores to NAEP scores. The state scores are up, the NAEP scores are flat. What’s going on?
He writes:
School reform in Louisiana was supposed to eliminate social promotion and the awarding of worthless high school diplomas.
Superintendent John White has staked his entire career as an education reformer on improving state standardized test scores of Louisiana students. To reformers like White, test scores are everything. In their philosophy of education, you can’t trust teachers to tell us and parents whether students are learning and progressing and are going to be ready for college or careers when they graduate. Reformers believe that Louisiana needs an objective way of finding out if our students are getting diplomas that indicate that they are ready to compete with students from other countries for the best jobs in the world economy.
John White was selected by former Governor Jindal to be our State Superintendent at the beginning of 2012 with the mission of implementing new laws that would evaluate, reward and fire teachers based on student test scores and to implement the replacement of many public schools with independent charter schools. The charter schools would live and die based on the attainment of high student test scores.
From the very beginning of our Louisiana education reforms, the reformers announced that they wanted to eliminate diploma mills that turned out graduates that had no real education and were not going to be fit for the job market or college. Corporate education reform was no longer going to allow diplomas to be awarded to functionally illiterate young people. Reformers believed that it was time to eliminate social promotion, whereby children were automatically promoted to the next grade even though they had not achieved satisfactory results on their math and ELA courses. The gate keepers would be cut scores on state tests that would indicate proficiency or failure.
Well, that didn’t work.
Everything in the White administration revolves around increasing student test scores. The school rating system installed by White and his TFA cronies applies maximum pressure on school administrators and teachers to do almost nothing but attempt to raise student test scores.
Louisiana state law requires that our state tests be compatible with the National NAEP test so that our student performance can be compared to other states.
The education reform laws also required that the new Louisiana standardized state tests must be compatible to nationally recognized tests including the National Assessment of Educational Progress (NAEP). In other words, a student rating of proficient on the state tests should be the same as proficient on the NAEP test. It was decided that a rating of Mastery on the state tests should be equivalent to a rating of Proficient on the NAEP.
So how reliable are our state LEAP and End-of-Course tests, compared to the NAEP? Does the progress of our students from year to year on LEAP match the progress measured by NAEP? Are we finally moving our students to proficiency and awarding them diplomas that future employers can trust are indications of real academic skills? Since the legislature had decided at the beginning of the reforms that we couldn’t trust the teachers to tell us whether a student was worthy of getting a diploma, did they also insist on a check-up system to see if we could trust the State Department of Education and their standardized tests to certify that a student was worthy of a diploma?
Oops, it looks like the legislature forgot to set up an independent check on our Department of Education to see if they were faithfully holding up their end of the bargain to end social promotion and grant real diplomas. There is no one officially checking to see if the LDOE tests are really measuring proficiency as comparable to the NAEP tests. But there is a way of checking the validity of our state tests compared to NAEP. There just is no law requiring anyone to make the comparison. So here is my effort to provide a legitimate comparison of the two testing systems.
On the state tests, students made dramatic improvements. On NAEP, no dice. No gains, some decline.
According to state testing, John White is a big success. Louisiana’s public school students are improving dramatically, and are well on their way to achieving Mastery or Proficiency by 2025. But according to national testing, achievement scores have barely improved in three areas and have dropped in 8thgrade math. Louisiana is near the bottom of the NAEP rankings. Most independent agencies now rate Louisiana as the lowest performer out of all the states in the measures of school performance.
Common core standards may be not be teachable for at least half of our students.
My opinion, which I can’t prove, is that the lack of progress in student proficiency is really a result of implementing the common cores standards which are basically unteachable for at least half of our students.
Louisiana is allowing the same abuses that have resulted in charges of fraud in the reported graduation rate of the Washington D.C. school system.
Now, not only are Louisiana students being promoted who demonstrated unsatisfactory test performance, but schools are allowed to waive the attendance requirement for promotion and graduation. Now students in high school who missed much more than the allowed absences and who failed their state tests, often still get a diploma by just taking a few hours of credit recovery courses. For example, students can now pass their Algebra I EOC test by scoring only 23.5% correct answers. This is exactly the same situation that caused the graduation rate in the Washington DC schools to be declared fraudulent. But here in Louisiana, no one in an official position is blowing the whistle.
So if you think there were illiterate students getting diplomas in the old days, that’s nothing compared to the rampant awarding of diplomas to anyone with a pulse today. I certainly do not believe that the state tests are valid enough to be used as the promotion standard. I have much more faith in the judgement of teachers. Unfortunately the law that is supposed to prevent the pressuring of teachers on promotion decisions is also being ignored in the push to boost the graduation rate at all costs.
As Arne Duncan used to say, again and again, they are lying to our children.
Diane Ravitch's blog 
Is there a mistake here?
“In other words, a student rating of proficient on the state tests should be the same as proficient on the NAEP test. It was decided that a rating of Mastery on the state tests should be equivalent to a rating of Proficient on the NAEP.”
NAEP proficiency and state exam proficiency and mastery are all supposed to be equivalent? Or state of exam mastery and NAEP proficiency were ultimately linked? So what about state exam proficiency, re NAEP?
Akademos,
The “cut scores” (behind the definition of “proficient”) are completely arbitrary. Those in charge can raise or lower them to make schools (and themselves) look good or bad. Administrators in charge want low scores when they start and high scores when they leave.
Yeah, I get that. I think there’s a missing phrase that relates those two quoted sentences.
And this is all in this context:
“But everything in the White administration revolves around increasing student test scores. The school rating system installed by White and his TFA cronies applies maximum pressure on school administrators and teachers to do almost nothing but attempt to raise student test scores.”
The best thing that could happen for education in Louisiana would be a hurricane that blows John White out of the state.
“The state scores are up, the NAEP scores are flat. What’s going on?”
Wilson:
“My own conclusion is that tests have so many independent sources of invalidity
that they do not measure anything in particular, nor do they place people in any
particular order of anything. But they do place them in an order, along a single line of
“merit,” and that is all they are required to do.”
“Assessment practice is permeated with mythology and ideology; with confusions
and contradictions; with epistemological and ontological slides; with misrepresentations
of frames of reference for different assessment modes; with logical type errors and
psychometric fudging, in which the constructs that determine error–labelling,
construction, stability, generality, prediction–are either ignored or severely constrained
in the determination and communication of error, in those rare cases where personal
error and likely miscategorisation is publicly admitted.”
Yet the scored based mythology and ideology continues the cognitive dissonance.
If the game of testing had produced me, ending the score based mythology, once
and for all, would end MY “mark” and it’s meaning…
Some more reasoning from Wilson:
“To the extent that these categorisations are accurate or valid at an individual level, these decisions may be both ethically acceptable to the decision makers, and rationally and emotionally acceptable to the test takers and their advocates. They accept the judgments of their society regarding their mental or emotional capabilities. But to the extent that such categorisations are invalid, they must be deemed unacceptable to all concerned.
Further, to the extent that this invalidity is hidden or denied, they are all involved in a culture of symbolic violence. This is violence related to the meaning of the categorisation event where, firstly, the real source of violation, the state or educational institution that controls the meanings of the categorisations, are disguised, and the authority appears to come from another source, in this case from professional opinion backed by scientific research. If you do not believe this, then consider that no matter how high the status of an educator, his voice is unheard unless he belongs to the relevant institution.
And finally a symbolically violent event is one in which what is manifestly unjust is asserted to be fair and just. In the case of testing, where massive errors and thus miscategorisations are suppressed, scores and categorisations are given with no hint of their large invalidity components. It is significant that in the chapter on Rights and responsibilities of test users, considerable attention is given to the responsibility of the test taker not to cheat. Fair enough. But where is the balancing responsibility of the test user not to cheat, not to pretend that a test event has accuracy vastly exceeding technical or social reality? Indeed where is the indication to the test taker of any inaccuracy at all, except possibly arithmetic additions?”
A Little Less than Valid: An Essay Review
http://edrev.asu.edu/index.php/ER/article/view/1372/43
Rigged cut scores are the stock-in- trade of so called reformers that manipulate scores to be what they want the public to see. They are often used a tool to close public schools. This is why opt-out still matters. Proficiency is arbitrary smoke and mirrors.
The whole discussion is ridiculous.
State tests and NAEP tests have never been aligned, nor were they intended to be.
Decisions about cut scores for levels of achievement (e.g., basic proficient, advanced) are not the same, levels of achievement are established by different methods, and NAEP tests are given to a sample of students, not every student–as intended by federal policies for state tests (and endangered by the opt-out movement).
Complaints about low state test scores relative to NAEP are part of an effort to have a single national test–the point of the Common Core Standards with “comparability” in scores from PARCC tests and SBAC tests. Achieve is among many defenders of the Common Core and still trying to infect NAEP scores with the farce that those standards are “authoritative.”
Hardly anyone talks about the many NAEP tests for subjects that do not produce year-to-year trend lines–ups and downs or flatlines. The hoopla is mostly about reading and math. Tests in music and visual art for example, have no trend lines at all. They are given about once a decade and with budget cuts, only in grade* when most students are not enrolled in courses.
https://www.washingtonpost.com/news/answer-sheet/wp/2018/05/20/what-they-said-before-and-what-they-are-saying-now/?utm_term=.8f182ac9ccf8
Laura,
I agree that Louisiana’s state tests and NAEP are different as you describe. I also get that the idea of aligning state tests to NEAP aligns with arguments for a single national test. However, Mr. Deshotels is not arguing for a national PARCC-style test.
It is the stated objective of LDOE to have Mastery on Louisiana’s state tests align to proficient on the NAEP. See slide 19 of this LDOE presentation.
https://www.louisianabelieves.com/docs/default-source/accountability/accountability-commission-presentation—october-2016.pdf?sfvrsn=4
Clearly alignment has not been maintained.
Louisiana’s fourth graders’ proficiency rate in mathematics on the NAEP dropped from 30 percent in 2015 to 27 percent in 2017 while their proficiency rate (Mastery or Advanced) on the state tests rose from 33 to 39 percent across the same two years.
Mr. Deshotels is documenting that the state tests are not living up to LDOE’s stated objective. Since John White became state superintendent in 2012 our school grading system has been totally revised at least three times, and our tests have been redesigned at a dizzying pace.
Don’t the companies that create the Louisiana state tests have an interest in making New Orleans look like a miracle? Are they not connected to the corporate and investment banking interests behind charters?
Louisiana tests are invalid because scores don’t correlate with NAEP?
Is that what the article is claiming?
So if NAEP scores are not up, what is the basis for the so-called “New Orleans miracle” that the “reformers” tout? Could it be there is no miracle at all? We might want to share this information with David Leonardt….
State Ed Departments get what they pay for:
standardized test scores custom made
for the narrative they want to sell.
Using “cut scores” is a magic elixir,
but there is more than one way to affect the final numbers
Manipulate the . . .