Maine was part of the Smarter Balanced Assessment Consortium. Students took the online tests. Complaints from students, parents, and teachers were so loud that the legislature dropped out of SBAC.
Now the the results are in, and no one knows what to make of them since they are not comparable with previous tests.
Many students opted out. The state average of opt outs averaged more than 10%, but in some schools more than 50% did not take the tests.
Ohio has withdrawn from the federally funded PARCC test, but the results came in from last spring’s tests. A little more than one-third “met expectations.” Put another way, nearly two-thirds “failed.”
Under the old state tests, 75-80% were proficient. Ohio softened the blow of high failure rate by creating a new category called “Approached Expectations.” This reduced the proportion of “failures.”
“That will have, for example, students that “Met Expectations” on PARCC rated as “Accelerated” by Ohio. And students will be labeled as “Proficient” by Ohio, even if they still just “Approached Expectations” of the 12 PARCC states.
“That means that many more kids will labeled as “Proficient” than the PARCC states would consider as meeting expectations.
“Jim Wright, ODE’s director of assessment, told the board this morning that shouldn’t be a concern.
“Educators across the country have warned that scores and ratings would drop with the new tests. The proposed ratings will bring a drop, just not the “cliff” that people warned about, Wright said.”
The test scores for most charter schools, like most public schools, declined sharply on the new Smarter Balanced tests of the Common Core.
Rocketship charters, in particular, did poorly.
The typical response of charter leaders was not to complain that the tests are biased and unreliable (which they are), but to say, showing grit, “We will have to up our game.” I suppose that means they weren’t trying hard enough until they got the scores.
Will the sobering news burst the charter bubble? Of course not. Too much money riding on their proliferation.
This is a terrific article about the Common Core test results. It explains in layman’s language how the test scores are calculated and converted to scale scores.
When you read the “results” in the newspaper or get the results for your child or your class, you need to understand that the “scores” are not really scores:
The only things that have been released are percentages of students who supposedly meet “proficiency” levels. Those are not test scores—certainly not what parents would understand as scores. They are entirely subjective measurements.
Here’s why. When a child takes a standardized test, his or her results are turned into a “raw score,” that is, the actual number of questions answered correctly, or when an answer is worth more than one point, the actual number of points the child received. That is the only real objective “score,” and yet, Common Core raw scores have not been released.
Raw scores are adjusted—in an ideal world to account for the difficulty of questions from year to year—and converted to “scale scores.” A good way to understand those is to think of the SAT. When we say a college applicant scored a 600 on the math portion of the SAT test, we do not mean he or she got 600 answers right, we mean the raw scores were run through a formula that created a scale score—and that formula may change depending on which version of the SAT was taken. Standardized test administrators rarely publicize scale scores and the Common Core administrators have not.
Then the test administrators decide on “cut scores,” that is, the numerical levels of scale scores where a student is declared to be basic, proficient or advanced
The cut scores are the passing marks. They are arbitrary and subjective decisions made by fallible human beings. They can raise the passing mark to create large numbers of “failures,” or they can lower the passing mark to create a “success” story, to celebrate their wonderful policies. In some cases, the cut score is set high, so many students “fail.” The next year, or year after, the cut scores are lowered, and HOORAY! Our Wise Leadership Has Created Success!
As Horn writes:
Now, when a news story says that proficiency percentages were “higher than expected,” you should know what was “expected.” The Common Core consortiums gave the strong impression that they would align their levels of “proficiency” with the National Assessment of Educational Progress (NAEP) nationwide standardized test. (That is, by the way, an absurdly high standard. Diane Ravitch explains that on the NAEP, “Proficient is akin to a solid A.”)
Score setting is a subjective decision, implemented by adjusting the scale and/or cut scores. If proficiency percentages are “higher than expected,” it simply means the consortium deliberately set the scores for proficiency to make results look better than the NAEP’s. And that is all it means.
It is no different from what many states did to standardized test results in anticipation of the Common Core exams. New York intentionally lowered and subsequently increased statewide results on its standardized tests. Florida lowered passing scores on its assessment so fewer children and schools would be declared failures. The District of Columbia lowered cut scores so more students would appear to have done well. Other states did the same.
The bottom line is this: The 2015 Common Core tests simply did not and cannot measure if students did better or worse. The “Smarter Balanced” consortium (with its corporate partner McGraw-Hill), the only one to release results so far, decided to make them look better than the NAEP, but worse than prior standardized tests. The PARCC consortium (with corporate partner Pearson) is now likely to do the same. It’s fair to say the results are rigged, or as the Washington Post more gently has put it, “proficiency rates…are as much a product of policymakers’ decisions as they are of student performance.”
You MUST MUST MUST MUST open the link to the cut scores announced by the Smarter Balanced Assessment Consortium, which Horn helpfully supplies. Scroll down to pp. 5-6. You will see that the cut scores predict that most students will “fail” in every grade. Only the top two levels are considered “passing,” that is, proficiency and advanced. In third grade math, 61% are predicted to “fail.” In fifth grade math, 67% are predicted to “fail.” In eighth grade math, 68% are predicted to “fail.”
The ELA predicted failure rates are slightly better, but even there, the majority of students are expected to “fail” because the cut score was so high.
If they chose different cut scores, the proportion passing or failing would be different, higher or lower.
This is not unique to the Common Core tests. This is the way all standardized testing is graded.
You can see how easy it is for political figures to manipulate the passing rates to their advantage.
Howard Blume reports today that the achievement gaps among children of different groups widened on the Smarter Balanced Assessment tests whose results were just reported.
The tests are “harder,” “more rigorous,” and so the students who already had low scores have even lower scores.
This is akin to raising the bar in a track event from 4 feet to 6 feet. Those who couldn’t clear the 4 foot bar will certainly not clear the higher bar.
If anyone remembers, we were told repeatedly that the Common Core would close achievement gaps between whites/Asians and Blacks/Hispanics, and between upper income/low income students.
It hasn’t, and it won’t.
The tests were designed to fail a majority of students of every group. Here are the cut scores for the SBAC tests. The developers predicted mass failures last fall.
Let’s just say that the Common Core and the tests aligned to them are a disaster for American education. Kids don’t necessarily try harder when they fail again and again. They may give up.
Many people suspect that the purpose of all this manufactured failure is to make parents eager for charter schools and vouchers. They may be right.
Blume writes:
“This is going to show the real achievement gap,” said Chris Minnich, executive director of the Council of Chief State School Officers. “We are asking more out of our kids and I think that’s a good thing.”
At the same time, he added, “there’s no question that when we raised the bar for students that we’re going to have to support our lower-achieving students even more so than we are now.”
Although scores declined for all students, blacks and Latinos saw significantly greater drops than whites and Asians, widening the already large gap that was evident in results from earlier years, according to a Times analysis.
Under the previous test, last given to public school students two years ago, the gap separating Asian and black students was 35 percentage points in English. The gap increased to 44 percentage points under the new test. Asian students’ results dropped the least on the new tests, which widened the gap between them and those who are white, black or Latino, the analysis showed.
White students also maintained higher relative scores than their black and Latino peers.
A similar pattern occurred with students from low-income families. Their scores in math, for example, declined at a steeper rate (51%) than those of students from more affluent backgrounds (16%). In the last decade, all ethnic groups made significant academic gains compared to where their scores started. But the gap separating the scores of blacks and Latinos from whites and Asians changed little….
In that subject, 69% of Asian students achieved the state targets compared to 49% of whites, 21% of Latinos and 16% of blacks.
Although even Asian students have room to improve, their relative performance stood out. In math, the percentage of Asians who met state targets declined 12%. White students went down 21%, Latinos 50%, black students 54%. More than half the students who took the test were Latino.
The future of California, Lucia said, will depend on students of color graduating from schools with the skills they need to succeed in college and careers.
A top-performing school district was San Marino Unified, which is located in a mostly high-income area and enrolls 56% Asian students; 84% of students met state learning goals in both English and math.
L.A. Unified, which enrolls a majority of low-income, minority students, fared much worse overall. Achievement gaps widened less in L.A. Unified than in the state as a whole but that’s largely because its white and Asian students declined more, according to the analysis.
In L.A. Unified, 67% of Asian students met state targets in English, compared to 61% of white students, 27% of Latinos and 24% of black students.
Laura H. Chapman, a frequent contributor to the blog, who has been a teacher, arts educator, curriculum designer, now retired, writes the following provocative contribution:
There is a well funded marketing campaign to sustain the Common Core and the associated tests.
One facet of the current campaign is designed to lower public expectations about the success of students on the SBAC and PARCC tests and to say, in effect, that cut scores on these tests will to set to approximate the operational definition of proficiency and the pass on NAEP tests, Only students who score at nearly the highest level on NAEP tests are dubbed proficient.
There is also a bit of distraction going on, because SBAC has already announced cut scores based on its field trials in 21 states. There is another little known fact: When PARCC and SBAC applied for federal funding, they promised to make their scores comparable.
PARCC will “coordinate with the Smarter Balanced Assessment Consortium on… artificial intelligence scoring, setting achievement levels, and anchoring high school assessments in the knowledge and skills students need to be prepared for postsecondary education and careers.” Partnership for Assessment of Readiness for College and Careers. (2010, December 23). Proposal for supplemental race to the top assessment award. Retrieved from http://www.edweek.org/media/parccsupplementalproposal12-23achievefinal.pdf p. 3
Similarly, the SMARTER Balanced Assessment Consortium (SBAC) asserts: “SBAC and PARCC are strongly committed to ensuring comparability between their assessments…[including] collaborative standard setting that will facilitate valid comparisons of achievement levels (cut scores) in each consortium’s summative test…” SMARTER Balanced Assessment Consortium. (2011, January 6). Supplemental funding budget narrative submitted to the U.S. Department of Education. p. 31. Retrieved from http://www.edweek.org/media/sbac-supplemental-budget-narrative_final.pdf p. 31
Producers of the SBAC tests have their set “cut scores” to report on four levels of performance. Level 1 signals failure. Level 2 indicates “at risk of failure.” Level 3 implies “safe harbor, doing well.” Level 4 means “proficient.” For students in grade 11, only Level 4 indicates readiness for entry-level, credit-bearing courses in college.
Across the grades, in math and ELA, about 11% of students are estimated to score at 4, the highest level and the one that indicates, in the eleventh grade, readiness for entry-level, credit-bearing courses in college. So, that will be very bad news, and it will make news when all of the test scores are gathered in.
The SBAC cut scores in math are estimated to assign 67% of grade 11 students to Level 1 or Level 2, with most (40%) at Level 1. In many states, students who score at Level 1 will also place teachers and administrators at risk of being fired, perhaps with the whole school in line for closure. In addition, many schools will just assign students even more test prep in math, at the risk of harming students’ love for learning and affinities for inquiries that are not driven by tests.
The cut scores for English Language Arts are estimated to place about 59% of students at Level 1 and Level 2, with about 32% at Level 1. Students with these scores are certain to be in the same boat, receiving more test prep. In states like Ohio that guarantee “proficiency” in reading by grade three, 62% of students are likely to fall short, up to 82% if that rule and the meaning of proficiency corresponds to SBAC’s Level 4 definition of “proficiency.”
In any case, the cut score issue is not a trivial matter given the high stakes that federal and state officials have deliberately and foolishly attached to tests, tests that are not “objective” and cut scores that are not “objective” but judgments–and these detached from any concern for the consequences to individual students and all public schools.
Proponents of the Common Core and tests are worried about the political fall-out when the test scores are released in a form that allows stack ratings among all the states that sighed up for the Common Core and have advertised that they have tests aligned with the standards.
They should be worried. Gurus of spin at the American Enterprise Institute suggest how to spin that news. They suggest that the scores should be played down, that news should avoid crisis rhetoric about poor performance. They recommend framing the testing outcomes as just another step on a path “to continuous improvement” in student learning.
That soft “slow-and-steady-as-we-go message” provides cover for policy makers who want to delay high stakes decisions based on these test scores but still use them as a baseline for judging gains in performance for the following year. This delaying tactic may buy time to reset expectations for learning, but it will not stop the obsessive use of test scores and relentless test prep than now dominates life in many schools.
For advocates of the “one size fits all” standards and tests, the comparability in scores from SBAC and PARCC tests means this: Every state that signed up for this grand and nearly maniacal experiment in standardized education will be rated as winners or losers by these supposedly “objective tests.”
The governors and the state education officials who signed adoption papers for this grand experiment in standardized education may be out of office, but current officials will be questioned about the results. Handling the political fall-out will be tricky, especially with an election season heating up, budget problems in many states, and the dueling minds and messages of politicians (notably Republicans, but also Democrats) who support or condemn the Common Core and tests, and many others who have no mindful views other than spin provided to them.
All of the investors in pushing the Common Core–Achieve, the National Governor’s Association the Council of Chief State School Officers, the Gates foundation and buddies along with venture capitalists–want to keep PARCC and SBAC in place, including Common Core “aligned” tests other than those from PARCC and SBAC.
Why? Do not believe the spin about the global economy and needed skills for the 21st century workplace, and all of the other sales pitches.
Test scores with high stakes consequences are the weapon of choice for expanding market-based education. The more students, teachers, and schools fail, the faster the collapse of public education.
As Diane Ravitch and others have said over and over, the cut scores for NAEP “proficiency” are high. NCLB never defined proficiency, Race to the Top did but only in terms of college and career ready specifications, not cut scores or tests.
Achieve has manufactured a “proficiency gap” that looks impressive to a casual viewer, but it is hot air. The proficiency gap is a version of the achievement gap. The kids and the teachers are the problem. The standards and the tests are perfect.
Paul Horton, history teacher at the University of Chicago Lab School, got exasperated about the steady stream of articles endorsing the Common Core in the Chronicle of Higher Education. So he wrote a letter warning the professoriate not to buy the corporate-funded CC propaganda. The letter should have been published as an opinion piece.
An excerpt:
“1. They are the product of a push by private foundations acting in the interest of multinational corporations to colonize public education in the United States and in other areas projected be developed as core production and assembly areas in the emerging global economy. A recent Washington Post article using a well-placed source within the Bill and Melinda Gates Foundation essentially confirmed what many critics have suspected: that Bill Gates effectively controls the Department of Education in the United States through his former employees who serve in leadership positions within the Department of Education.
Our education secretary also does a lot of listening to Michael Barber of Pearson Education. Although Mr. Gates and Sir Michael, as well as other reformers, are doubtless well intentioned, they view the colonization of K-12 education in this country and elsewhere as a “win-win.” In their view, the quality of education will improve with greater accountability, and they will make billions creating and delivering accountability for students, teachers, and education schools.
To implement their plan, they are willing to jettison all ideas of collective responsibility for public education in a classic privatization pincer move: Chicago School of Economics ideas of “free choice” and “free markets” are used to legitimate privatization through virtual control of the editorial boards of major papers—the Murdoch chain, the Tribune chain, The Washington Post (now run by a neoliberal libertarian), and The New York Times—as well as center-liberal media like PBS and NPR. Money is funneled into NPR and PBS by organizations that support privatizing school reform in the name of “support for education programing.”
A Gates-funded Washington consulting firm, GMMB, works 24/7 to sell the Common Core Standards and all other elements of the Race to the Top mandates that call for more charter schools, a standardized-testing regime, and value-added assessments of teachers based on this testing regime. Likewise representatives of the Washington-based Fordham Institute work together with GMMB to send weekly talking points to major editorial boards and education reporters to ensure that representatives from an “independent foundation” are relentlessly quoted.
Not surprisingly, the Fordham Institute is hardly independent, and is heavily subsidized by the Bill and Melinda Gates Foundation, Michael Bloomberg, and the Broad Foundation, and many more funders of privatizing education. While GMMB attempts to control the discourse in the country’s major media outlets (Arne Duncan’s past press secretary is helping to coordinate this propaganda campaign within GMMB), McKinsey sells Microsoft and Pearson packages to fit the Race to the Top mandates.
The Los Angeles Independent Schools boondoggle that packed Pearson Common Core Curriculum lessons within Microsoft tablets and software is the wave of the future. Districts are sold packages that they cannot afford to comply with federal mandates that are pushed by private multinational corporations. What I am attempting to describe is the tip of a corporate iceberg that amounts to corporate control of education policy with very little participation of classroom teachers, parents, or school boards.
The idea that the Common Core Standards are the product of a democratic process is simply misrepresentation of fact—a big lie that GMMB, our education secretary, Bill Gates, Pearson Education, and the Fordham Institute propagate. What many rightfully be called corporate-education reform has bypassed the democratic process. For this reason alone, university faculty and administrators should not support the Common Core Curriculum and the Race to the Top.”
Talk about a messed-up, incomprehensible system!
Many students across the nation have returned to school or will return this week, without their Common Core test scores!
Didn’t the cheerleaders for the tests say that students needed to know, parents needed to know, teachers needed to know what those scores are? Didn’t they say that no one would know how students are doing compared to students in other districts and states without those scores? I don’t happen to know any parents who actually care how their child compares to another child across the nation, but someone does. Maybe someone in the U.S. Department of Education.
Blogger Perdido Street School points out that things go slowly because the benchmarks are set after the tests are given.
Just as with the New York Common Core tests, the benchmarks aren’t set until long after the students take their tests.
With the old New York State Regents exams, the benchmark scores were set before students took their tests and were posted right after the test ended.
That seems like a fair and honest way to do things – set the passing mark before students take the test.
But in the Era of Common Core, when educrats and reformers wanted to rig the tests for 70% failure rates, all of these Common Core tests, including the high school tests, are benchmarked long after students take their tests and the results are in.
Rigged?
You betcha!
If not, why not set the benchmarks before, the way they used to with the Regents exams?
The passing marks (cut scores) on the Smarter Balanced Assessment were set last November. They were set in such a way that most students were certain to “fail.” The SBAC predicted that most students would fail. The executive director, Joe Wilhoit, quoted in the article below, predicted that “over time, the performance of students will improve.” Maybe they will, maybe they won’t. It is a dirty trick to play on students. The cut scores are close to the “proficient” achievement level in NAEP. In twenty-three years of testing the states, Massachusetts is the only state in the nation in which 50% of students reached the proficient level. That indicates that it will be many years–if ever–until half of the students are able to reach these absurdly high cut scores. If they are used for promotion and graduation in the future, most students will not be promoted and will not graduate.
Catherine Gewertz of Education Week wrote:
In a move likely to cause political and academic stress in many states, a consortium that is designing assessments for the Common Core State Standards released data Monday projecting that more than half of students will fall short of the marks that connote grade-level skills on its tests of English/language arts and mathematics.
The Smarter Balanced Assessment Consortium test has four achievement categories. Students must score at Level 3 or higher to be considered proficient in the skills and knowledge for their grades. According to cut scores approved Friday night by the 22-state consortium, 41 percent of 11th graders will show proficiency in English/language arts, and 33 percent will do so in math. In elementary and middle school, 38 percent to 44 percent will meet the proficiency mark in English/language arts, and 32 percent to 39 percent will do so in math.
Level 4, the highest level of the 11th grade Smarter Balanced test, is meant to indicate readiness for entry-level, credit-bearing courses in college, and comes with an exemption from remedial coursework at many universities. Eleven percent of students would qualify for those exemptions.
The establishment of cut scores, known in the measurement field as “standard-setting,” marks one of the biggest milestones in the four-year-long project to design tests for the common standards. It is also the most flammable, since a central tenet of the initiative has been to ratchet up academic expectations to ensure that students are ready for college or good jobs. States that adopted the common core have anticipated tougher tests, but the new cut scores convert that abstract concern into something more concrete.
Smarter Balanced is one of two main state consortia that are using $360 million in federal funds to develop common-core tests. The other group, the Partnership for Assessment of Readiness for College and Careers, or PARCC, is waiting until next summer—after the tests are administered—to decide on its cut scores. Smarter Balanced officials emphasized that the figures released Monday are estimates, and that states would have “a much clearer picture” of student performance after the operational test is given in the spring.
Roxana Marachi, a professor at San Jose State University in California, wrote an open letter to the State Board of Education. She warned them that the results of the Smarter Balanced Assessments, which will be released today, are not valid or reliable or fair. “False data are false data. Period. And to compare future results with current 2015 scores as “baseline” would be just as fraudulent as it would be to promote the 2015 scores as somehow valid.”
Students who are English learners will be harmed significantly by these tests, since SBAC itself predicted a failure rate of 90%, she writes.
These tests violate the most basic principles of the the American Psychological Association:
“We know from decades of research that beliefs matter in student learning and motivation. Without an understanding that the scores are meaningless, students will be likely to internalize failing labels with corresponding beliefs about their academic potential. And unless otherwise informed, families will be likely to believe what the State Department of Education communicates about their children’s readiness for college and career based on an assessment that fails to meet basic standards for testing and accountability.
“Jonathan Pelto has written extensively about SmarterBalanced testing in Connecticut:
“Considering that many of the world’s greatest scientists, authors, actors, teachers and leaders were once English Language Learners one would think the public education system in the United States would be designed to promote and support opportunities for those who need extra help learning the English Language. Moreover you would think education policymakers would be working to find ways to take advantage of the opportunities that having a multilingual population present.”
Marachi writes:
“This seems an ethical dilemma for educational leaders. If they are to be honest with students and families and communicate truthfully that the test scores are meaningless, they would have to acknowledge that the public has been misled (whether knowingly or not) by those promoting the assessments. Acknowledging the current situation would also include accepting the fact that hundreds of millions of taxpayer dollars have been wasted (and are slated to continue to be wasted) should the assessments continue to fail meeting basic standards for testing and accountability.
“Yet, what appears to be the case is that the invalid tests are being falsely promoted as accurate measures of “college and career readiness.” The LA Times just published a piece entitled, “‘Don’t Panic’ Officials Say as California Braces for Lower Student Test Results.” It appears state officials are fully aware of the potential harm and motivational fallout yet “Don’t Panic” is the best message being offered as a remedy rather than full disclosure about the lack of validity of the tests.”
Marachi quotes Dr. Doug McRae, a testing expert, who said:
“Including current scores in student academic records without evidence of validity, reliability, and fairness of the assessments would be “immoral, unethical, unprofessional, and to say the least, totally irresponsible.”
Marachi closes with a Million-dollar Challenge, which should be addressed to every state board member in the nation, as well as to Secretary Arne Duncan, who funded these tests, as well as to David Coleman, the architect of the Common Core standards.
“In closing and in the spirit of critical thinking, I respectfully request that the State Board of Education take on the following challenge. The ultimate endorsement of confidence in your release of SBAC scores would be for each Board Member to publicly take the 11th Grade SBAC Math/ELA tests and to publish your scores at the next State Board of Education meeting. If the assessments are confirmed to be functional and can be verified as accurately, securely, and fairly assessing skills necessary for “college and career readiness”, then every State Board Trustee (all of whom are assumed to be college-educated and career-successful) should receive scores that exceed passing performance. At the very least, this process should allow you the opportunity to fully endorse the assessment product that has been bought and administered to children.
“If this request is declined or somehow otherwise considered unfair, then why would you demand the same of youth entrusted to your care?”
Diane Ravitch's blog 