Archives for the month of: August, 2013
« Older Entries
Newer Entries »

Who Set the NY Cut Scores—and What We Still Need to Know

By dianeravitch
August 9, 2013 //
19

In this post, teacher Maria Baldassarre-Hopkins describes the process in which she and other educators participated, setting cut scores for the new Common Core tests in New York.

She signed a confidentiality agreement, so she is discreet on many questions and issues.

At the end of the day, Commissioner King could say that educators informed the process but in reality they made recommendations to him, which he was free to accept, modify, or ignore.

As many teachers have pointed out, in blogs and comments, no responsible teacher would create a test with the expectation that 70% of students are sure to fail. It would not be hard to do. You might, for example, give students in fifth grade a test designed for eighth graders. Repeat in every grade and the failure rate will be high. Or you might test students on materials they never studied. Some will get it, because of their background knowledge, but most will fail.

Why would you want most students to fail?

Commissioner King has repeatedly warned superintendents, principals, and everyone else that they should expect the proficiency rates to drop by 30-35-37% and they did.

This is a manufactured crisis. We know who should be held accountable.

It is Commissioner John King and Regents Chancellor Merryl Tisch. They wanted a high failure rate. They got what they wanted.

***********

A response to the post above, by Fred Smith. Fred worked for many years as an assessment experts at the New York City Board of Education. He has now become an invaluable resource for those who are fighting the misuse and abuse of high-stakes testing.

Fred writes:

Folks,
 
Kudos to Maria Baldassarre-Hopkins.  This is an extremely important piece–an outline and an articulate account of how the 2013 cut scores were set. We’re finally getting a glimpse inside the testing program’s “black box”–how cut points are/were established.
 
Three points grab me and support contentions I share with other observers:
 
First, the cut scores are after-the-fact. “Cut scores were to be decided upon after (emphasis hers) NYS students in grades 3-8 took the tests.”  I believe the standards were set in late June/early July.
 
Second, the review committee’s work is advisory–Despite the committee’s elaborate review process, the end results are recommendations to the commissioner.
 
Third –  “(During the review) We were given more data in the form of p-values for each question in the OIB – the percentage of students who answered it correctly on the actual assessment.”  This and the timing of the review strongly suggest that item-level data (item statistics) from the April 2013 operational tests were used to inform the determination of cut scores. That is, data generated by the test population were used–changing the concept of a standards-based test (as in testing aligned with the common core learning standards) to one that depends on the performance of students who took the test. 
 
This makes the Level 2, 3 and 4 thresholds dependent on how well kids did on the exams–bringing the test score distribution into play and rendering judgments about cut scores and student achievement relative to the composition of the students who took a particular set of items at a particular time–a normative framework instead of a standards-based one.  These factors will vary from year to year, and since 2013 was a baseline year with little it could be anchored to, it is even murkier to see how SED can justify what was done.
 
Let’s not forget either that the items on the April 2013 exams were largely generated via the indefensible June 2012 stand-alone field testing, a procedure that could not have yielded reliable or valid information to construct the core-aligned statewide tests–and, as a further consequence, would call the item stats the review committee worked with into question.
 
SED’s slide show presentation to the Regents in late July about the cut scores, this week’s news management spin campaign and its web site power point barrage on the release of the scores do not address important remaining questions about the quality of the 2013 exams and the cut scores. 
 
There is information the SED obviously has in its possession (and desperately wants to keep hidden), as strikingly noted by Ms. Hopkins. We must demand and obtain: 1- P-values (difficulty levels) for all field test items that were selected for inclusion on the operational April tests–both the field test p-values and the corresponding operational test p-values. 2- In addition, we must have complete item analysis data — showing the percentage of students who chose the correct answers, the percentage who chose each distractor (each incorrect mislead) and the percentage of omissions (no response to item). 3 – We must be given the same information demanded in #1 and # 2 but broken down by ethnicity and separately by need/resource capacity. 
 
Even if SED refuses to produce all of the 2013 operational items that it owns for our scrutiny, there is no justification for refusal to provide the statistical data we are demanding–because none of the data involve exposure of the items and their content.  SED and Pearson have no legitimate excuses for keeping us in the dark based on the immediate availability and nature of the information we are seeking.
 
The only way forward for all of us who want to have public schools that work is to cry out for sunshine, transparency and truth-in-testing.   Short of that we can have no faith in anything coming out of Albany about its latest vision of reform.  The messengers of bad news are on the run.  Blow the trumpets. Get your representatives on board.  Don’t let them slip and slide.  This is a pivotal year
Fred
Categories Common Core, New York, Testing

The A-F Grading System: A Politically Motivated Fraud

By dianeravitch
August 9, 2013 //
23

Jeff Bryant writes a comprehensive review of what he calls “Bennett-gate,” and shows that the A-F grading systems initiated by Jeb Bush is itself a phony way to judge the quality of schools.

He cites Matt Di Carlo, who reviewed Indiana’s grading system, and determined that the grades reflect the characteristics of the students in them:

“Di Carlo’s analysis showed, “Almost 85 percent of the schools with the lowest poverty rates receive an A or B, and virtually none gets a D or F.” Conversely, over half of the schools with the highest percentages of the poorest students received “an F or D, compared with about 22 percent across all schools.”

His conclusion, “as is the case with most states’ systems, policy decisions will proceed as much by student performance/characteristics as by actual school effectiveness.” (emphasis original)

“Under Indiana’s system, a huge chunk of schools, most of which serve advantaged student populations, literally face no risk of getting an F, while almost one in five schools, virtually every one of which with a relatively high poverty rate, has no shot at an A grade, no matter how effective they might be.”

By definition, the A-F system must label some schools with a D or F, so those schools are set up for privatization.

One of the beneficiaries was Charter Schools, USA, a for-profit corporation that hired Tony Bennett’s wife when they moved to Florida.

Frankly, the idea that a school should get a letter grade, like a restaurant, is ridiculous on its face.

Imagine if your child came across from school with a report card that contained nothing but a single letter. As a parent, you would be outraged at the stupidity and simple-mindedness of such a way of gauging “quality.”

A report card should be comprehensive, including both resources available as well as outcomes, and there should be multiple ways of assessing both resources and outcomes, such as teacher turnover, student poverty levels, etc.

No report card will capture every dimension of school performance, but a single letter captures almost no dimension of school performance.

That is why the A-F system is a fraud and a scam, meant to set up schools for privatization.

And let’s be clear: When schools fail, those who should be held accountable first are the leaders of the state and the district. They are the ones who decide when and where to allocate crucial resources. They should not crow about closing schools when it is they who failed to provide the necessary supports for the schools.

 

 

Categories Accountability, Administrators, superintendents, Florida, Indiana

Letters to NY Times: When Failure Is the Norm

By dianeravitch
August 9, 2013 //
21

Read these letters written in response to the New York Times’ terrible editorial favoring the test score debacle and the collapse of scores across the state.

But why listen to educators and parents? What do they know as compared to an editorial writer who sits in an air-conditioned office and ponders every day?

Categories Common Core, Parents, Teachers, Testing

Schneider: Who Created Common Core?

By dianeravitch
August 9, 2013 //
25

President Obama and Secretary Duncan frequently say that the Common Core was “state-led,” perhaps because it is illegal for the U.S. Department of Education to interfere with curriculum or instruction. Maybe it would be best to say that Common Core was hatched inside the Beltway and paid for by the Gates Foundation.

Here is an interesting infographic. Mercedes Schneider calls it “an intricate plot.”

Categories Common Core

NY Superintendent: Trying to Make Sense of Common Core Test Results

By dianeravitch
August 9, 2013 //
30

Jere Hochman superintendent of the Bedford Central School District in New York, tries to make sense of the latest test results in this blog.

He has a series of spot-on metaphors.

The state’s policy is based, he writes, is best described as “Fire. Aim. Ready.”

He adds:

“Raising the bar?  High expectations?  Every student means every?  Rigorous standards?  Benchmark assessments?  No problem.  But don’t make kids and teachers collateral damage due to accelerated, unmapped, make-up-the-rules-as-you-go-along implementation.”

What is it like to be controlled by a state education department that makes up the rules as it goes along, turning students, teachers, and administrators into collateral damage for their half-baked plans and policies?

Categories Accountability, Common Core, New York, Testing

David Steiner Defends Test Score Collapse, Critic Corrects Him

By dianeravitch
August 9, 2013 //
35

David Steiner, who preceded John King as New York Commissioner of Education, wrote an article defending the collapse of test scores in New York. Like Joel Klein and Arne Duncan, he agrees that we are finally telling the truth about the widespread failure of public schools (and, one might add, the even greater failure of charter schools, which had a higher fail rate than public schools). It’s amazing how those who were in charge, who made decisions about resource allocation, curriculum, standards, teacher qualifications, class size, and everything else that affect schools, now stand back and absolve themselves of any accountability.

The first letter responding to Steiner’s article nailed an important difference between Massachusetts, which has led the nation on NAEP in reading and math, and New York, whose scores have been almost flat for the past decade. Gregory McCrea wrote:

“Mr. Steiner leaves out one important point when he compares reform efforts in New York and Massachusetts. At the same time Massachusetts was demanding higher standards through their Education Reform Act, they were dramatically increasing funding to public schools. In fact, from 1993-2002, state spending on public schools increased 8% a year, for a total of over $2 billion.
New York has failed to do the same. Instead, funding for schools in New York has decreased or flat-lined through the political shell game known as the Gap Elimination Adjustment. Despite the politician’s suggestions otherwise, funding has not increased and GEA continues to be the most significant drain on public school funding across the state. More recently, boards of education have had to deal with a tax cap railroaded through by the governor and the legislature. As in Massachusetts decades before, the New York tax cap has made school districts more heavily dependent on state aid which only worsens the funding equation.

“Anyone who actually works in a New York public school (something most of the State Ed. bureaucrats have never done) will tell you that the dramatic shift in testing requirements combined with cuts in funding will decimate learning opportunities for New York’s children. Urban and rural schools alike, wrought with poverty, will be forced to redirect funds toward unproven curriculum models and canned materials based on the false promises of Common Core alignment and improved test scores. Music programs will be cut, art teachers will be directed to teach reading modules, class sizes will increase dramatically, and districts will be forced to cut valuable extra-curricular activities. Students will lose enriched learning opportunities and be herded into “interventions” to increase learning and achievement, the very thing that will most certainly not occur.

In his rush to defend his former employer, Mr. Steiner has narrowed his focus on increased standards for all and ignored the influence of funding, poverty, and parental involvement on student achievement (the latter two have the most significant impact). I am disappointed, but not surprised, that he could not offer a more complete review of the challenges facing schools today.”

Categories Accountability, Common Core, New York, Testing

What If the Common Core Standards Are a Huge Mistake?

By dianeravitch
August 9, 2013 //
98

The law specifically prohibits the U.S. Department of Education from interfering or directing curriculum or instruction.* There must be a hole in that law big enough to drive a truck through, and drive the Obama administration did.

As we all know, the Obama administration used the $5 billion in Race to the Top funding, and its power (contested) to issue waivers, to push, prod, and bribe states into “voluntarily” abandoning their own standards and adopting the untried Common Core standards. Some states dropped weak standards, some dropped better standards that had proved their worth.

Secretary Duncan says these standards will make everyone “college and career ready.” The nation’s major corporations agree. So do the nation’s two big teachers’ unions.

But how do they know what the effect of the Common Core standards will be?

We know that they cause a dramatic decline in test scores. Their boosters say that is great, great news, couldn’t be better. Making the tests harder, they say, will make everyone smarter, and soon everyone will be college and career ready. But how do they know? Where is the evidence?

What if they are wrong?

*SEC. 9527. PROHIBITIONS ON FEDERAL GOVERNMENT AND USE OF FEDERAL FUNDS.

(a) GENERAL PROHIBITION- Nothing in this Act shall be construed to authorize an officer or employee of the Federal Government to mandate, direct, or control a State, local educational agency, or school’s curriculum, program of instruction, or allocation of State or local resources, or mandate a State or any subdivision thereof to spend any funds or incur any costs not paid for under this Act.
(b) PROHIBITION ON ENDORSEMENT OF CURRICULUM- Notwithstanding any other prohibition of Federal law, no funds provided to the Department under this Act may be used by the Department to endorse, approve, or sanction any curriculum designed to be used in an elementary school or secondary school.
(c) PROHIBITION ON REQUIRING FEDERAL APPROVAL OR CERTIFICATION OF STANDARDS-
(1) IN GENERAL- Notwithstanding any other provision of Federal law, no State shall be required to have academic content or student academic achievement standards approved or certified by the Federal Government, in order to receive assistance under this Act.

Categories Testing, U.S. Department of Education

The Error That Caused the New York Test Scores to Collapse

By dianeravitch
August 9, 2013 //
66

Here is the reason for the collapse of test scores in New York City and New York State.

State officials decided that New York test scores should be aligned with the achievement levels of the federally-administered National Assessment of Educational Progress (NAEP).

This is an excerpt from a press release prepared by Mayor Bloomberg’s office:

“The new State test results are in line with previous results for student’s readiness for college and careers and show New York City students have maintained gains made over the past decade. The percentage of New York City students meeting the new, higher bar for proficiency in math (29.6 percent) is similar to the percent of students measured proficient on the 2011 National Assessment for Educational Progress (NAEP) tests (28.0 percent) – up from 20.5 percent on the NAEP test in 2003. The percentage of New York City students meeting the new, higher bar for proficiency in English (26.4 percent) is similar to the results on the most recent NAEP English test (26.5 percent), up from 22.0 percent on the NAEP test in 2003.”

Now, leave aside for the moment the odd fact that the mayor is boasting about the appallingly low percentage of students in New York City who met the new proficiency standards after a decade of his control of the schools. The key point here is that the mayor, his Chancellor Dennis Walcott, Regents’ Chancellor Merryl Tisch, and State Commissioner John King all agreed that the state and city scores should be comparable to the NAEP “proficiency” level.

That is a huge mistake. It explains why the scores are invalid.

The state didn’t just “raise the bar.” It aligned its passing mark to a completely inappropriate model.

The state scores have four levels: level 4 is the highest, level 1 is the lowest. In the present scoring scheme, students who do not reach level 3 and 4 have “failed.”

NAEP has three levels: “Advanced” is the highest (only about 3-8% of students reach this level). “Proficient” is defined by the National Assessment Governing Board as “solid academic performance for each grade assessed. This is a very high level of academic achievement.”). “Basic” is “partial mastery” of the skills and knowledge needed at each grade tested.

“Proficient” on NAEP is what most people would consider to be the equivalent of an A. When I was a member of the NAEP governing board, we certainly considered proficient to be very high level achievement.

New York’s city and state officials have decided that NAEP’s “proficiency” level should be the passing mark.

They don’t understand that a student who is proficient on NAEP has attained “a very high level of academic achievement.”

Any state that expects all or most students to achieve an A on the state tests is setting most students up for failure.

If students need to reach “proficiency” just to pass, there will obviously be a very large number of students who “fail.”

B students and C students will fail.

The NAEP achievement levels have always been controversial. Many researchers and scholarly bodies have said they were unreasonably high and thus “fundamentally flawed.” That term “fundamentally flawed” occurs again and again in the literature of NAEP critics. This article by James Harvey is a good summary of these arguments.

Some on this blog have asked whether NAEP is a criterion-referenced test, and the answer is no. A criterion-referenced test is one that almost everyone can pass if they master the requisite skills. A test to get a drivers’ license is a criterion-referenced test. Anyone who studies the laws can pass the written test and qualify for a drivers’ license.

NAEP is not a criterion-referenced test. Massachusetts is the only state where as much as 50% of the students (and only in fourth grade) are rated proficient in reading. The NAEP tests are not designed to be criterion-referenced tests; they are a mix of questions that are easy, moderate, and difficult.

The achievement levels were created when Checker Finn was chair of NAGB. I think they are defensible if people understand that the achievement levels do not represent grade levels. If the public wants a measure of “grade level,” then “basic” probably comes closest to grade level. “Proficient” is not grade level; as NAGB documents state, it represents “a very high level of academic achievement.”

More important, the NAEP achievement levels were never intended to be measures of grade level, and New York officials are wrong to interpret them as such, especially when they mistakenly use “proficient” as the passing mark.

Any state that uses NAEP “proficient” as its definition of “grade level” is making a huge mistake; it will set the bar unreasonably high and will mislabel many students and misjudge the quality of many schools.

And that is exactly what happened in the New York testing fiasco.

If the state sticks to its present course of using NAEP “proficient” as its passing mark, it will encourage criticism of the Common Core standards as unrealistic and stoke parental outrage about Common Core testing.

People know their children, and they know their own school. The politicians may convince them that American education is floundering (even if it is not), but they can’t convince them that their own child and their own school are “failing” when parents know from their own experience that it is not true.

The corporate reformers now using the Shock Doctrine to bash the schools and disparage students may find that their tactic has backfired. They succeed only in adding fuel to the growing movement to stop the misuse of standardized testing.

What is happening in New York is likely to undermine public confidence in the state’s highest education officials and create new converts to the Opt-Out of Testing movement.

The Shock Doctrine may be a boomerang that helps to bring down the madness of No Child Left Behind, Race to the Top, Common Core, the Pearson empire, and every other part of the reformy enterprise.

New York may have inadvertently created by the most powerful recruiting tool for the Opt Out movement.

Categories Accountability, Common Core, NAEP, New York, Testing

Gary Rubinstein: Dramatic Collapse of Charter School Test Scores

By dianeravitch
August 9, 2013 //
44

The corporate reform movement has spun an elaborate narrative in which charter schools are the solution to our nation’s allegedly dreadful public schools. “Waiting for Superman” became their message, used to win new converts. And the Common Core tests were supposed to put the nail in the coffin, demonstrating the utter failure of public schools.

For the past several years, study after study has shown that charter schools o not get better test scores than public schools if they enroll similar students. But, the reformers say, New York City was the exception. There, reformers kept finding “miracle schools,” where every student succeeded. And the New York City Department of Education boasted about its careful screening process for selecting charter authorizers. Here, the reformers claimed, was the realization of charter superiority.

But the myth just exploded. The narrative is a hoax. The Common Core tests that were supposed to destroy public education devastated the charter sector. Stephanie Simon of Politico.com was first to notice that some celebrated charters like KIPP and Democracy Prep did worse than the public schools.

Now Gary Rubinstein examined performance for all charters in New York City and determined that the sector as a whole did worse than public schools on the Common Core tests.

In fact, the score collapse of the charter sector dwarfed that of the public sector. Gary writes:

“The most stunning example is the famed Harlem Village Academy which had 100% passing in 2012, but only 21% passing in 2013 for a 79% drop (you can see that sad dot all the way at the right of the scatter plot). Democracy Prep Harlem Charter, run and staffed by many TFAers, dropped 84% in 2012 to 13% in 2013. KIPP Amp dropped from 79% in 2012 to just 9% in 2013. The Equity Project (TEP) which pays $125,000 for the best teachers had finally gotten some test scores they can brag about with 76% in 2012, but that has now sunk to just 20% in 2013. The Bronx Charter School Of Excellence, which recently received money from a $4.5 million grant to help public schools emulate what they do, dropped from 96% in 2012 to 33% in 2013. So these are the schools that are the red ‘outliers’ hovering near the bottom right of the scatter plot. In general, the average charter school went down by 51 percentage points compared to 34 percentage points for the average public school. The most plausible explanation for charters dropping so much more than public schools is that their test prep methods were not sufficient for the more difficult tests. In other words “you’re busted.”

The reformer narrative just blew up.

Categories Charter Schools, Common Core, New York City, Testing

Why Should Parents Raise Money to Pay for Common Core Testing?

By dianeravitch
August 8, 2013 //
23

This comes from Jennifer Croslin Smith, a Tennessee parent and a founder of http://stoptntesting.com/. No more bake sales or raising money for school uniforms. Nope:

“We were just told tonight at our Back to School night that our PTO is foregoing raising money for iPads this year so we can instead purchase computers for Common Core testing which will begin in 2014-2015. We are lucky if we raise $20,000/year at ______–yet all of that money (from what I understand) is going to fund computers for standardized testing. The state mandated the adoption of Common Core so, as far as I am concerned, they can pay for the computers we need to take these tests. It is not fair that parents should pay for something that the state should be providing for already. What about those schools that don’t have parents wealthy enough to cover this unfunded mandate? What are they supposed to do? This makes me sick.”

Categories Common Core, Parents, Tennessee, Testing
« Older Entries
Newer Entries »