Archives for category: Standardized Testing

Not everyone who scores Pearson tests is hired from Craig’s List or Kelly Temps. Julie Campbell, a fifth-grade teacher in Néw York recently scored student responses. She stresses that she is not opposed to Common Core or to standardized tests, but she is very troubled by the kind of thinking that is rewarded in the tests.

 

 

Because she signed a confidentiality agreement, she does not discuss items on this year’s exams, but released questions from last year.

 

 

She writes:

 

 

“First things first, one of the most disturbing trends that I have found examining this year’s and last year’s (released) tests is a shift in thinking toward a kind of intellectual relativism. In other words, any claim that a student makes is correct if he or she substantiates it with some evidence. On the surface this doesn’t sound terribly problematic, but when you start to examine some of the anchor papers, the dilemma with this vein of thinking becomes shockingly apparent. The truth is, not all claims are correct and not all evidence is created equal. Making a feeble claim and using evidence out of context to support that claim is an all too common occurrence on these tests….

 

“According to Pearson “you choose what you think is right” is the first inference. The list of upsides and downsides is one detail. The student then uses an unrelated second detail about joining clubs and school and makes a second inference that you may really end up enjoying it. Formulaically speaking: inference + 2 details will always yield a correct answer[2]. What we have here is a confusing and clumsy answer to a confusing and clumsy question.

 

“One might argue that this way of scoring allows students to scrape up extra points and is actually a boon to teachers and students alike. It boosts scores! Hurrah!

 

“But in fact, it creates a terrifyingly slippery slope. I think about climate change deniers, the Creationist Museum in Kentucky that shows humans and dinosaurs roaming Earth side-by-side, 9-11 conspiracy theorists, and the Holocaust itself! Throughout history, people have made misguided claims and have supported their thinking with spurious details and evidence. Don’t our children deserve better?

 

“Another disturbing pattern that emerges as one reads the anchor responses for the ELA is what I call “The Easter Egg Hunt.” When it comes to short answer questions in particular, the question that is actually being posed rarely matches the answer required. The wordier the written response, the more likely it is that the student will stumble upon the correct answer, find the decorative egg. (Strategy!) Time after time there is a clandestine condition that must be met in order for an answer to get full credit – “Magic Words.” As my scoring instructor illustrated, it’s kind of like tossing all of the words into a bucket and looking for certain key phrases or ideas to float up to the top.”

 

 

The nitty-gritty of the scoring process demonstrates that we have outsourced the most important functions of education to a mega-corporation that is incapable of assessing critical thinking. No standardized test can,no matter who writes it or scores it. Standardization itself is antithetical to the intended result.

Most people have no idea about the privatization movement. They don’t know that the narrative of crisis (“our schools are failing, failing, failing”)–repeated again and again–is intended to clear the way for privatization.

Peter Greene explains the insidious plan here.

Step one, create a crisis.

Step two, take power away from the community, dissolve the local school board, give it to the mayor, the governor.

Step three: cash in.

When Congress returns from its Independence Day recess on July 7, the U.S. Senate is scheduled to take up bipartisan legislation to overhaul “No Child Left Behind.” The bill the Senate will consider significantly reduces federal test-based accountability requirements but continues the federal mandate to test every child every year in elementary and middle school. Grassroots pressure on your two U.S. Senators can make the federal government follow the lead of many states in reversing policies that encourage standardized exam overuse and misuse.

National Help Roll Back Federal Testing Overkill — Urge Your U.S. Senators to Support the Tester Amendment Now!
http://www.fairtest.org/roll-back-standardized-testing-send-letter-congres

Major Education Groups to Congress: Get Rid of “No Child” Law Already
http://www.washingtonpost.com/local/education/education-groups-to-congress-please-get-rid-of-no-child-left-behind-already/2015/06/22/3c3b094e-18ff-11e5-93b7-5eddc056ad8a_story.html

California Principal Backs Down From Punishing Students Whose Families Opted Out of State Tests
http://www.foxnews.com/us/2015/06/24/california-school-backtracks-on-common-core-opt-out-punishment/

Delaware Lawmakers Overwhelmingly Approve Opt Out Bill, Forward to Governor for Signature
http://www.delawareonline.com/story/news/education/2015/06/25/lawmakers-ok-letting-students-skip-standardized-testing/29287065/

District of Columbia How Standardized Tests Are Impeding Learning
http://greatergreaterwashington.org/post/27176/heres-how-standardized-tests-are-impeding-learning-in-dc/

Florida No Word on Penalties for Test Maker After Computer Exam Problems
http://www.tampabay.com/blogs/gradebook/still-no-word-on-penalties-for-air-over-floridas-spring-testing-woes/2234605

Florida School Board Says State Downplaying Extent of Computer Test Foul Up Impact
http://news.wfsu.org/post/leon-school-board-says-doe-downplaying-impact-spring-testing-glitches

Indiana District Supers Say Don’t Evaluate Schools With Unsound Test Data
http://www.jconline.com/story/opinion/readers/2015/06/26/op-ed-time-school-standards/29333899/

Massachusetts Support of High-Stakes Exams Misses the Immeasurable
http://www.bostonglobe.com/opinion/letters/2015/06/23/globe-support-testing-misses-unmeasurable/0QqPyTcRjoIpBcYoAVeabI/story.html?s_campaign=8315

Minnesota Testing Faces Big Cuts
http://www.mprnews.org/story/2015/06/29/minnesota-school-testing-cuts

Missouri Districts Still Waiting for Delayed Testing Results
http://www.semissourian.com/story/2207439.html

Montana Feds Should Not Punish State for Testing Company Screw-ups
http://missoulian.com/news/opinion/editorial/missoulian-editorial-don-t-penalize-montana-for-test-glitches/article_7d9f86cc-01de-55f2-aa65-79aac3a17d43.html

Montana Two Teachers Run for State Education Superintendent to Reign In Testing
http://mtpr.org/post/two-teachers-enter-race-montana-superintendent-public-instruction

New Jersey One Step Closer to Banning K-2 Standardized Tests
http://www.nj.com/education/2015/06/nj_one_step_closer_to_ban_on_k-2_standardized_test.html

New Mexico 10,000 Opted Out Statewide Because They Know the Score About PARCC Tests
http://www.taosnews.com/opinion/article_52aa75a2-176a-11e5-82bd-331359fb774d.html

New York Relaxes Gag Rule Preventing Teachers From Discussing Test Questions

New York Reasons to Be Hopeful for Assessment Reform
http://wamc.org/post/karen-magee-testing-and-evaluations-reasons-be-hopeful#stream/0

Ohio School Testing Metrics Punish Disadvantaged Districts and Students

No Child Left Behind’s school performance metrics may be punishing disadvantaged school districts and students.

Ohio Poised to Drop PARCC Test
http://blogs.edweek.org/edweek/state_edwatch/2015/06/ohio_poised_to_ditch_parcc_common-core_test_in_budget_sent_to_gov_kasich.html

Oregon Signs Bill Making Test Opt Outs Easier Despite Arne Duncan’s Threats
http://registerguard.com/rg/news/local/33225736-75/oregon-governor-signs-smarter-balanced-opt-out-law.html.csp

Oregon New Law Could Lead to More Opt Outs
http://ijpr.org/post/new-oregon-law-could-lead-more-testing-opt-outs

Pennsylvania Local School Board Passes Resolution Urging State to Ease Up on Testing
http://www.timesherald.com/general-news/20150623/phoenixville-area-school-district-calls-for-fewer-standardized-tests

Pennsylvania Spanish Speaking Students Say “No” to Standardized Tests
http://www.npr.org/2015/06/26/417870093/spanish-speaking-students-say-no-to-standardized-tests

Tennessee: Are Test Statistics a True Measure of Learning in Public Schools?

Are Statistics a True Measure of Learning in Schools?

Virginia Meeting Testing Goals Harder as Number of English Language Learners Soars
http://hamptonroads.com/2015/06/schools-weigh-how-meet-standards-el-students-rise

Washington Test-Makers Blame Scheduling Problems for Scoring Delay
http://www.seattletimes.com/education-lab/testing-group-poor-scheduling-to-blame-for-scoring-delays/
Washington Graduation at Risk for 2,000 Due to Political Stalemate on Testing
http://www.seattletimes.com/seattle-news/education/graduation-at-risk-for-2000-because-of-stalemate-on-testing/

University Admissions: June SAT Timing/Scoring Error Leads to Calls for Summer Retest, Refunds, and Rebates
http://www.news-press.com/story/news/2015/06/25/sat-scores-june-validity-questions-retake-refund/29294517/

Bob Schaeffer, Public Education Director
FairTest: National Center for Fair & Open Testing
office- (239) 395-6773 fax- (239) 395-6779
mobile- (239) 699-0468
web- http://www.fairtest.org

A reader shares this information with us. The state will administer assessments to children in kindergarten but parents have the right to opt out. Will the parents know? If you live in Washington, make sure you inform parents of their right to opt out their children from this unnecessary assessment. Let the children play.

 

 

So here in the Pacific North West, our legislature just finished a special session and they’re now into the third. One part of our legislature, the House, just passed Washington HB 1491…”Expand Early Childhood Education Across the State” and has nice fine print related to testing kindergarteners and 3-4 year-olds. That’s right. TESTING. FOR KINDERGARTENERS. Section 2, [2][a] “Improve short-term and long-term educational outcomes for children as measured by assessments including, but not limited to, the Washington kindergarten inventory of developing skills in RCW 28A.655.080. The only saving grace? RCW 28A.150.315 is in force WITH THE EXCEPTION OF STUDENTS WHO HAVE BEEN EXCUSED FROM PARTICIPATION BY THEIR PARENTS OR GUARDIANS. That’s right. The K-12 public schools will now HAVE to administer the test with all the attendant costs and time suck but, as a parent, you won’t be hearing how you can opt-out. Lots of more work to do. Then again, I’m so pissed about it that it gives me the energy I need to fight, fight, fight.

You might (or might not) enjoy watching this 7-minute interview I did with former teacher Bob Greenberg. Bob has created a large archive of interviews like this one. It was filmed in my living room. The collection is called “The Brainwaves.” He sets up a camera and says “talk.” He doesn’t ask questions or interrupt.

The National Assessment of Educational Progress (NAEP) has been measuring national samples of students in grades 4 and 8 (and sometimes 12) since the early 1970s. It has been measuring state samples since 1992, and began assessing a few urban districts in 2003. It assesses students every two years in reading and math, and every several years in history, science, civics, and other subjects.

 

NAEP has always collected background information, which is self-reported about students’ reading habits, television viewing time, teacher practices, and other quantifiable aspects of tea hing and learning.

 

Now, NAEP will add grit, motivation, and mindset to the background information collected.

 

It will be interesting to see how these noncognitive traits are measured. Will students judge their own grit? Are they good judges of their grit? Will we someday know which states and cities have students with the most grit? And once we know, will officials create courses in how to improve grit?

 

I am reminded of a strange finding that emerged from international background questions two decades ago. Students were asked if they were good in math. Students in nations with the highest test scores said they were not very good in math; students in nations where test scores were middling thought they were really good at math.

 

What does it all mean? I don’t know, but it satisfies someone’s need for more data.

Imagine this great victory for teachers in New York: They will now be allowed to discuss test questions that have been released to the public!

 

Is this progress? No. Suppose teachers spot unreleased questions that are clearly wrong, poorly worded, confusing, incoherent. If they have not been released to the public, the teachers are not allowed to criticize them or call attention to errors.

 

Peter Greene wondered if the New York Times recognized the absurdity of its headline, which claimed that the state was going to “relax” the gag order.

 

He wrote:

 

 

See, now the state will allow teachers to discuss items on the test after they have been publicly released, whereas previously, teachers could only discuss test items after they had been publicly released.

 

The gag order protects Pearson. If the gag order prevailed, we would have never known about the nutty question on a Pearson test about “the pineapple and the hare.” That question was not publicly released. It became public not because of teachers but because students complained about it, and it leaked to the New York City Parent blog.

 

This “gag order” is insulting to teachers. It should be eliminated. Its only purpose is to protect the interests of the testing companies. They should release all their questions. No one will know which will be on future tests. If there are thousands of test questions available, students can use them to see what is expected of them. And if they are released, parents and teachers will have a chance to evaluate their quality. That may be what scares the testing companies most.

 

Take off the gag!

 

 

Mercedes Schneider posted a letter written by a Néw York algebra teacher to parents of his students.

He begins:

“Dear Algebra Parents,

 

“The results from this year’s Common Core Algebra exam are now available and have been posted on the high school gymnasium doors. They are listed by student ID number and have no names attached to them. The list includes all students who took the exam, whether they were middle school students or high school students.

 

“I’ve been teaching math for 13 years now. Every one of those years I have taught some version of Algebra, whether it was “Math A”, “Integrated Algebra”, “Common Core Algebra”, or whatever other form it has shown up in. After grading this exam, speaking to colleagues who teach math in other school districts, and reflecting upon the exam itself, I have come to the conclusion that this was the toughest Algebra exam I have ever seen.

 

“With that in mind, please know that all 31 middle school students who took the exam received a passing score. No matter what grade your son or daughter received, every student should be congratulated on the effort they put into the class this year.

 

“Although everyone passed, many of you will not be happy with the grade that your son or daughter received on the exam (and neither will they). While I usually try to keep the politics of this job out of my communications, I cannot, in good conscience, ignore the two-fold tragedy that unfolded on this exam. As a parent, you deserve to know the truth.

 

“I mentioned how challenging this exam was, but I want you to hear why I feel this way.”

Les Perelman, former director of undergraduate writing at MIT has been a persistent critic of machine-scored writing on tests. He has previously demonstrated that students can outwit the machines and can game the system. He created a machine called BABEL, or Basic Automatic B.S. Essay Language Generator. He says that the computer cannot distinguish between gibberish and lucid writing.

 

He wrote the following as a personal email to me, and I post it with his permission.

 

Measurement Inc., which uses Ellis Paige’s PEG (Project Essay Grade) software to grade papers all but concedes that students in classrooms where the software has been used have been using the BABEL generator or something like it to game the program. Neither vendor mentions that the same software is also being used to grade high stakes state tests, and in the case of Pearson, is being considered by PARCC to grade Common Core essays.

 

http://www.pegwriting.com/qa#good-faith

 

What is meant by a “good faith” essay?

 

 

It is important to note that although PEG software is extremely reliable in terms of producing scores that are comparable to those awarded by human judges, it can be fooled. Computers, like humans, are not perfect.

 

PEG presumes “good faith” essays authored by “motivated” writers. A “good faith” essay is one that reflects the writer’s best efforts to respond to the assignment and the prompt without trickery or deceit. A “motivated” writer is one who genuinely wants to do well and for whom the assignment has some consequence (a grade, a factor in admissions or hiring, etc.).

 

Efforts to “spoof” the system by typing in gibberish, repetitive phrases, or off-topic, illogical prose will produce illogical and essentially meaningless results.

 

Also, both PEG Writer and Pearson’s WriteToLearn concede in buried FAQ’s that their probabilistic grammar checkers don’t work very well.

 

PEG Writing by Measurement Inc.
http://www.pegwriting.com/qa#grammar

 

PEG’s grammar checker can detect and provide feedback for a wide variety of syntactic, semantic and punctuation errors. These errors include, but are not limited to, run-on sentences, sentence fragments and comma splices; homophone errors and other errors of word choice; and missing or misused commas, apostrophes, quotation marks and end punctuation. In addition, the grammar checker can locate and offer feedback on style choices inappropriate for formal writing.

 

Unlike commercial grammar checkers, however, PEG only reports those errors for which there is a high degree of confidence that the “error” is indeed an error. Commercial grammar checkers generally implement a lower threshold and as a result, may report more errors. The downside is they also report higher number of “false positives” (errors that aren’t errors). Because PEG factors these error conditions into scoring decisions, we are careful not to let “false positives” prejudice an otherwise well constructed essay.

 

Pearson Write to Learn
http://doe.sd.gov/oats/documents/WToLrnFAQ.pdf

 

The technology that supports grammar check features in programs such as Microsoft Word often return false positives. Since WriteToLearn is an educational product, the creators of this program have decided, in an attempt to not provide students with false positives, to err on the side of caution. Consequently, there are times when the grammar check will not catch all of a student’s errors.

 

MS Word used to produce a significant number of false positives but Microsoft in the current versions appears to have raised the probabilistic threshold so that it now underreports errors.

It is a universal truth, well known, that when budget cuts are imposed by the state, teachers of the arts are the first to go. I recently met with a leader of the arts community in Houston who told me that she wanted to make a gift of art supplies but could not few elementary schools with art teachers.

Some advocates for the arts–music education, especially, claim that the study of music increases test scores.

Peter Greene says: Don’t do that! See here too.

There so many important reasons to treasure music, and the pursuit of higher test scores is not one of them.

“Music is universal. It’s a gabillion dollar industry, and it is omnipresent. How many hours in a row do you ever go without listening to music? Everywhere you go, everything you watch– music. Always music. We are surrounded in it, bathe in it, soak in it. Why would we not want to know more about something constantly present in our lives? Would you want to live in a world without music? Then why would you want to have a school without music?
Listening to music is profoundly human. It lets us touch and understand some of our most complicated feelings. It helps us know who we are, what we want, how to be ourselves in the world. And because we live in an age of vast musical riches from both past and present, we all have access to exactly the music that suits our personality and mood. Music makes the fingers we can use to reach into our own hearts.
Making music is even more so. With all that music can do just for us as listeners, why would we not want to unlock the secrets of expressing ourselves through it? We human beings are driven to make music as surely as we are driven to speak, to touch, to come closer to other humans. Why would we not want to give students the chance to learn how to express themselves in this manner?….

“In music, everyone’s a winner. In sports, when two teams try their hardest and give everything they’ve got, there’s just one winner. When a group of bands or choirs give their all, everybody wins. Regrettably, the growth of musical “competitions” has led to many programs that have forgotten this — but music is the opposite of a zero-sum game. The better some folks do, the better everybody does. In music, you can pursue excellence and awesomeness without having to worry that you might get beat or defeated or humiliated. Everybody can be awesome….

“Do not defend a music program because it’s good for other things. That’s like defending kissing because it gives you stronger lip muscles for eating soup neatly. Defend it because music is awesome in ways that no other field is awesome. Defend it because it is music, and that’s all the reason it needs. As Emerson wrote, “Beauty is its own excuse for being.” A school without music is less whole, less human, less valuable, less complete. Stand up for music as itself, and stop making excuses.”