Archives for category: Testing

Anemona reports in the New York Times that the College Board has abandoned its plans to deliver ((sell) an online SAT. More than 1200 colleges and universities are now test-optional. The University of California’s decision a few weeks ago to forego thevSAT as an admission requirement was a huge blow to the College Board’s business plans.

The College Board said on Tuesday that it would postpone plans to offer an online version of the SAT for high school students to take at home this year, further muddying a ritual of the college application process that had already been thrown into chaos by the coronavirus.

After canceling test dates this spring, the board announced in mid-April that it was developing a digital version of the SAT to be introduced if the pandemic continued to require social distancing in the fall, which would make it hard for the nonprofit organization to provide enough testing dates and centers.

But in its latest statement, the board said the technological challenges of developing an online test that all students could take had led to the decision to drop it. Some 2.2 million students took the SAT last year, the College Board said.

“Taking it would require three hours of uninterrupted, video-quality internet for each student, which can’t be guaranteed for all,” the board said, acknowledging the technology gap facing lower-income students, which could further exacerbate inequities in access to higher education.

The organization added that it would continue to deliver an online version of the SAT at some schools, but would not “introduce the stress that could result from extended at-home testing in an already disrupted admissions season.”

Bob Schaeffer, the head of FairTest, which is opposed to the use of standardized tests in college admissions, said the College Board was “simply conceding the inevitable.”

Its decision came after the organization had a rocky experience last month introducing a digital version of the Advanced Placement exams, which it also oversees. Many students complained that they were not able to submit their answer sheets electronically, and their tests were disqualified.

Mr. Schaeffer’s group and several students and parents have filed a class-action lawsuit seeking to force the College Board to score the rejected answer sheets. The College Board said less than 1 percent of students who had taken the test were affected.

The College Board asked colleges and universities on Tuesday to “show flexibility” to the millions of students who were not able to take the SAT this spring because of cancellations. It asked colleges to extend deadlines for receiving test scores, and to give equal consideration to students who were unable to take the test because of the pandemic.

The SAT’s rival exam, the ACT, said on Tuesday that it still planned to offer a remote option in the fall.

David Berliner, one of our nation’s most eminent researchers, advises parents not to worry that their children are “falling behind.” School is important. Instruction is important. But “soft skills” and non—cognitive skills matter more in the long term than academic skills. Relax.

He sent this advice to the blog:

Worried About Those “Big” Losses on School Tests Because Of Extended Stays At Home? They May Not Even Happen,
And If They Do, They May Not Matter Much At All!

David C. Berliner
Regents Professor Emeritus
Mary Lou Fulton Teachers College
Arizona State University
Tempe, AZ.

Although my mother passed away many years ago, I need now to make a public confession about a crime she committed year in and year out. When I was young, she prevented me from obtaining one year of public schooling. Surely that must be a crime!

Let me explain. Every year my mother took me out of school for three full weeks following the Memorial Day weekend. Thus, every single year, from K through 9th grade, I was absent from school for 3 weeks. Over time I lost about 30 weeks of schooling. With tonsil removal, recurring Mastoiditis, broken bones, and more than the average ordinary childhood illnesses, I missed a good deal of elementary schooling.
How did missing that much schooling hurt me? Not at all!

First, I must explain why my mother would break the law. In part it was to get me out of New York City as the polio epidemic hit U.S. cities from June through the summer months. For each of those summers, my family rented one room for the whole family in a rooming house filled with working class families at a beach called Rockaway. It was outside the urban area, but actually still within NYC limits.

I spent the time swimming every day, playing ball and pinochle with friends, and reading. And then, I read some more. Believe it or not, for kids like me, leaving school probably enhanced my growth! I was loved, I had great adventures, I conversed with adults in the rooming house, I saw many movies, I read classic comics, and even some “real” literature. I read series after series written for young people: Don Sturdy, Tom Swift, the Hardy Boys, as well as books by Robert Louis Stevenson and Alexander Dumas.

So now, with so many children out of school, and based on all the time I supposedly lost, I will make a prediction: every child who likes to read, every child with an interest in building computers or in building model bridges, planes, skyscrapers, autos, or anything else complex, or who plays a lot of “Fortnite,” or “Minecraft,” or plays non-computer but highly complex games such as “Magic,” or “Ticket to Ride,” or “Codenames” will not lose anything measurable by staying home. If children are cared for emotionally, have interesting stuff to play with, and read stories that engage them, I predict no deficiencies in school learning will be detectable six to nine months down the road.
It is the kids, rich or poor, without the magic ingredients of love and safety in their family, books to engage them, and interesting mind-engaging games to play, who may lose a few points on the tests we use to measure school learning. There are many of those kinds of children in the nation, and it is sad to contemplate that.

But then, what if they do lose a few points on the achievement tests currently in use in our nation and in each of our states? None of those tests predict with enough confidence much about the future life those kids will live. That is because it is not just the grades that kids get in school, nor their scores on tests of school knowledge, that predict success in college and in life. Soft skills, which develop as well during their hiatus from school as they do when they are in school, are excellent predictors of a child’s future success in life.

Really? Deke and Haimson (2006), working for Mathmatica, the highly respected social science research organization, studied the relationship between academic competence and some “soft” skills on some of the important outcomes in life after high school. They used high school math test scores as a proxy for academic competency, since math scores typically correlate well with most other academic indices. The soft skills they examined were a composite score from high school data that described each students’ work habits, measurement of sports related competence, a pro-social measure, a measure of leadership, and a measure of locus of control.

The researchers’ question, just as is every teacher’s and school counselor’s question, was this: If I worked on improving one of these academic or soft skills, which would give that student the biggest bang for the buck as they move on with their lives?

Let me quote their results (emphasis by me)
Increasing math test scores had the largest effect on earnings for a plurality of the students, but most students benefited more from improving one of the nonacademic competencies. For example, with respect to earnings eight years after high school, increasing math test scores would have been most effective for just 33 percent of students, but 67 percent would have benefited more from improving a nonacademic competency. Many students would have secured the largest earnings benefit from improvements in locus of control (taking personal responsibility) (30 percent) and sports-related competencies (20 percent). Similarly, for most students, improving one of the nonacademic competencies would have had a larger effect than better math scores on their chances of enrolling in and completing a postsecondary program.

​This was not new. Almost 50 years ago, Bowles and Gintis (1976), on the political left, pointed out that an individual’s noncognitive behaviors were perhaps more important than their cognitive skills in determining the kinds of outcomes the middle and upper middle classes expect from their children. Shortly after Bowles and Gintis’s treatise, Jencks and his colleagues (1979), closer to the political right, found little evidence that cognitive skills, such as those taught in school, played a big role in occupational success.

Employment usually depends on certificates or licenses—a high school degree, an Associate’s degree, a 4-year college degree or perhaps an advanced degree. Social class certainly affects those achievements. But Jenks and his colleagues also found that industriousness, leadership, and good study habits in high school were positively associated with higher occupational attainment and earnings, even after controlling for social class. It’s not all about grades, test scores, and social class background: Soft skills matter a lot!

Lleras (2008), 10 years after she studied a group of 10th grade students, found that those students with better social skills, work habits, and who also participated in extracurricular activities in high school had higher educational attainment and earnings, even after controlling for cognitive skills! Student work habits and conscientiousness were positively related to educational attainment and this in turn, results in higher earnings.

It is pretty simple: students who have better work habits have higher earnings in the labor market because they are able to complete more years of schooling and their bosses like them. In addition, Lleras’s study and others point to the persistent importance of motivation in predicting earnings, even after taking into account education. The Lleras study supports the conclusions reached by Jencks and his colleagues (1979), that noncognitive behaviors of secondary students were as important as cognitive skills in predicting later earnings.
So, what shall we make of all this? I think poor and wealthy parents, educated and uneducated parents, immigrant or native-born parents, all have the skills to help their children succeed in life. They just need to worry less about their child’s test scores and more about promoting reading and stimulating their children’s minds through interesting games – something more than killing monsters and bad guys. Parents who promote hobbies and building projects are doing the right thing. So are parents who have their kids tell them what they learned from watching a PBS nature special or from watching a video tour of a museum. Parents also do the right thing when they ask, after their child helps a neighbor, how the doing of kind acts makes their child feel. This is the “stuff” in early life that influences a child’s success later in life even more powerfully than do their test scores.

So, repeat after me all you test concerned parents: non-academic skills are more powerful than academic skills in life outcomes. This is not to gainsay for a minute the power of instruction in literacy and numeracy at our schools, nor the need for history and science courses. Intelligent citizenship and the world of work require subject matter knowledge. But I hasten to remind us all that success in many areas of life is not going to depend on a few points lost on state tests that predict so little. If a child’s stay at home during this pandemic is met with love and a chance to do something interesting, I have little concern about that child’s, or our nation’s, future.

Bowles, S., & Gintis, H. (1976). Schooling in Capitalist America. New York: Basic Books.

Deke, J. & Haimson, J. (2006, September). Expanding beyond academics: Who benefits and how? Princeton NJ: Issue briefs #2, Mathematica Policy Research, Inc. Retrieved May 20, 2009 from:http://www.eric.ed.gov:80/ERICDocs/data/ericdocs2sql/content_storage_01/0000019b/80/28/09/9f.pdfMatematicapolicy research Inc.

Lleras, C. (2008). Do skills and behaviors in high school matter? The contribution of noncognitive factors in explaining differences in educational attainment and earnings. Social Science Research, 37, 888–902.

Jencks, C., Bartlett, S., Corcoran, M., Crouse, J., Eaglesfield, D., Jackson, G., McCelland, K., Mueser, P., Olneck, M., Schwartz, J., Ward, S., and Williams, J. (1979). Who Gets Ahead?: The Determinants of Economic Success in America. New York: Basic Books.

A few days ago, I had a Zoom meeting with educators at Rutgers University, where I was invited to talk about education and social justice. Of course we talked about the pandemic and what happens next. But the theme of the day was equity.

I hope you enjoy it.

More than 200 advocates of public education endorsed this open letter to Joe Biden, which was published on Valerie Strauss’s blog “The Answer Sheet” at the Washington Post.

They call on presumptive Democratic nominee Biden to reject the stale and failed policies of the past 20 years.

Their letter (our letter, since I signed it) begins with this preamble and then offers a list of specific proposals that together represent a fresh vision for American education:

Dear Vice President Biden:


As the Democratic Party presumptive nominee, you have the power to fight for the public schools and colleges and universities that our students deserve. We are concerned educators, public education advocates, union members, parents, and students, writing to request that you demonstrate your commitment to that agenda.


Over the past decade, politicians on both sides of the aisle have made devastating cuts to public education, while privatizing public schools, scapegoating educators, and providing massive tax breaks to corporations and the rich.

These attacks have resulted in a national teacher shortage and reduced educational opportunities for many of our students — especially students of color, those from low-income households, LGBTQ students, and students with disabilities.


The public health and economic emergencies resulting from the COVID-19 pandemic have only made public education more vulnerable. It is no exaggeration to say that the future of public education itself is at stake.

Read the list of sensible, research-based, intelligent policy proposals, which would inspire students and teachers and improve education for all students.

Please open and read.

Tweet with the hashtag #EducationIsAPublicGood and @JoeBiden

Paul Tough has written several books, including most recently, “The Years That Matter Most: How College Makes or Breaks Us.” He also wrote a book about Geoffrey Canada and the Harlem Children’s Zone, and the best-selling “How Children Succeed.”

In this article in the New York Times, Tough explains that the decision by the University of California to drop the SAT may be the beginning of the end for that test. And it’s a good thing.

He writes:

If you’re a college student (or an aspiring one) from a financially struggling family, the coronavirus pandemic has brought with it a steady downpour of bad news: closed campuses, slashed financial-aid budgets and, coming soon, big cuts in state funding for public colleges and universities. But through these dark clouds one ray of more hopeful news has shone. Standardized admissions tests, which many aspiring low-income students see as the greatest barrier to their college goals, are being eliminated this spring as entrance requirements by one institution after another.

At first, the list of colleges deciding during the pandemic to go “test-optional” (meaning that applicants can choose whether or not to submit test scores) included mostly small private institutions — Williams, Amherst, Tufts, Vassar — and the decisions were often presented merely as temporary changes or pilot projects.

But last week brought much bigger news: Janet Napolitano, the president of the University of California, recommended to the system’s Board of Regents that the entire U.C. system go test-optional for the next two years, followed by two years during which the university would become not just test-optional but “test-blind.” In 2023 and 2024, Ms. Napolitano proposed, Berkeley and U.C.L.A. and every other U.C. school wouldn’t consider SAT or ACT scores at all in their admissions decisions.

The university administration, Ms. Napolitano explained, would spend these years trying to come up with its own better and fairer standardized admission test. If it failed, U.C. wouldn’t go back to accepting the SAT and ACT; instead, it would eliminate the consideration of standardized tests in admissions for California students once and for all.

This was a sweeping proposal, especially for such an influential institution as the University of California. And what was so surprising about Ms. Napolitano’s recommendations — which will be put to a vote by the Board of Regents on Thursday — was that they came less than a month after the university’s faculty senate had unanimously accepted the report of a task force supporting the continued use of the tests and proposing to keep them in place for at least the next nine years.

If the Regents concur with Ms. Napolitano this week, it will be a crucial turning point in a national debate about standardized testing that has been going on for decades. Do standardized tests help smart, underprivileged college applicants? Or do they hurt them?

Proponents of standardized tests often make the case that the tests are the least unfair measure in a deeply unfair system. It’s certainly true that the system is unfair from start to finish. Rich kids enjoy advantages over poor kids that begin in prenatal yoga sessions and continue through summer tennis camps, after-school robotics classes and high-priced college-essay coaching sessions. But the data show that standardized tests don’t level that playing field; they skew it even further.

The best predictor of college success overall is a simple one: high school grades. This makes a certain sense. An impressive high school G.P.A. reflects a combination of innate talent and dedicated hard work, and that’s exactly what you need to excel in college. And while standardized test scores have long been found to be highly correlated with students’ financial status, that’s much less true with high school G.P.A. In a recent study, Saul Geiser, a researcher at Berkeley, found that the correlation between family income and SAT scores among University of California applicants is three times as strong as the correlation between their family income and their high school G.P.A.

You can see the same pattern when you look at applicants by race. When Mr. Geiser used high school G.P.A. to identify the top 10 percent of Californians applying for admission to the U.C. system, 23 percent of the pool was black or Latino. When he used SAT scores to identify the top 10 percent, 5 percent was black or Latino.

Here’s another way to look at the numbers: The students who are most likely to benefit from any university’s decision to eliminate the use of standardized tests are those who have high G.P.A.s in high school but comparatively low standardized test scores. These are, by definition, hard-working and diligent students, but they don’t perform as well on standardized tests. Let’s call them the strivers.

A few years ago, researchers with the College Board, the organization that administers the SAT, analyzed students in that cohort and compared them with their mirror opposites: those with relatively high test scores and relatively low high school G.P.A.s. Let’s call them the slackers: self-assured test takers who for one reason or another didn’t put as much effort into high school.

The College Board’s researchers made two important discoveries about these groups. First, there were big demographic differences between them. The slackers with the elevated SAT scores were much more likely to be white, male and well-off. And the strivers with the elevated high school G.P.A.s were much more likely to be female, black or Latina, and working-class or poor.

The researchers’ second discovery was that students in the striver cohort, despite their significant financial disadvantages, actually did a bit better in college. They had slightly higher freshman grades and slightly better retention rates than the more affluent, higher-scoring slackers.

Despite the persistent and compelling evidence that standardized tests penalize low-income students, a lot of us want to believe the opposite: that standardized tests are the tool that can help selective colleges pluck brilliant low-income students out of low-performing high schools. These Cinderella stories do sometimes happen, and when they do, they’re inspiring. But these anecdotal exceptions are overwhelmed by the experience of a large majority of ambitious low-income students, for whom standardized tests have the opposite effect: They construct a wall that separates them from prestigious universities, a wall with a narrow doorway that only well-off kids seem to know how to squeeze through.

If the Board of Regents approves Ms. Napolitano’s recommendations, it won’t get rid of all the structural barriers standing in the way of California’s striving low-income students. Not by a long shot. But it will have taken an important step toward making that wall a little lower and that doorway a little wider.

Peter Greene recognizes one of the great education heroes of our age, Dr. Lester Perelman, who retired a few years ago from MIT, where he taught writing. Les Perelman carefully and thoroughly debunked “robograding” of student essays. ETS had a robograder that allegedly graded thousands of essays in a minute or less.

Perelman showed that students could write nonsensical paragraphs containing blatant inaccuracies yet get a high score from the robograder.

Greene points out that Perelman singlehandedly shot down Australia’s plan to adopt robograding for student essays.

Perelman reviewed the Australian writing assessment and summarized how to get a high test score:

Learn a bunch of big spelling words, and throw them in. Don’t worry about meaning, but do worry about spelling them correctly. Repeat the ideas in the prompt often.

Five paragraph essay all the way. Every paragraph should be four sentences; don’t worry about repeating yourself to get there. Start the last paragraph with “In conclusion,” then repeat your thesis from graph #1. Somewhere work in a sentence with the structure “Although x (sentence), y (sentence). (Perelman’s example– Although these instructions are stupid, they will produce a high mark on the NAPLAN essay.)

Use “you” and ask questions. Use connectors like “moreover” or “however.” Start sentences with “In my opinion” or “I believe that” (not for the first or last time, Strunk and White are spinning in their graves). Repeat words and phrases often, and throw in passive voice (whirrrrr). Throw in one or two adjectives next to nouns.

For narrative essays, just steal a story from a movie or tv show– markers are explicitly instructed to ignore that they recognize a story.

And the final and most important rule– never write like this except for essay tests like the NAPLAN.

For his role in junking the Australian fascinating with robograding and helping to undermine its obsession with national testing, Perelman was honored by the New South Wales Teachers Federation as a “Champion of Public Education.”

In his acceptance speech, Perelman said:

Free public education is the cornerstone of a stable democratic and free society.

The main problem with edu-business [for profit entities in education] is that the most important products of education, such as critical thinking and analysis, are both the least tangible and the least profitable. They are expensive both in staffing and in assessment. Edu-business wants to MacDonald-ize education, make it cheap to produce and distribute, highly profitable and with little nutritive value. It wants, like Dickens’ Gradgrind, to focus on relatively unimportant facts and rules that can literally be mechanically taught and mechanically counted. Edu-business values psychometricians over practitioners, testers over teachers, reliability over validity.

Peter Greene observed:

It’s a little long for a t-shirt, but it might be worth the effort.

Valerie Strauss writes in the Washington Post about a class-action lawsuit filed against the College Board:


A class-action lawsuit has been filed in federal court on behalf of students who took online Advanced Placement tests last week and ran into technical trouble submitting their answers. It demands that the College Board score their answers instead of requiring them to retake the test in June, and provide hundreds of millions of dollars in monetary relief.


The lawsuit, dated Tuesday, says that students’ inability to submit answers was the fault of the exam creators, and it charges that the College Board engaged in a number of “illegal activities,” including breach of contract, gross negligence, misrepresentation and violations of the Americans With Disabilities Act. It also seeks more than $500 million in compensatory damages as well as punitive damages.


The College Board owns the AP program, although the AP tests are created and administered by the Educational Testing Service. Both of those organizations were named as defendants in the lawsuit, which was filed in a U.S. District Court in California.


Peter Schwartz, College Board Chief risk officer and general counsel, said in a statement: “This lawsuit is a PR stunt masquerading as a legal complaint being manufactured by an opportunistic organization that prioritizes media coverage for itself. It is wrong factually and baseless legally; the College Board will vigorously and confidently defend against it, and expect to prevail.”
He also said, “When the country shut down due to coronavirus, we surveyed AP students nationwide, and an overwhelming 91 percent reported a desire to take the AP Exam at the end of the course. Within weeks, we redesigned the AP Exams so that they could be taken at home. Nearly 3 million AP Exams have been taken over the first seven days. Those students who were unable to successfully submit their exam can still take a makeup and have the opportunity to earn college credit.”


The College Board said last week that it had found the problems students faced submitting answers were largely caused by outdated browsers and students’ failure to see messages announcing the end of an exam.

This is the first time that AP tests have been given online at home, a result of the shutdown of schools because of the coronavirus pandemic. The tests were previously given at school.

But the College Board said it had surveyed students and that most wanted to take the tests online, noting that the scores can factor into college admissions decisions and that students can receive college credit for high scores. The online tests, in numerous subjects, were shortened from several hours to 45 minutes.


Critics had warned that online testing is not fair to students who have no computer, access to Internet or quiet work spaces from which to study and work, or to students with disabilities who do not have appropriate accommodations — challenges the College Board acknowledged and said it tried to ameliorate. Critics also questioned the validity of the shortened exams.


The lawsuit was filed by parents on behalf of students who could not submit answers, as well as by the National Center of Fair and Open Testing, a nonprofit organization known as FairTest that works to end the misuse of standardized tests. (The lawsuit cites a post on The Answer Sheet blog with news about the problems students were facing.)
“

The College Board was warned about many potential access, technology and security problems by FairTest and other groups that had documented crashes when other computerized tests were introduced,” said FairTest interim director Bob Schaeffer. “Nevertheless, the board rushed ‘untested’ AP computerized exams into the marketplace in order to preserve its largest revenue-generating program when they could no longer administer in-school tests.”




The College Board, a nonprofit organization that operates substantially like a business, said that students last week took 2.186 million AP exams in various subjects during the first week of the two-week May testing window, and that “less than 1 percent of students were unable to submit their responses.”


The College Board did not provide the exact number of students who had problems but did note in an email that some students took more than one test. That makes it impossible for the public to know exactly how many students were affected.


Most of the students who had problems found that they could not submit all or some of their answers. Many took photos or videos of their responses, but the College Board told them their responses could not be scored and that they would have to retake their exams in June.


Then, on Sunday, the College Board announced that students taking exams during this week of testing could email responses if they found they had trouble submitting. Students who took the tests last week, however, could not submit their answers for scoring and still had to retake them in June.

The lawsuit asks that the College Board accept any test answers from last week’s AP tests that can be shown to have been completed in time by time stamp, photo and email.

It charges that the College Board ignored warnings that giving AP tests online would discriminate against students with disabilities and those who did not have access to technology or the Internet at home to take the exams.

The plaintiffs are seeking compensatory damages of more than $500 million and “punitive damages in an amount sufficient to punish defendants” and “to deter them from engaging in wrongful conduct in the future.”

The suit was filed by Phillip A. Baker from Baker, Keener & Nahra LLP in Los Angeles and Marci Lerner Miller from Miller Advocacy Group in Newport Beach.

Nancy Bailey is well aware of the dangers to public education today, especially the threats of privatization, data mining, and technological takeover. She saw that the campaigns of Joe Biden and Bernie Sanders created an education unity group and she wondered who was included and who was not included.

Here is her analysis.

She begins with who was left out:

Many want to say good riddance to Education Secretary Betsy DeVos and her boss. But educators and parents fighting for public education, and the ninety percent of students who attend public schools, deserve a more inclusive group of people to push back on harmful school reform. The Biden/Sanders Unity Education Task Force leaves much to be desired.

For example, parents of children with disabilities struggle to teach their children during Covid-19. Classes for their children were never fully funded before the disease. Sen. Bernie Sanders promised better in his Thurgood Marshall Plan. Searching with a magnifying glass, I see no representation for students with disabilities on this panel.

Black and brown parent advocates have started a petition to make the education task force more inclusive.

Where are the scholars from the: National Education Policy Center? Network for Public Education? Defending the Early Years? Economic Policy Center? Where are teachers from the Badass Teachers Association, or representation by those who organized and marched in the Red for Ed rallies? What about parents and school board members who fight for children?

By now, almost everyone knows that the College Board offered a shortened version of AP exams–only 45 minutes–and that thousands of students took the exams at home, online, only to have their answers rejected. When asked about this phenomenon, which was so deeply upsetting to the affected students, the College Board responded that the problem was the students’ browsers. Some students (including one who commented on this blog) pointed out that they took more than one AP exam, and their answers were accepted for one exam, but rejected for the second or third.

Mercedes Schneider writes that something is wrong with the College Board, not the students’ browsers. This is not their first technical failure, nor is it likely to be the last. The College Board says that 99% of the students who took AP exams submitted their answers successfully, but we have to take their word for it.

Should we?

Randi Weingarten and I talked about what happens next: after the pandemic, how we protect schools and children from “opportunistic” tech entrepreneurs, what does Cuomo have up his sleeve, can we trust Biden to ditch Race to the Top bogus ideas?

Our conversation was recorded and live-streamed by the Network for Public Education. Carol Burris introduced us. The conversation wa facilitated by Darcie Cimarusti and Marla Kilfoyle, the fabulous staff of NPE.