Archives for category: Research
« Older Entries
Newer Entries »

“Democracy”: What Does Research Say About AP Courses?

By dianeravitch
July 3, 2017 //
48

We have had an interesting conversation on the blog about the value of AP courses. It was tied to Jay Mathews’ use of AP courses to rank the quality of high schools: the more AP courses, the better the high school.

I have made clear two points: One, when I was in high school in the 1950s, there were no AP courses, so I have had no experience with them; and my children graduated high school without ever taking an AP course. This, I have no personal experience with AP. Two, I strongly object to the College Board marketing AP courses on the spurious grounds that they promote equity and civil rights. The College Board is making millions by doing so. It should be as honest as those selling cars, beer, and cigarettes.

Our friend and reader “Democracy” posted this comment:

“It sure is interesting that the pro-AP commenters on this thread do not – and cannot – cite any solid evidence that Advanced Placement is any more than hype. To be sure – there are good AP teachers. Also to be sure – as a program – AP just is NOT what the proponents claim. Far from it.

Much of the AP hype that exist in the U.S. can be traced to Jay Mathews, who slobbers unabashedly over the “merits” of AP. Mathews not only misrepresents the research on AP but also publishes the yearly Challenge Index, which purportedly lists the “best” high schools in America based solely on how many AP tests they give.

Jay Mathews writes that one of the reasons his high school ““rankings have drawn such attention is that “only half of students who go to college get to take college-level courses in high school.” What he does NOT say is that another main reason his rankings draw scrutiny is that they are phony; they are without merit. Sadly, far too many parents, educators and School Board members have bought into the “challenge” index that Mathews sells.

The Challenge Index is – and always has been – a phony list that doesn’t do much except to laud AP courses and tests. The Index is based on Jay Mathews’ equally dubious assumption that AP is inherently “better” than other high school classes in which students are encouraged and taught to think critically.

As more students take AP –– many more are doing so…they’ve been told that it is “rigor” and it’s college-level –– more are failing the tests. In 2010, for example, 43 percent of AP test scores were a 1 or 2. The Kool-Aid drinkers argue that “even students who score poorly in A.P. were better off.” Mathews says this too. But it’s flat-out wrong.

The basis for their claim is a College Board-funded study in Texas. But a more robust study (Dougherty & Mellor, 2010) of AP course and test-takers found that “students – particularly low-income students and students of color – who failed an AP exam were no more likely to graduate from college than were students who did not take an AP exam.” Other studies that have tried to tease out the effects of AP while controlling for demographic variables find that “the impact of the AP program on various measures of college success was found to be negligible.”

More colleges and universities are either refusing to accept AP test scores for credit, or they are limiting credit awarded only for a score of 5 on an AP test. The reason is that they find most students awarded credit for AP courses are just generally not well-prepared.

Former Stanford School of Education Dean Deborah Stipek wrote in 2002 that AP courses were nothing more than “test preparation courses,” and they too often “contradict everything we know about engaging instruction.” The National Research Council, in a study of math and science AP courses and tests agreed, writing that “existing programs for advanced study [AP] are frequently inconsistent with the results of the research on cognition and learning.” And a four-year study at the University of California found that while AP is increasingly an “admissions criterion,” there is no evidence that the number of AP courses taken in high school has any relationship to performance in college.

In The ToolBox Revisited (2006) Clifford Adelman scolded those who had misrepresented his original ToolBox research by citing the importance of AP “in explaining bachelor’s degree completion. Adelman said, “To put it gently, this is a misreading.” Moreover, in statistically analyzing the factors contributing to the earning of a bachelor’s degree, Adelman found that Advanced Placement did not reach the “threshold level of significance.”

College Board executives often say that if high schools implement AP courses and encourage more students to take them, then (1) more students will be motivated to go to college and (2) high school graduation rates will increase. Researchers Kristin Klopfenstein and Kathleen Thomas “conclude that there is no evidence to back up these claims.”

In fact, the unintended consequences of pushing more AP may lead to just the reverse. As 2010 book on AP points out “research…suggests that many of the efforts to push the program into more schools — a push that has been financed with many millions in state and federal funds — may be paying for poorly-prepared students to fail courses they shouldn’t be taking in the first place…not only is money being misspent, but the push may be skewing the decisions of low-income high schools that make adjustments to bring the program in — while being unable to afford improvements in other programs.”

Do some students “benefit” from taking AP courses and tests? Sure. But, students who benefit the most are “students who are well-prepared to do college work and come from the socioeconomic groups that do the best in college are going to do well in college.”

So, why do students take AP? Because they’ve been told to. Because they’re “trying to look good” to colleges in the “increasingly high-stakes college admission process,” and because, increasingly, “high schools give extra weight to AP courses when calculating grade-point averages, so it can boost a student’s class rank.” It’s become a rather depraved stupid circle.

One student who got caught up in the AP hype cycle –– taking 3 AP courses as a junior and 5 as a senior –– and only got credit for one AP course in college, reflected on his AP experience. He said nothing about “rigor” or “trying to be educated” or the quality of instruction, but remarked “if i didn’t take AP classes, it’s likely I wouldn’t have gotten accepted into the college I’m attending next year…If your high school offers them, you pretty much need to take them if you want to get into a competitive school. Or else, the admissions board will be concerned that you didn’t take on a “rigorous course load.” AP is a scam to get money, but there’s no way around it. In my opinion, high schools should get rid of them…”

Jay Mathews calls AP tests “incorruptible.” But what do students actually learn from taking these “rigorous” AP tests?

For many, not much. One student remarked, after taking the World History AP test, “dear jesus… I had hoped to never see “DBQ” ever again, after AP world history… so much hate… so much hate.” And another added, “I was pretty fond of the DBQ’s, actually, because you didn’t really have to know anything about the subject, you could just make it all up after reading the documents.” Another AP student related how the “high achievers” in his school approached AP tests:

“The majority of high-achieving kids in my buddies’ and my AP classes couldn’t have given less of a crap. They showed up for most of the classes, sure, and they did their best to keep up with the grades because they didn’t want their GPAs to drop, but when it came time to take the tests, they drew pictures on the AP Calc, answered just ‘C’ on the AP World History, and would finish sections of the AP Chem in, like, 5 minutes. I had one buddy who took an hour-and-a-half bathroom break during World History. The cops were almost called. They thought he was missing.”

An AP reader (grader), one of those “experts” cited by Mathews notes this: “I read AP exams in the past. Most memorable was an exam book with $5 taped to the page inside and the essay just said ‘please, have mercy.’ But I also got an angry breakup letter, a drawing of some astronauts, all kinds of random stuff. I can’t really remember it all… I read so many essays in such compressed time periods that it all blurs together when I try to remember.”

Dartmouth no longer gives credit for AP test scores. It found that 90 percent of those who scored a 5 on the AP psychology test failed a Dartmouth Intro to Psych exam. A 2006 MIT faculty report noted “there is ‘a growing body of research’ that students who earn top AP scores and place out of institute introductory courses end up having ‘difficulty’ when taking the next course.” Mathews called this an isolated study. But two years prior, Harvard “conducted a study that found students who are allowed to skip introductory courses because they have passed a supposedly equivalent AP course do worse in subsequent courses than students who took the introductory courses at Harvard” (Seebach, 2004).

When Dartmouth announced its new AP policy, Mathews ranted and whined that “The Dartmouth College faculty, without considering any research, has voted to deny college credit for AP.” Yet it is Jay who continually ignores and diminishes research that shows that Advanced Placement is not what it is hyped up to be.

In his rant, Mathews again linked to a 2009 column of his extolling the virtues of the book “Do What Works” by Tom Luce and Lee Thompson. In “Do What Works,” Luce and Thompson accepted at face value the inaccuracies spewed in “A Nation At Risk” (the Sandia Report undermined virtually everything in it). They wrote that “accountability” systems should be based on rewards and punishments, and that such systems provide a “promising framework, and federal legislation [NCLB] promotes this approach.” Luce and Thompson called NCLB’s 100 percent proficiency requirement “bold and valuable” and “laudable” and “significant” and “clearly in sight.” Most knowledgeable people called it stupid and impossible.

Luce and Thompson wrote that “data clearly points to an effective means” to increase AP participation: “provide monetary rewards for students, teachers, and principals.”
This flies in the face of almost all contemporary research on motivation and learning.

As I’ve noted before, College Board funded research is more than simply suspect . The College Board continues to perpetrate the fraud that the SAT actually measures something important other than family income. It doesn’t. Shoe size would work just as well.

[For an enlightening read on the SAT, see: http://www.theatlantic.com/magazine/archive/2005/11/the-best-class-money-can-buy/4307/%5D

The College Board produced a “study” purporting to show that PSAT score predicted AP test scores. A seemingly innocuous statement, however, undermined its validity. The authors noted that “the students included in this study are of somewhat higher ability than…test-takers” in the population to which they generalized. That “somewhat higher ability” actually meant students in the sample were a full standard deviation above those 9th and 10th graders who took the PSAT. Even then, the basic conclusion of the “study” was that students who scored well on the PSAT had about a 50-50 chance of getting a “3”, the equivalent of a C- , on an AP test.

A new (2013) study from Stanford notes that “increasingly, universities seem
to be moving away from awarding credit for AP courses.” The study pointed out that “the impact of the AP program on various measures of college success was found to be negligible.” And it adds this: “definitive claims about the AP program and its impact on students and schools are difficult to substantiate.” But you wouldn’t know that by reading Jay Mathews or listening to the College Board, which derives more than half of its income from AP.

What the College Board doesn’t like to admit is that it sells “hundreds of thousands of student profiles to schools; they also offer software and consulting services that can be used to set crude wealth and test-score cutoffs, to target or eliminate students before they apply…That students are rejected on the basis of income is one of the most closely held secrets in admissions.” Clearly, College Board-produced AP courses and tests are not an “incorruptible standard.” Far from it.

The College Board routinely coughs up “research studies” to show that their test products are valid and reliable. The problem is that independent, peer-reviewed research doesn’t back them up. The SAT and PSAT are shams. Colleges often use PSAT scores as a basis for sending solicitation letters to prospective students. However, as a former admissions officer noted, “The overwhelming majority of students receiving these mailings will not be admitted in the end.” But the College Board rakes in cash from the tests, and colleges keep all that application money.

Some say – and sure does look that way – that the College Board, in essence, has turned the admissions process “into a profit-making opportunity.”

Mathews complains about colleges who no longer award AP credit. He says (wink) “Why drop credit for all AP subjects without any research?” Yet again and again he discounts all the research.

Let’s do a quick research review.

A 2002 National Research Council study of AP courses and tests was an intense two-year, 563-page detailed content analysis. The main study committee was comprised of 20 members who are not only experts in their fields but also top-notch researchers. Most also write on effective teaching and learning. Even more experts were involved on content panels for each discipline (biology, chemistry, physics, math), plus NRC staff. Mathews didn’t like the fact that the researchers concluded that AP courses and tests were a “mile wide and an inch deep” and they did not comport with well-established, research-based principles of learning. He dismissed that study as the cranky “opinion of a few college professors.”

The main finding of a 2004 Geiser and Santelices study was that “the best predictor of both first- and second-year college grades” is unweighted high school grade point average, and a high school grade point average “weighted with a full bonus point for AP…is invariably the worst predictor of college performance.” And yet – as commenters noted here – high schools add on the bonus. The state of Virginia requires it.

Klopfenstein and Thomas (2005) found that AP students “…generally no more likely than non-AP students to return to school for a second year or to have higher first semester grades.” Moreover, they write that “close inspection of the [College Board] studies cited reveals that the existing evidence regarding the benefits of AP experience is questionable,” and “AP courses are not a necessary component of a rigorous curriculum.”
In other words, there’s no need for the AP imprimatur to have thoughtful, inquiry-oriented learning.

Phillip Sadler said in 2009 that his research found “students who took and passed an A.P. science exam did about one-third of a letter grade better than their classmates with similar backgrounds who did not take an A.P. course.” Sadler also wrote in the 2010 book “AP: A Critical Examination” that “Students see AP courses on their transcripts as the ticket ensuring entry into the college of their choice,” yet, “there is a shortage of evidence about the efficacy, cost, and value of these programs.” Sadly, AP was written into No Child Left Behind and Race to the Top and it is very much a mainstay of corporate-style education “reform,” touted by the likes of ExxonMobil and the US Chamber of Commerce.

For years, Mathews misrepresented Clifford Adelman’s 1999 ToolBox. As Klopfenstein and Thomas wrote in 2005, “it is inappropriate to extrapolate about he effectiveness of the AP Program based on Adelman’s work alone.” In the 2006 ToolBox Revisited Adelman issued his own rebuke:

“With the exception of Klopfenstein and Thomas (2005), a spate of recent reports and commentaries on the Advanced Placement program claim that the original ToolBox demonstrated the unique power of AP course work in explaining bachelor’s degree completion. To put it gently, this is a misreading.”

The book, ‘AP A Critical Examination’ (2010) lays out the research that makes clear AP has become “the juggernaut of American high school education,” but “ the research evidence on its value is minimal.” It is the academic equivalent of DARE (Drug Abuse Resistance Education). DARE cranks out “research” that shows its “effectiveness,” yet those studies fail to withstand independent scrutiny. DARE operates in more than 80 percent of U.S. school districts, and it has received hundreds of millions of dollars in subsidies. However, the General Accounting Office found in 2003 that “the six long-term evaluations of the DARE elementary school curriculum that we reviewed found no significant differences in illicit drug use between students who received DARE in the fifth or sixth grade (the intervention group) and students who did not (the control group).”

AP may work well for some students, especially those who are already “college-bound to begin with” (Klopfenstein and Thomas, 2010). As Geiser (2007) notes, “systematic differences in student motivation, academic preparation, family background and high-school quality account for much of the observed difference in college outcomes between AP and non-AP students.” College Board-funded studies do not control well for these student characteristics (even the College Board concedes that “interest and motivation” are keys to “success in any course”). Klopfenstein and Thomas (2010) find that when these demographic characteristics are controlled for, the claims made for AP disappear.

I’m left wondering about this wonder school where “several hundred freshmen” take “both AP World and AP Psychology” and where by graduation, students routinely “knock out the first 30 ore more college credits.”

Where is this school, and what is its name? Jay Mathews will surely be interested.”

Categories High School Graduation, Higher Education, Research

New Study Shows Negative Effects of Vouchers in D.C. Program

By dianeravitch
April 27, 2017 //
5

Voucher advocates have protected D.C.’s voucher program, known as “Opportunity Scholarships,” since it was created in 2004 despite lack of strong evidence for its benefits. Evaluations have found little or no improvement in test scores. This new evaluation shows negative effects on test scores in the elementary grades for those who enrolled in voucher schools. This echoes studies in Louisiana, Indiana, and Ohio, where voucher students lost ground as compared to their peers who were offered vouchers but stayed in public schools. In the past, the D.C. evaluation team was led by Patrick Wolf of the University of Arkansas, the high temple of school choice. The evaluation team for this new study was led by Mark Dynarski of Pemberton Research and a group of Westat researchers. Dynarski, you may recall, wrote a paper for the Brookings Institution calling attention to the negative impact of vouchers in Louisiana and Indiana. Previous evaluations showed higher graduation rates in voucher schools, but also–as is now customary in voucher schools–high rates of attrition. Of those who don’t drop out and return to public schools, the graduation rate is higher.

The Washington Post reports:

 

Students in the nation’s only federally funded school voucher initiative performed worse on standardized tests within a year after entering D.C. private schools than peers who did not participate, according to a new federal analysis that comes as President Trump is seeking to pour billions of dollars into expanding the private school scholarships nationwide.

The study, released Thursday by the Education Department’s research division, follows several other recent studies of state-funded vouchers in Louisiana, Indiana and Ohio that suggested negative effects on student achievement. Critics are seizing on this data as they try to counter Trump’s push to direct public dollars to private schools.

Vouchers, deeply controversial among supporters of public education, are direct government subsidies parents can use as scholarships for private schools. These payments can cover all or part of the annual tuition bills, depending on the school.

Education Secretary Betsy DeVos has long argued that vouchers help poor children escape from failing public schools. But Sen. Patty Murray (Wash.), the top Democrat on the Senate Education Committee, said that DeVos should heed the department’s Institute of Education Sciences. Given the new findings, Murray said, “it’s time for her to finally abandon her reckless plans to privatize public schools across the country.”

DeVos defended the D.C. program, saying it is part of an expansive school-choice market in the nation’s capital that includes a robust public charter school sector.

 

“When school choice policies are fully implemented, there should not be differences in achievement among the various types of schools,” she said in a statement. She added that the study found that parents “overwhelmingly support” the voucher program “and that, at the same time, these schools need to improve upon how they serve some of D.C.’s most vulnerable students.”

DeVos’ statement suggests that neither vouchers nor charters will ever outperform public schools. The goal of choice is choice, not better academic achievement or better education, not to “save poor kids from failing schools,” but to provide choice.

 

 

 

 

Categories District of Columbia, Research, Vouchers

National Education Policy Center: A Review of Claims about Vouchers in Milwaukee

By dianeravitch
April 26, 2017 //
3

We can always count on researchers at the National Education Policy Center to review reports issued by think tanks and advocacy groups, some of which are the same.

This review analyzes claims about Milwaukee’s voucher schools. It is funny to describe them as successful, since Milwaukee is really the poster city for the failure of school choice. It has had vouchers and charters since 1990 and is near the very bottom of the NAEP tests for urban districts, barely ahead of sad Detroit, another city afflicted by charters. Both cities demonstrate that school choice does not fix the problems of urban education or urban students and families.

Find Documents:

Press Release: http://nepc.info/node/8612
NEPC Review: http://nepc.colorado.edu/thinktank/review-milwaukee-vouchers
Report Reviewed: http://www.will-law.org/wp-content/uploads/2017/03/apples.pdf

Contact:
William J. Mathis: (802) 383-0058, wmathis@sover.net
Benjamin Shear: (303) 492-8583, benjamin.shear@colorado.edu

Learn More:

NEPC Resources on Accountability and Testing
NEPC Resources on Charter Schools
NEPC Resources on School Choice
NEPC Resources on Vouchers

BOULDER, CO (April 25, 2017) – A recent report from the Wisconsin Institute for Law and Liberty attempts to compare student test score performance for the 2015-16 school year across Wisconsin’s public schools, charter schools, and private schools participating in one of the state’s voucher programs. Though it highlights important patterns in student test score performance, the report’s limited analyses fail to provide answers as to the relative effectiveness of school choice policies.

Apples to Apples: The Definitive Look at School Test Scores in Milwaukee and Wisconsin was reviewed by Benjamin Shear of the University of Colorado Boulder.

Comparing a single year’s test scores across school sectors that serve different student populations is inherently problematic. One fundamental problem of isolating variations in scores that might be attributed to school differences is that the analyses must adequately control for dissimilar student characteristics among those enrolled in the different schools. The report uses linear regression models that use school-level characteristics to attempt to adjust for these differences and make what the authors claim are “apples to apples” comparisons. Based on these analyses, the report concludes that choice and charter schools in Wisconsin are more effective than traditional public schools.

Unfortunately, the limited nature of available data undermines any such causal conclusions. The inadequate and small number of school-level variables included in the regression models are not able to control for important confounding variables, most notably prior student achievement. Further, the use of aggregate percent-proficient metrics masks variation in performance across grade levels and makes the results sensitive to the (arbitrary) location of the proficiency cut scores. The report’s description of methods and results also includes some troubling inconsistencies. For example the report attempts to use a methodology known as “fixed effects” to analyze test score data in districts outside Milwaukee, but such a methodology is not possible with the data described in the report.

Thus, concludes Professor Shear, while the report does present important descriptive statistics about test score performance in Wisconsin, it wrongly claims to provide answers for those interested in determining which schools or school choice policies in Wisconsin are most effective.

Find the review by Benjamin Shear at:

http://nepc.colorado.edu/thinktank/review-milwaukee-vouchers

Find Apples to Apples: The Definitive Look at School Test Scores in Milwaukee and Wisconsin, by Will Flanders, published by the Wisconsin Institute for Law and Liberty, at:

http://www.will-law.org/wp-content/uploads/2017/03/apples.pdf

Categories Milwaukee, National Education Policy Center, Research, Vouchers

Wonderful News! Kevin Welner of NEPC Honored by American Educational Research Association

By dianeravitch
April 25, 2017 //
3

One of the most reliable sources for analysis of education research is the National Education Policy Center.

The American Educational Research Association has just honored the NEPC director Kevin Welner for his role in leading the organization and the fine work that it does. If you recall, in addition to its regular reports on research and its reviews of think tank products, NEPC produces an annual Bunkum Award to the worst education research of the year.

BOULDER, CO (April 24, 2017) – NEPC Director Kevin Welner has been awarded the 2017 American Educational Research Association’s Outstanding Public Communication of Education Research Award. The award honors scholars exemplary in their capacity to communicate important education research to the public, including education communities. It recognizes a scholar who has demonstrated the capacity to deepen the public’s understanding and appreciation of the value of education research in civic decision-making.

Welner is a well-known interpreter of education research for general audiences. He appears regularly in the media, presents at public forums, and has authored numerous op-ed essays on education policy topics. His work has been showcased in the Washington Post “Answer Sheet” blog as well as on NPR’s “Here and Now.” In addition he was a keynote speaker at the White House Reach Higher conference “Beating the Odds: Successful Strategies from Schools & Youth Agencies that Build Ladders of Opportunity.”

After learning of his award, Welner commented, “The accomplishments this award recognizes rest on a foundation laid over the past two decades by many others. Alex Molnar’s work at the Center for Education Research Analysis and Innovation (University of Wisconsin-Milwaukee) and the Education Policy Studies Laboratory (Arizona State University); Jeanne Oakes’s work at the Institute for Democracy, Education, and Access (UCLA); and Ken Howe’s work at the Education and Public Interest Center (CU Boulder) have all helped make what I’m doing possible.” Welner went on to note that he works with talented colleagues on the NEPC staff and across the country. “The NEPC staff and NEPC’s 125 fellows make enormous contributions to our collective effort. Providing high-quality research and analysis in support of democratic deliberation about ​education policy is the mission of NEPC. This award tells me that we are on course.”

NEPC co-founder and Publications Director, Alex Molnar, praised Welner, commenting, “I can think of no one more deserving of this award. It is an honor to work with such a talented and ethical scholar whose enduring commitment to equity, social justice, and democratic decision-making enriches our scholarship, improves education practice, and strengthens our civic life.”

I am happy to add Kevin Welner to the honor roll of this blog, an award that is harder to achieve than the Academy Award or an Emmy.

Categories Honor Roll, Research

Major Scientific Discovery: Kids Need to Move More

By dianeravitch
March 28, 2017 //
35

Happily, the New York Times published an article saying that children need to move more. There is actual scientific research proving that immobility is not healthy. It is not healthy for adults but especially not for children, whose bodies are growing.

Some nations, like Finland, realized this a long time ago, so students have recess and outdoor play after every class. Recess! Time to go outside and run and play! The authorities think this is good for the children.

Back in the olden times, most American schools had recess a few times a day, at least once or twice, anyway.

But over the past two decades, the need to raise standardized test scores has dwarfed the importance of movement and play. Test scores are the purpose of schooling, right?

The article includes several references to Apps and videos so that teachers will know how children should move. That’s in case you don’t know or forgot children move.

The Finns have another approach. No videos. No Apps. They open the school doors and let the children go outside, where there is playground equipment. Without any direction, children move all by themselves.

Categories Accountability, Childhood, Education Industry, Play, Research, Testing

One Month from Today Is EARTH DAY: March for Science!

By dianeravitch
March 22, 2017 //
20

The first Earth Day occurred in 1970, and it raised awareness of the importance of protecting the environment.

Now, more than ever, we must all stand together to support the environment, to defend science, to denounce fake theories, and to protect the earth.

The Trump administration thinks that climate change is a hoax. Trump’s budget makes deep cuts in the budget of the Environmental Protection Agency, which has enjoyed bipartisan support for almost 50 years.

Earth Day is April 22.

Join your friends and neighbors to speak up for clean air and clean water. Join with them to protest the budget cuts to scientific research in many other agencies.

Plan now for April 22.

Categories Environment, Research, Science

NEPC Bunkum Award for Shoddiest Think Tank Report of 2016: And the Winner is…..

By dianeravitch
February 25, 2017 //
7

Every year the National Education Policy Center announces the winner of its not-at-all coveted Bunkum Award for the shoddiest think tank report of the previous year.

This year’s winner is the Center for American Progress, for its report purporting to show that the Common Core standards raise the achievement of poor students.


Winner of NEPC’s 2016 Bunkum Award\

BOULDER, CO (PRWEB) FEBRUARY 23, 2017

The 89th Academy Awards will be celebrated this weekend, which means it’s also time to announce the winner of the 2016 National Education Policy Center Bunkum Award. We invite you to enjoy our 11th annual tongue-in-cheek “salute” to the shoddiest think tank report reviewed in 2016.

This year’s Bunkum winner is the Center for American Progress (CAP), for its report, Lessons From State Performance on NAEP: Why Some High-Poverty Students Score Better Than Others.

To learn who our editors judged to be Bunkum Award-worthy, be sure to watch the 2016 Bunkum Award video presentation, read the Bunkum-worthy report and the review, and learn about past Bunkum winners and the National Education Policy Center’s Think Twice Think Tank Review project:

http://nepc.colorado.edu/think-tank/bunkum-awards/2016

About the Think Twice Think Tank Review Project:

Many organizations publish reports they call research – but are they? These reports often are published without having first been reviewed by independent experts – the “peer review” process commonly used for academic research.

Even worse, many think tank reports subordinate research to the goal of making arguments for policies that reflect the ideology of the sponsoring organization.

Yet, while they may provide little or no value as research, advocacy reports can be very effective for a different purpose: they can influence policy because they are often aggressively promoted to the media and policymakers.

To help the public determine which elements of think tank reports are based on sound social science, NEPC’s “Think Twice” Think Tank Review Project has, every year since 2006, asked independent experts to assess strengths and weaknesses of reports published by think tanks.

Few of the think tank reports have been found by experts to be sound and useful; most, however, are found to have little, if any, scientific merit. At the end of each year NEPC editors sift through the reviewed reports to identify the worst offender. We then award the organization publishing that report NEPC’s Bunkum Award for shoddy research.

Find Documents:

Press Release: http://nepc.info/node/8504

NEPC Review: http://nepc.colorado.edu/thinktank/review-CAP-standards

The National Education Policy Center (NEPC) Think Twice Think Tank Review Project (http://thinktankreview.org) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. The project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu

Categories Accountability, Common Core, Corporate Reform, Hoax, Research, Testing

Gene V. Glass: The Department of Education’s “Gold Standard” Produces “Fool’s Gold”

By dianeravitch
February 21, 2017 //
13

Gene V. Glass is one of our nation’s most distinguished education researchers.

This post is an important analysis of the failure of the U.S. Department of Education’s Institute of Education Sciences, which was reorganized during the George W. Bush administration.

As a new administration moves into the US Department of Education, the opportunity arises to review and assess the Department’s past practices. A recent publication goes to the heart of how US DOE has been attempting to influence public education. Unfortunately, in an effort to justify millions of dollars spent on research and development, bureaucrats pushed a favorite instructional program that teachers flatly rejected.

The Gold Standard

There is a widespread belief that the best way to improve education is to get practitioners to adopt practices that “scientific” methods have proven to be effective. These increasingly sophisticated methods are required by top research journals and for federal government improvement initiatives such as Investing in Innovation (i3) Initiative to fund further research or dissemination efforts. The US DOE established the What Works Clearinghouse (WWC) to identify the scientific gold-standards and apply them to certify for practitioners which programs “work.” The Fed’s “gold standard” is the Randomized Comparative Trial (RCT). In addition, there have been periodic implementations of US DOE policies that require practitioners to use government funds only for practices that the US DOE has certified to be effective.

However, an important new article published by Education Policy Analysis Archives, concludes that these gold-standard methods misrepresent the actual effectiveness of interventions and thereby mislead practitioners by advocating or requiring their use. The article is entitled “The Failure of the U.S. Education Research Establishment to Identify Effective Practices: Beware Effective Practices Policies.”

The Fool’s Gold

Earlier published work by the author, Professor Stanley Pogrow of San Francisco State University, found that the most research validated program, Success for All, was not actually effective. Quite the contrary! Pogrow goes further and analyzes why these gold-standard methods can not be relied on to guide educators to more effective practice.

Researchers have told us that we need “randomized comparative trials” to reach “research-based conclusions.”

In fact, says Glass and the article he cites, this is not what happens. And the results of these trials turn out to be easily manipulated and falsified.

He writes:

Key problems with the Randomized Comparative Trial include (1) the RCT almost never tells you how the experimental students actually performed, (2) that the difference between groups that researchers use to consider a program to be effective is typically so small that it is “difficult to detect” in the real world, and (3) statistically manipulating the data to the point that the numbers that are being compared are mathematical abstractions that have no real world meaning—and then trying to make them intelligible with hypothetical extrapolations such the difference favoring the experimental students is the equivalent of increasing results from the 50th to the 58th percentile, or an additional month of learning. The problem is that we do not know if the experimental students actually scored at the 58th or 28th percentile. So in the end, we end up not knowing how students in the intervention actually performed, and any benefits that are found are highly exaggerated.

The sad part of the story is that we now have a new administration that is both ignorant of research and indifferent to it. DeVos has seen the failure of school choice in her own state, which has plummeted in the national rankings since 2003, and it has had no impact on her ideology. Ideology is not subject to testing or research. It is a deep-seated belief system that cannot be dislodged by evidence.

Categories Betsy DeVos, Data, Research, U.S. Department of Education

Betsy DeVos Had an Awfully Difficult First Week on the Job

By dianeravitch
February 20, 2017 //
47

Teachers who teach children with multiple disabilities and children who are homeless may think that they have a tough job, but consider what a very hard time Betsy DeVos had in her first week as Secretary of Education, very likely the first paying job she has ever held. She visited a public middle school in D.C., where protestors harassed her and tried to keep her out. When she eventually entered the school, she said nice things to the staff, but after she left she insulted them as being in a “receive” mode. She gave a few interviews and said she hoped to launch more charter schools, more vouchers, more cybercharters, and presumably shrink the number of public schools as she opens up opportunities for students to go anywhere other than public schools.

In one interview, she told syndicated conservative columnist Cal Thomas that she did not think the protests against her were spontaneous. The implication was that those evil teachers’ unions had plotted against her. The other implication was that parents and teachers would welcome her noble presence in their public school, even though she was unimpressed with what she saw. Someone, she said, was trying to make her life “a living hell.” No matter what the plotters do, she pledged she would not be deterred from her mission of “helping kids in this country,” by enabling them to leave public schools for privatized alternatives.

She suggested to Thomas that it might be a good idea to bring tens of thousands of children to the Capitol to demonstrate for charters and vouchers. She said it had worked in Florida. Of course, this is now a standard part of the privatization script, using children as political pawns to demand more public funding for private choices, thus disabling public schools by diminishing their resources.

She pledged to support alternatives to public schools, without citing a scintilla of evidence that these choices would help kids and without acknowledging that the proliferation of choices harms the great majority of children who don’t choose to leave public schools. Her mindset is purely ideological. She did not offer any suggestions about how to help the vast majority of children who attend public schools. She has one idea, and she is sticking to it: choice. The absence of evidence for that one idea from her home state never comes up. Michigan has tumbled in national rankings as choice has expanded.

When Cal Thomas asked what could be done for children who had an “absent father,” she responded that this problem has to be addressed at the classroom level. “It’s not an easy or a single answer, but again it goes back to having the power to influence those things at the classroom level.” It is not clear what she meant or if she herself knew what she meant. How is the teacher in the classroom prepared to make up for an absent father? Is this what passes for profundity?

Then there was this very interesting exchange, in which Cal Thomas and Betsy DeVos exposed their deepest beliefs:

“Q. Throughout most of the public school system, which began in the late 19th century and flourished in the 20th, education included values, McGuffey Readers and even prayer and Bible reading, until the Supreme Court outlawed both in the ‘60s. Do you see a correlation between the loss of American values, a sense of morality, a concept of the transcendent, right and wrong, objective truth that have been banished in our relativistic age and lack of achievement in some places in our schools?

“A. I think it’s a significant factor. Many of the schools I’ve seen, especially the charters, have a focus on character development and again the whole child development. That’s one of the reasons parents are choosing alternatives like this.”

To begin with, public education got its start in the mid-nineteenth century, not the late nineteenth century.

I am one of the few living Americans who has actually read the entirety of the McGuffey readers. Children today would find them dull, simplistic, and obsolete.

The assumption that public schools lack values because they do not have Bible readings and prayers is nonsense. When I went to public schools in Houston in the 1940s and 1950s, we had daily Bible readings and prayers, but the schools were racially segregated. Few teachers had more than a bachelor’s degree. I would say without question that our public schools today have better quailed teachers today and a stronger value system than they did when we read the Bible and prayed every day. As a Jew in a Christian public school system, I ignored the implicit proselytizing, but from the perspective of the decades, I can say that our schools then did not practice what they preached. We never discussed current social or political issues. Too controversial. We were not well prepared for the real problems of our society.

The values of the dominant religion were imposed on me but I never had any wish to impose mine on anyone else.

Now, as we live in a religiously and culturally diverse society, Thomas and DeVos sound like two antiquarians. They want to turn the clock back 100 years, maybe two hundred years.

There is nothing innovative about DeVos’ ideas. She has lived in a billionaire bubble all her life, surrounded by her like-minded kith and kin of rich white Republican evangelicals. She has nothing to teach our teachers or students. She knows nothing about how to improve public schools. Her beloved charters, vouchers, and cybercharters have not proven to be better than public schools, and in many states, are demonstrably worse than traditional public schools with certified teachers.

We live in a big, ever-changing world, and it is far too late to go back to 1920 or 1820, no matter how devoutly DeVos would like to restore the suprenpmacy of whites and evangelical Christians. They too must learn to live and let live.

Categories Accountability, Betsy DeVos, Charter Schools, Education Industry, Equity, Privatization, Research, Segregation, Separation of church and state, Virtual Charter Schools, Vouchers

Laura Chapman on the Mysterious Non-Evaluation of ED Department Charter Funding

By dianeravitch
January 22, 2017 //
12

Laura Chapman, retired educator and crack researcher, comments on the strange case of the millions (billions?) of dollars awarded to charter schools that have never been evaluated as to their use, misuse, or effectiveness:

 

 

I doubt if you will ever find an evaluation report for our charter school investments, from USDE or IES.
USDE shoved money out the door for anything charter–startup, replication (franchising), and facilities and facilities financing.

 
Negative reviews of the grant applications were ignored. I read some of these reviews. Even the questions for the reviewers were rigged to minimize “accountability.”

 
We paid federal dollars for absurdities, including advertising for charter students; cross-country junkets to recruit teachers and leaders; uniforms for the students including backpacks with logos, various goodies for “awards.”

 
I have yet to find any reports from states back to USDE on what happened to the money channeled to states. No federal reports on schools opened, closed, etc. No credible peer-reviewed independent studies on student outcomes.

 
USDE let the charter authorizers and franchisers call the shots. Some of the grant applications had redacted information–unreadable chunks of text blacked out because this information was “proprietary” and might “leak,” offering a competitive advantage to other charters. USDE rolled over on that request from the grant applicants. What was redacted? Test scores and enrollments were redacted.

 
I just checked the 2016 active contracts of USDE bearing on charters. Only two are there, and both have been granted extensions from the original contracts.

 
WESTAT, INC. The purpose of this procurement is to obtain technical services for the U.S. Department of Education Charter Schools Program to support the Credit Enhancement Program with grantee monitoring. Monitoring grant projects means examining policies, systems, and procedures to ensure compliance with Federal statutes, regulations, Guidance, grant applications, and performance agreements.

Dates: 9/25/15 to 9/24/17 extended to 9/24/19
Amount: $514,554. This is a very small contract for WESTAT: It is a major subcontractor for many federal agencies.

 
SAFAL PARTNERS, INC. The purpose of this contract is to obtain technical assistance for the U.S. Department of Education Charter Schools Program for a range of activities, including online assistance, meetings, reports, studies, and assistance in a variety of focus areas, that COULD include human capital resources, facilities, authorizing, accountability, students with disabilities, English learners, military-connected children, and others.
Dates: from 9/27/13 to 9/26/17 extended to 9/26/18-
Amount: $12,872,533.
SAFAL Partners leads the National Charter School Resource Center, which hosts other USDE subcontractors. SAFAL Partners appears to be the go-to outfit for charter-friendly research. Clients include the Bill and Melinda Gates Foundation, Laura and John Arnold Foundation, Education Pioneers, George W. Bush Institute, Teach for America Houston among others.

 
A 2016 report from SAFAL Partners titled “Student Achievement in Charter Schools: What the Research Shows” is a very limited and dubious “comparison” of five studies, with caveats dismissed (e.g., fewer ELL and special education students in charter schools, only math and reading scores, test scores not from the same tests, and more). This report is more PR more than credible research.

Categories Charter Schools, Funding, Research, U.S. Department of Education
« Older Entries
Newer Entries »