A reader commented on an ongoing discussion of Advanced Placement courses and the rating of high schools according to how many students, ready or not, take them.
She writes:
“The highly publicized rankings based on AP tests taken did a lot of harm in my sons’ high school. AP courses used to attract the students who were genuinely ready for college-level work, but about seven years ago, the school system instituted a policy that required every student who took an AP course to take the AP exam, and also began to push students into AP who did not really need to be there. There were parent meetings about AP and IB and the message was sent that if our kids wanted to be high achievers, they would take these classes. Soon, through school system pressure and peer pressure, a two-tiered system developed where the “smart” kids were in AP, and “everyone else” was in regular classes. The school began to make the “Challenging High Schools” list, but the teachers (who are evaluated by students scores on the tests and not just by how many take them) were now under great pressure to keep their scores up. They could not afford to slow down, or to differentiate for students with learning differences, or do much at all for those who were just not ready for the level of difficulty of the AP courses. It was sink or swim, and a lot of kids sank.
“My older son was a high achiever and a good tester, so he did fine, although we got a rude shock when he got to college and realized that acceptance of AP scores varies HIGHLY from university to university and even from school to school within a university. Almost none of his high scores actually allowed him to skip college courses.
“My younger son has some learning differences, and AP courses were frustrating and overwhelming for him, while at the same time, his non-AP courses were, as he put it, “filled with slackers”. Fortunately we had a very good counselor who was able to transfer him to classes that were a better fit. He has now graduated, but I hear from friends that the school’s policy for next year is NO TRANSFERS OUT of AP, even if the student is failing. So many kids were bailing on AP that it was messing up class sizes.
“I firmly believe that students do better if they are challenged, but AP is not for every student. It is not even for most students.”
Diane Ravitch's blog 
In my state there is an arrangement with a major state university for high school students to be concurrently enrolled in the equivalent college level course, for which they get transferable college credits. Instructors must have an MA in the subjects they teach, their materials are examined, and they are observed by faculty members from the university.
I strongly favor this because it puts the emphasis back on the experience of taking the course (as it should be) rather than on a canned test at the end. The AP syllabus for my subject is extremely narrow and prescriptive; it is nakedly a test prep class, and I am so happy that there is an intellectually nourishing alternative that offers the same benefits.
With the concurrent college enrollment program, I have been able to maintain high academic standards while also retaining an appropriate amount of instructional autonomy (just like the professors whose classes mine are supposed to resemble). The students get official college transcripts reflecting their credit hours. Graduates tell me that, in many cases, they have been able to waive classes. I am sure there are similar programs in other states.
LikeLike
The problem, and I’m assuming that this is universally true, is that once you apply for concurrent enrollment, you have to deal with drop dates and passing the class. If you don’t, because it was for college credit through some school, the failing mark will stay on your transcript. If a child isn’t ready for the rigor, and you aren’t willing to pass everyone, it can hurt them.
A girl from a colleague’s former high school lost a scholarship for this exact reason.
LikeLike
Small schools may not be able to offer enough classes for talented students. My son prepared for AP exams in a variety of ways.
LikeLike
The reformer push for all students to take AP courses is destroying the value of AP. The very name– advanced placement– bespeaks a setting apart.
The credibility of ACT as an indicator of college readiness and success is also in the toilet. The test norms are being lowered since all students must take the test. I have read where some higher ed institutions do not even require ACT and SAT any longer:
“The number of test-optional institutions grew after the most recent revision of the SAT and ACT, in 2005. Of those schools, some exempt applicants who meet GAP or class-rank criteria while others require scores only for course placement purposes or for internal research.
“‘[Colleges and universities] recognize that neither the SAT nor ACT measures what students most need to succeed in higher education,’ FairTest Public Education Director Bob Schaeffer said in a statement Wednesday. ‘Even the tests’ sponsors admit that an applicant’s high school record remains a better predictor of college performance than either exam is.'”
http://www.huffingtonpost.com/2012/11/28/sat-act-not-required-colleges_n_2206391.html
LikeLike
I haven’t read that about the lowering of the norms of the ACT and SAT. I’m not surprised however, because of the numbers of students in my high school who are scoring in the 30s in increasing significantly, and without the accompanying school performace.
LikeLike
If the quality of the test is held constant (which I doubt), then since the overall caliber of student taking the test is lower (general populous as opposed to kids who intend to attend college), this will affect the norms. Having noted that, I know that the average score has been on the rise over decades– this is affected by all of the test prep available– it allows students to game the system. PLUS, any changes in how ACT scores the test (the actual calibration of the numbers) can make the numbers “look” higher than they should be (can change the meaning of, say, a 27 on the ACT in 1990 vs. a 27 in 2013).
The calibrating is similar to what happens in the clothing industry– to make a woman “feel” smaller, let’s change the number inside of a size 8 dress to size 6. Same dress, different number.
I see the same issue happening as you have described: Kids scoring in the 30s whose class work does not reflect what a 30 on the ACT once meant. So, ironically, reformers are worshiping tests whose watered-down scores now mean little to the colleges that once used them.
LikeLike
One reason that ACT/SAT scores and class grades might be in conflict is that class grades have an important comportment and biddability that is not present in the exams.
LikeLike
I should not post this late. What I meant to say is that class grades have an important comportment and biddability component that is not present in the exam scores.
LikeLike
Mercedes @ 12:11,
“I see the same issue happening as you have described: Kids scoring in the 30s whose class work does not reflect what a 30 on the ACT once meant.”
A 30 before, a 30 now, a 30 later doesn’t “mean” anything. That is to say that the ACT, like any standardized test suffers the same errors of construction, same errors of giving and taking the test, and the same errors of disseminating the results as proven by Noel Wilson in “Educational Standards and the Problem of Error” found @ http://epaa.asu.edu/ojs/article/view/577/700 to render the process totally invalid. There was no halcyon days of the ACT (or any standardized test for all that matters). Now they might be adjusting scores etc. . . but that is all what I refer to as mental masturbation as the whole process is invalid. Since invalid information gets one invalid results, why waste the time???
It amounts to what R. Ackhoff calls “doing the wrong thing righter”:
The proliferation of educational assessments, evaluations and canned programs belongs in the category of what systems theorist Russ Ackoff describes as “doing the wrong thing righter. The righter we do the wrong thing,” he explains, “the wronger we become. When we make a mistake doing the wrong thing and correct it, we become wronger. When we make a mistake doing the right thing and correct it, we become righter. Therefore, it is better to do the right thing wrong than the wrong thing right.”
Our current neglect of instructional issues are the result of assessment policies that waste resources to do the wrong things, e.g., canned curriculum and standardized testing, right. Instructional central planning and student control doesn’t – can’t – work. But, that never stops people trying.
The result is that each effort to control the uncontrollable does further damage, provoking more efforts to get things in order. So the function of management/administration becomes control rather than creation of resources. When Peter Drucker lamented that so much of management consists in making it difficult for people to work, he meant it literally. Inherent in obsessive command and control is the assumption that human beings can’t be trusted on their own to do what’s needed. Hierarchy and tight supervision are required to tell them what to do. So, fear-driven, hierarchical organizations turn people into untrustworthy opportunists. Doing the right thing instructionally requires less centralized assessment, less emphasis on evaluation and less fussy interference, not more. The way to improve controls is to eliminate most and reduce all.
Former Green Beret Master Sergeant Donald Duncan did when he noted in Sir! No Sir! that: “I was doing it right but I wasn’t doing right.”
And from one of America’s premier writers:
“The mass of men [and women] serves the state [education powers that be] thus, not as men mainly, but as machines, with their bodies. They are the standing army, and the militia, jailors, constables, posse comitatus, [administrators and teachers], etc. In most cases there is no free exercise whatever of the judgment or of the moral sense; but they put themselves on a level with wood and earth and stones; and wooden men can perhaps be manufactured that will serve the purpose as well. Such command no more respect than men of straw or a lump of dirt.”- Henry David Thoreau [1817-1862], American author and philosopher
LikeLike
Rating high schools on the number of kids taking AP courses, taking AP tests and passing AP tests is not an indicator of anything. It’s a system that is easily gamed.
I say this as a teacher of AP European History for the last 16 years.
I don’t pressure kids to take my class but I highly encourage kids, especially somewhat average students, who intend to go to college of any type. I do so because my class is writing and research heavy and I believe it can help them experience the types of assignments they will get in college. I tell them not to worry about their grades or even taking the test because it is not what I’m trying to help them get from the class.
I’ve generally gotten good feedback on this approach from students and parents. They like that they have at least one AP on their transcript. They put in the time to improve. They get some challenge and rigor. And they often contact me about a year or two after graduation to let me know how it prepared them for college (both positive and needs improvement).
Labeling the class AP provides me with the justification to make it challenging as well. I like that I can use it to attract typically non-AP kids who have the work ethic and curiosity to go for something harder for their own development. My class is an elective so ultimately it is their choice.
But to base school quality on AP metrics is not desirable or indicative of anything regarding the school itself.
LikeLike
My experience with teaching an AP course is that the ones who drop out of it are not the ones who are failing or making a D, but those who are making a B when they are used to making A’s. Some students don’t want the challenge, they want the GPA, and when it gets too tough to get that A easily, they bail. My school is instituting a “no drop policy” for AP classes this year to try to prevent this gaming of the system.
We also allow students to choose between taking the course for AP credit (with the test), dual credit with a local college (which they pay for), or for honors credit (no AP test). It’s a confusing system, but the dual credit system has been in place since the 1990s, and AP is new (since MO added AP course offering to its measurement of schools this past year). Allowing the choice means more students will take the courses, but much of that pressure is off, except for the grade issues I mentioned above. I hear from those who stick with it that they were better prepared for college courses wjen they got to college, whether or not they made an A or scored high on the test.
No one should be forced or pressured into taking an AP course, not by school administrators, teachers, peers, or parents. But all who’d like to, and are academically and mentally prepared for the work involved, should be given the chance to try. I believe it’s to their benefit to stick it out, no matter the grade. I think the added weight that’s imposed on AP courses, through media, education departments, or administrators, is the problem, not the courses themselves or the students who are taking them. Then the whole process just becomes all about another high-stakes exam.
LikeLike
My child signed up for AP Euro offered 10th grade at her school. She thrives on making straight A’s. When we contacted the school because she was experiencing severe anxiety and wanted to change her schedule during the add/drop period prior to school beginning – the staff informed us that they would not approve the schedule change.
Her concerns stem from knowing upper classmen that have not scored well on the AP exam. These students also made her believe that their enrollment in the combined classes really ate into the time they needed to dedicate to other classes.
She is still fearful and feels extremely pressured about the upcoming year. She is a diligent worker and a planner. Type A personality. I believe she is capable of staying on top of the assignments if she doesn’t bomb out from anxiety first. Any advice to help calm her prior to school beginning will be greatly appreciated.
LikeLike
I haven’t heard that AP classes in Chicago also can provide for dual credit with a local college. That could be helpful for anyone wanting to better afford college.
Anyone know if this varies state-by-state?
LikeLike
I believe this does vary by state, and even by school district in our state. My district went from offering solely dual credit to offering it and AP, but I think we’re unusual in that. We entered the AP game only recently and are still feeling our way around this. We kept the dual credit program as an option because it is popular and it does give many students a leg up on college, and the hours are very affordable for most ($60/credit hour). I would check with your state education department to see if the option is available.
LikeLike
Not every university will take dual enrollment credits. Some refuse to take transfer credit for incoming first year students.
LikeLike
Advanced Placement courses and tests are part of the “competitive” mania that characterize the current phase of public school “reform.” In the case of AP, corporate “reformers” deem it necessary to produce STEM workers (never mind that there is nothing close to a shortage) and to allow the U.S. to remain globally “competitive.” Students consider it necessary to maintain a “competitive” edge in college admissions, since AP courses are typically weighted to increase grade point average. And colleges consider SAT/ACT scores and AP courses as being critical to their standings in the “competitive” rankings of academic institutions.
To put it mildly, these “competitive” races are based on myths.
It’s all provable nonsense. For example, each year the World Economic Forum (WEF) uses a sophisticated set of metrics to rank nations on their international competitiveness and what causes it.
When the U.S. dropped from 2nd to 4th in 2010-11, four factors were cited by the WEF for the decline: (1) weak corporate auditing and reporting standards, (2) suspect corporate ethics, (3) big deficits (brought on by Wall Street’s financial implosion) and (4) unsustainable levels of debt.
Last year (2011-12), major factors cited by the WEF are a “business community” and business leaders who are “critical toward public and private institutions,” a lack of trust in politicians and the political process with a lack of transparency in policy-making, and “a lack of macroeconomic stability” caused by decades of fiscal deficits, especially deficits and debt accrued over the last decade that “are likely to weigh heavily on the country’s future growth.” [Note: The WEF did NOT cite public schools as being problematic to innovation and competitiveness. ]
And this year (2012-13) the WEF dropped the U.S. to 7th place, citing problems like “increasing inequality and youth unemployment” and “the United States is among the countries that have ratified the fewest environmental treaties.“ The WEF noted that in the U.S.,”the business community continues to be critical toward public and private institutions” and “trust in politicians is not strong.” Political dysfunction has led to “a lack of macroeconomic stability” that “continues to be the country’s greatest area of weakness.”
The “competitiveness” rankings of colleges put out by US News & World Report are mythic as well. If, for example, Harvard and Yale and Dartmouth (to pick three) are so doggone good, then why did so many Harvard and Yale and Dartmouth grads fail so miserably –– in terms of economic knowledge and also in terms of ethics and morality – (take, for example, George W. Bush, Jamie Dimon, Larry Summers and Hank Paulson, to name only a few) in the prelude to the Great Recession?
[Or watch this….What causes the change in seasons? See Harvard grads try to explain it….http://www.youtube.com/watch?v=p0wk4qG2mIg ]
As to the weighting of AP classes, the main finding of a 2004 Geiser and Santelices study (looking at more than 80,000 students who entered the University of California system over three years) was that “the best predictor of both first- and second-year college grades” is unweighted high school grade point average, and a high school grade point average “weighted with a full bonus point for AP…is invariably the worst predictor of college performance.” The best predictor is UN-weighted GPA.
But high schools award bonus points, and that’s why many students take AP, to pad their transcripts. Even more perversely, the College Board, which produces the SAT, now recommends that schools “implement grade-weighting policies…starting as early as the sixth grade.”
Yes, that’s correct. The College Board recommends that the competitive nonsense be pushed into the SIXTH grade! If that sounds stupid, or conniving, perhaps even fraudulent, that’s because it is.
Here’s what research tells us about AP programs:
An intense two-year, 563-page detailed content analysis of AP courses and tests was conducted by the National Research Council in 2002. The main study committee was comprised of 20 members who were not only experts in their fields and top-notch researchers. More experts were involved on content panels for each discipline (biology, chemistry, physics, math), plus NRC staff. The researchers concluded that AP courses and tests were a “mile wide and an inch deep” and they did not comport with well-established, research-based principles of learning.
The main finding of a 2004 Geiser and Santelices study was cited above. In The ToolBox Revisited (2006) Clifford Adelman found that “Advanced Placement has almost no bearing on entering postsecondary education,” and when examining and statistically quantifying the factors that relate to bachelor’s degree completion, Advanced Placement does NOT “reach the threshold level of significance.”
A 2006 MIT faculty report noted ““there is ‘a growing body of research’ that students who earn top AP scores and place out of institute introductory courses end up having ‘difficulty’ when taking the next course.” Harvard “conducted a study that found students who are allowed to skip introductory courses because they have passed a supposedly equivalent AP course do worse in subsequent courses than students who took the introductory courses at Harvard” (Seebach, 2004).
Dartmouth found that high scores on AP psychology tests do NOT translate into college readiness for the next-level course. Indeed, students admit that ““You’re not trying to get educated; you’re trying to look good;” and, “”The focus is on the test and not necessarily on the fundamental knowledge of the material.” AP is far more about gaming the college acceptance process than it is learning.
Phillip Sadler said in 2009 that his research found “students who took and passed an A.P. science exam did about one-third of a letter grade better than their classmates with similar backgrounds who did not take an A.P. course.” Sadler added this in the 2010 book AP: A Critical Examination: “Students see AP courses on their transcripts as the ticket ensuring entry into the college of their choice…there is a shortage of evidence about the efficacy, cost, and value of these programs.”
AP has become “the juggernaut of American high school education,” but “ the research evidence on its value is minimal.” AP may work well for some students, especially those who are already “college-bound to begin with” (Klopfenstein and Thomas, 2010)
As Geiser (2007) notes, “systematic differences in student motivation, academic preparation, family background and high-school quality account for much of the observed difference in college outcomes between AP and non-AP students.” College Board-funded studies do not control well for these student characteristics (even the College Board concedes that “interest and motivation” are keys to “success in any course”). Klopfenstein and Thomas (2010) find that when these demographic characteristics are controlled for, the claims made for AP disappear.
Public education has become caught up in goofy, childish game of of whose-daddy-can- beat-up-the-other-daddies. Remember the education “crisis” that followed the launching of Sputnik? Admiral Hyman Rickover, father of the nuclear Navy and an advocate for a rigorous, advanced education for an elite few, complained about the “ time wasted in public schools which must be made up later on.” Rickover said that public school critics would no longer let anyone “fool the American people into believing that education can safely be left to the ‘professional’ educators.” And Edward Teller, physicist, father of the hydrogen bomb, and conservative hero, warned that the Russians “will advance so fast in science and leave us so far behind that their way of doing things will be the way, and there will be nothing we can do about it.” Well, we know how all that turned out, but the “competitive” education race was on. The finger of blame for a “threat” to national security was pointed directly at public education.
A generation later, the “threat” to national security was resurrected in A Nation at Risk, which warned of a “rising tide of mediocrity.” That too was a phony claim.
The Sandia Report (Journal of Educational Research, May/June, 1993), published in the wake of A Nation at Risk, concluded that:
* “..on nearly every measure we found steady or slightly improving trends.”
* “youth today [the 1980s] are choosing natural science and engineering degrees at a higher rate than their peers of the 1960s.”
* “business leaders surveyed are generally satisfied with the skill levels of their employees, and the problems that do exist do not appear to point to the k-12 education system as a root cause.”
“The student performance data clearly indicate that today’s youth are achieving levels of education at least as high as any previous generation.”
Maybe one of these days, we’ll have the courage to think more critically and deeply, and we’ll see past the flimsy rhetoric. Maybe one day we’ll recommit ourselves to a public education system whose core mission is to develop and nurture democratic character and citizenship. That’s precisely the kind of reform we need.
And it’s also the kind of reform the current crop of “reformers” are not the least bit interested in.
LikeLike
Nice summary, democracy. It’s a shame that AP courses are now being used to rank schools. We need more honest push back from colleges. Offer the courses for interest and challenge if and when a student is ready. Steve and joemack53 have the right idea.
LikeLike
I taught high school math and science for 35 years. I had a combined calculus/physics class for many of those years, with neither officially counted as AP courses. The calculus course actually covered more than Calc AB, and students who wished to take those exams were encouraged to do so (we would have study sessions outside of class for test prep). After eleven school years of math, these kids were ready for Calculus. Physics did not cover enough to make it through the whole AP curriculum (one year of physics was the norm for our students). I saw my physics course as a way to get kids excited about the subject so they would take more physics in college.
That all changed when school rankings included AP participation. I did not care about those rankings, as my students were successful and their parents happy. I hate centralization and common curricula. Schools need to form their own identity.
LikeLike
Here we go with the broad anti-AP brush again. This story seems to be more about forcing kids to take the AP exams, not the AP courses, as your misleading post title implies. No school in its right mind would FORCE kids to take the AP exam, and the vast majority don’t. Did this school pay for the exams, or did the parents/students have to pony up the fee? That makes a huge difference.
Also, your writer should have taken a pro-active approach in dealing with her children’s counseling department. Anyone who is surprised that all institutions don’t accept all AP exam credits hasn’t done his homework. All of that information is easily accessible on the College Board’s AP Central website. I take my students through it every year.
For the record, I teach AP, and my students can drop the class if they struggle, and they are never forced to take an AP exam. That’s my school’s policy, and I’d bet the majority of AP schools are the same.
Excuse me if I come across a bit angry, but I’m also an official scorer of one of the AP exams, and I’ve spent nearly a week making sure that each student – who is someone’s son, someone’s daughter, someone’s student – gets a fair reading as I score his or her exam.
LikeLike
Nope, at our school, once you are registered for an AP course, you have to complete it and you are required to take the exam. Our school pays for the exams so the students are obligated. The kids know that most won’t count at their university but they are told they have to have them to be considered a serious student and they are given a 6 point scale when factored into GPA thus raising rank.
LikeLike
that sure is the trend…..and why?…because so many kinds were opting out of the exam….why? they took AP courses to inflate their GPAs…..
LikeLike
@Jeff Larsen: the assumption is that “AP is better” than other “college-prep” courses…..but that’s a flawed assumption. Read my comment above.
LikeLike
This whole argument is absurd. There’s no uniformity to AP classes; some are excellent, others are terrible, most are somewhere in between. There’s a very challenging test at the end (MUCH more challenging than what is typical for high school, or for most undergraduate classes for that matter), and that test helps to calibrate the standards and the curriculum. Otherwise it’s up to the teacher to make the class meaningful, as is the case with any class.
AP just means there’s a test at the end. The test results have been badly misused in the media, but as an AP teacher I share Jeff’s irritation. We’re barking up the wrong tree here.
LikeLike
And by the way, democracy, your long comment above is all about the misuse of AP. Of course titles and scores and tests and whatnot are all misused in this day and age. That’s a given.
LikeLike
At my tony suburban public high school routinely ranked in Newsweek as one of the top 50 public high schools, AP classes are pushed like drugs. As a volunteer who labeled folders for students taking the exams, there were many many kids who take 7 a day plus four on line. And I have my doubts that they are doing those alone.The high school valedicatorian bragged about her 21 APs. It’s nothing but a pressure cooker and the school isn’t worried about who takes them, they just need to keep those classes filled. The teachers who teach them have status and are free to be abusive because “well, they get great results on the AP exams”. The amount of work is ridiculous, in fact, my daughter has five incredibly in depth summer assignments for her AP Lang class that doesn’t start until fall which angers me beyond words. WHen I try to reel my daughter back, and trust me, she is only taking four which I think are too many, she loses it and says it will hurt her class ranking. God forbid they want to take an interesting elective, because if it isn’t an honors elective, they won’t take it. The flack my daughter got for taking chorus and peer counseling was enormous and she regrets it because it hurt her class rank!The race is on, it has nothing to do with better education, and it’s all about how it helps the school, not the student.
LikeLike
It is interesting how ones perspective differs depending on the state and perhaps district. In my state there is no GPA benefit for AP classes. Half of the high school valedictorians in my local high school do not take enough academic classes to qualify for admission to any 4 year state college or university.
Most of the high schools in the state are too small to offer AP classes. Even the relatively large local high school offers only 9 AP classes, but some double count (calc AB and calc BC ) and the lower level of others (Physics B not Physics C). Proximity to a university may explain some of this, however, as advanced students often simply take the university courses (no dual enrollment, however).
LikeLike
Concept of valedictorian should be outlawed as an undemocratic practice.
LikeLike
Wow, Ro. Your opening statement alone says so much.
I teach toward the other end: We were required to administer the ACT to all students for the first time this March, and I had a student fall asleep during the test. It meant nothing to him, and I understood his position. He has no intention of attending college, so this forced ACT was a waste of his time.
LikeLike
@ M. Schneider:
The ACT may be required for many students, but also it surely is mostly a “waste of time.”
The ACT is only marginally better than the SAT at predicting college success. What both measure best is family income, and colleges know it.
That illustrates a major flaw in college admissions; they willfully give grant money to students who need it least, based on ACT and SAT scores.
Matthew Quirk reported on this in “The Best Class Money Can Buy:”
“The ACT and the College Board don’t just sell hundreds of thousands of student profiles to schools; they also offer software and consulting services that can be used to set crude wealth and test-score cutoffs, to target or eliminate students before they apply…That students are rejected on the basis of income is one of the most closely held secrets in admissions; enrollment managers say the practice is far more prevalent than most schools let on.”
http://www.theatlantic.com/magazine/archive/2005/11/the-best-class-money-can-buy/4307/2/
And the net result is this: “More and more, schools are chasing the small number of students who have the money or the test scores that help an institution get ahead. As those students command higher and higher tuition discounts, they leave a smaller and smaller proportion of the financial-aid budget for poor students, who are increasingly at risk of being left out of higher education.”
As to the ACT, like the SAT, it too is far more hype than a genuine educational tool with merit.
The authors of a study in Ohio found the ACT has minimal predictive power. For example, the ACT composite score predicts about 5 percent of the variance in freshman-year Grade Point Average at Akron University, 10 percent at Bowling Green, 13 percent at Cincinnati, 8 percent at Kent State, 12 percent at Miami of Ohio, 9 percent at Ohio University, 15 percent at Ohio State, 13 percent at Toledo, and 17 percent for all others. Hardly anything to get all excited about.
Here is what the authors say about the ACT in their concluding remarks:
“…why, in the competitive college admissions market, admission officers have not already discovered the shortcomings of the ACT composite score and reduced the weight they put on the Reading and Science components. The answer is not clear. Personal conversations suggest that most admission officers are simply unaware of the difference in predictive validity across the tests. They have trusted ACT Inc. to design a valid exam and never took the time (or had the resources) to analyze the predictive power of its various components. An alternative explanation is that schools have a strong incentive – perhaps due to highly publicized external rankings such as those compiled by U.S. News & World Report, which incorporate students’ entrance exam scores – to admit students with a high ACT composite score, even if this score turns out to be unhelpful.”
So, the explanation has two general possibilities, neither one very good. The first is that college admissions staff (and top college administrators) are ignorant of the severe “shortcomings” of the ACT and SAT. If that is the case, why in the world do these people have jobs in academia? They may as well be doctors who still practice bloodletting.
The second possibility – the real answer – is that they engage in this nonsense (and it really is nonsense) willfully. Because it’s all about “image”…and money. For example, Matthew Quirk noted that former VCU president Eugene Trani used to carry “a laminated card in his pocket to remind him of the school’s strategic goal of making it to the next tier. For every year the school stays in the higher tier he will receive a $25,000 bonus…” He was – and is – surely NOT the exception. In a very real (and nefarious) sense, this is a continuation of who gets voted “best dressed” in high school. It’s the real-world equivalent of giving tax cuts to the already wealthy to “stimulate” the economy. Maybe the fictional Forrest Gump described it best: “Stupid is as stupid does.”
LikeLike
@ Ro: you’ve described it well…..and as you point out – and as I note in my comment above – it’s all about “looking good”….it’s all about the “competition” for admission to “selective” colleges, and they too are engaged in their very own “competitive” races…..all-in-all, it is most assuredly not about learning.
LikeLike
I took AP Calculus in my senior year over 30 years ago – I loved it, and got excellent grades. But the AP exam was full of questions about things I’d never heard of and I didn’t pass. I was horribly disappointed and had to enroll in Calculus 1 the next year in college – and it was the exact same material that my high school teacher had taught.
I’m not sorry I took it in high school, but in some ways it seemed like a wasted year.
LikeLike
Until we pull the root out of the educational standards and standardized testing regimes the “dandy lion”* will continue to re-sprout and spread. All this talk about ACT scores, etc. . . is like weed whacking the leaves and flower off a dandelion, it’ll come back. Noel Wilson has given us the intellectual tool to dig out the taproot of these educational malpractices that are educational standards and standardized testing. Until you folks read and understand his “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 all this mental masturbation will continue unabatedly but never getting at the heart/root of the problem-that educational standards and standardized testing is completely bogus and does irreparable harm to many students (and now teachers, schools and districts).
I challenge all to read and understand the study but I won’t hold my breath but will re-post a brief summary (which is akin to reading the Cliff Notes for Moby Dick and believing one can appreciate the whole text) and some comments from another thread.
*I don’t consider the dandelion to be a weed. Since being young I’ve always “liked” the plant for what it’s worth.
LikeLike
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine.
1. A quality cannot be quantified. Quantity is a sub-category of quality. It is illogical to judge/assess a whole category by only a part (sub-category) of the whole. The assessment is, by definition, lacking. The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing we are lacking much information about said interactions.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. This is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the
Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.” In other word all the errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. As a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms shit-in shit out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it measures NOTHING as the whole process is error ridden and therefore invalid. And the whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
LikeLike
Let’s start with the first claim: a quality can not be quantified. Hot, cold, heavy, light, far, near, strong, weak, small and large are all qualities that people have quantified. Are these attempts to quantify these qualities useless or usefull?
LikeLike
“It is illogical to judge/assess a whole category by only a part (sub-category) of the whole.” (I should leave this rebuttal to Duane.) By ignoring the second part of the argument you are setting yourself up for the blind men and the elephant joke.
LikeLike
I find the claim that quantity is a subset of quality to be at best confusing, at worst to be a useless distinction. I look forward to heating Duane’s explanation of this distinction. I also look forward to hearing how useless weather reports are because they quantify hot. As a fellow fisherman, however, I agree that quantifying heavy is a very very bad idea. My youngest child unfortunately photographed the larges large,mouth I have ever caught with the scale and it has forever put a limit on heavy.
LikeLike
I suppose it is time for someone’s wise observation: “No generalization is worth a damn.”
LikeLike
Hearing, not heating..
LikeLike
TE,
Yea, I finally got a response to what I consider to be some of the most important ideas concerning the teaching and learning process. Thanks.
Let’s start with hot/cold which is the concept of heat. Yes the quantity of heat can be measured no doubt but that is just one part of what the concept of heat is about (the accuracy of the measurement is a whole other story). Being hot or cold has many factors and to focus only on the temperature limits the description of how hot or cold one feels. Actually knowing the temperature says nothing about how a person perceives the temperature. The air temperature may be 65 degrees but depending on the time of year that may seem to be either “hot” (during a sub freezing cold spell) or it may seem “cold” (like during last summer’s searing heat wave hear where we had like a dozen days of temperatures over 100 degrees). Not only that but what a person is wearing affects the feeling of heat as does the humidity, whether there is a breeze or not, etc. . . . So that to attempt to “quantify” the concept of “heat” one can lose a lot of information/description of the very human activity of experiencing heat.
The same concern, that of limiting description of human interactions with the environment through using numerical descriptors (quantity of something) holds true for your examples of near/far (a mile away is very near if one is traveling at 100 miles an hour but very far if one is crawling on the desert sand sans water), for weak/strong (I was quite strong to my children but now that they have grown I am quite weaker than I used to be-with many factors coming into play on that one), same example of my relationship to my children can be used with the small/large dichotomy even though we can measure distance, strength (although the question becomes what constitutes being strong) and size.
What I am getting at is that to expect numerical values to be the end all be all as descriptors of human interactions with the environment is quite limiting and actually not that helpful over all. Part of the quality of human interaction with the environment can be quantified but that quantification can never be the total descriptor of that interaction and for all practical purposes really limits the said description.
But, in using educational standards (and again I ask that someone come up with a working definition that most can agree on) and standardized testing which by definition works in the realm of quantification of the very human interaction of the student with the testing device we have a paucity of description of said interaction. The lack of depth of description of the event of test taking hinders the teaching and learning process through focusing on numbers and not content (when was the last time a student got to review his or her standardized test results question by question? Never?).
To answer your last question, both depending upon the purpose of the description of the human interaction with the environment.
And what 2old2tch stated is true. We can measure the weight, height, length but that does not give us the information we need to determine the “elephantness” of an elephant. One has to really on non numerical statements to understand that beast (pun intended).
One final note: When I talk of a sub category not being able to describe completely and accurately the whole category, 2old2tch’s elephant analogy illustrates the concept quite well.
Man, going to have to get back on a more “normal” schedule before school starts!
LikeLike
Duane,
There is a difference between a metric telling is everything about how a person will perceive hot or heavy or far and a metric that tells us nothing about how a person perceives hot, heavy or far.
I keep my cast iron skillet in the oven. Would knowing the temperature of the skillets handle be useful in predicting a trip to the hospital if I asked you to grab the skillet out of the oven? I may not know precisely how it felt to you to pick it up, but by knowing the temperature I can make a prediction about how badly burned your hand will be.
LikeLike
“Man, going to have to get back on a more “normal” schedule before school starts!”
Naps and 2AM bedtimes. I have apparently reached the expiration date for anything but subbing. Those 6AM calls are not my cup of tea. Why can’t people decide to be sick the day before?
LikeLike
To quote myself-ha ha!:
“Actually knowing the temperature says nothing about how a person perceives the temperature.”
In much the same way knowing a test score says nothing about how the student comprehends (perceives) the subject matter. It only says a little about how a student interacted (the event of taking the test) with a particular test on a particular day. The paucity of information (description of said event) is actually astounding. To lend credence to a score in this situation is hubristic at best, harmful to students at worse due to the subjectivization (internalization) factor.
LikeLike
“I keep my cast iron skillet in the oven. Would knowing the temperature of the skillets handle be useful in predicting a trip to the hospital if I asked you to grab the skillet out of the oven?” and “but by knowing the temperature I can make a prediction about how badly burned your hand will be.”
Your prediction may or may not be accurate as perhaps I have already made it a habit to use an oven mitt/pot holder whenever taking anything out of the oven. Predicting the future with such little information (which is what the usage of a standardized test score attempts to do-you know higher scores on ACT means better college completion rates, right?) is fraught with logical dangers and one is better served by having a narrative along with the “hard numbers” in order to “predict the future” and even then predicting the future is a futile business or else I’d be at the casinos making a ton of money.
But I’m not quite sure what your example has to do with illogically quantifying someones learning/knowledge (a quality of human interaction).
LikeLike
I took the claim to be broader than simply to be about education, but that no quality can be quantified. If you had said completely quantified I might come close to agreeing, but I think your position is that no quality can be usefully quantified. Do you mean at some qualities like hot, heavy, far, etc. can be usefully quantified while others, such as knowledge of Spanish can not be usefully quantified?
LikeLike
Boy I love string beans! TE,
What I am pointing out, obviously not very well that it is a logical inconsistency for a sub category to be the descriptor of the category itself. So that, yes, no quantity can totally describe the quality of which it (quantity) is a part of. That would be a paradox or falsehood. A quantity descriptor can be used as part of the overall discussion of the quality of something but cannot override or even be a single end all be all description. Temperature is a quantity and is one descriptor of the quality of heat but not the end all be all in regards to describing heat.
To respond to “Do you mean at some qualities like hot, heavy, far, etc. can be usefully quantified while others, such as knowledge of Spanish can not be usefully quantified?
Yes heat, heavy, distances etc. . . can be quantified but that is only one small piece of information in the very human interaction with the environment and leaves out many other considerations. So one ends with a less than satisfactory explanation of the human interaction. And definitely yes, to the teaching and learning process as indicated by “knowledge of Spanish cannot be usefully quantified”.
My claim is that the “numerization”/quantification of human interactions is quite lacking in explaining said interactions and that a narrative is more likely than not to give a better description of that interaction. Now, each description will be idiosyncratic with each person having various narratives of said interaction. And these can’t be placed in the realm of scientific explanation/quantification which is what psychometrics attempts to do with standardized testing.
LikeLike
Quantifying a quality is not a a subcategory of anything but a measure of some some aspect of the quality. Some measures are useful, some less useful. That a measure does not capture everything about a quality does not mean that it is not useful, however.
That is true for education as well. Surely you would be willing to say that at this moment in time there are native speakers of Spanish and there are people who have no knowledge of Spanish. By dividing them into two groups you have quantified them.
LikeLike
My grandson got a 36 on the math section of the ACT. Am I supposed to be sad about that?
LikeLike
No, personally I would be quite happy for your grandson! It shows he knows at least how to take a standardized test quite well (well, at least the math portion). But given the inherent invalidity of the whole process it would be a pyrrhic congrats.
LikeLike
Duane, do you not think that there is a good chance that his grandson actually knows something about math that is beyond the typical?
LikeLike
Duane, these points are all absurd.
This doesn’t make sense. Since when did tests measure “quantity”?
Tests don’t measure the full aesthetics and qualities of those interactions. No one out there is arguing that they do. Even freakin’ Pearson and other nefarious test companies argue that they do.
Any given interaction has meaning beyond merely that interaction. Don’t think so? Would you have a doctor operate on you if you knew that doctor was guilty of gross negligence in a previous operation? If not, why not? Just because he messed up someone else’s liver doesn’t mean he’s going to mess up yours, right?
Of course it’s true that a test or a series of tests don’t define a person, but no one is arguing that.
Every test is imperfect. To argue that there should be no tests because tests aren’t perfect is like arguing there should be no cars because cars aren’t perfect.
Oh come on man, are you serious? What kind of desperate grad school jive is this? Are you sure he doesn’t mean that “error” is an area of opportunity for growth? That’s where I thought he was heading, but your conclusion there took the wind out of my sails.
Okay, so: if it has any errors in it it’s “invalid.” The only place where that’s absolutely true is logic and mathematics. This “if it’s not a perfect test then it’s a useless test” premise is just ridiculous. Something doesn’t have to be perfect to be meaningful.
You’re repeating the same absurdity over and over again. Again: because a test, or anything else, isn’t perfect, does not make it meaningless. Shakespeare’s plays aren’t perfect. Does that make them meaningless?
“Sorting and separating” does not necessarily mean some benefit more than others. If one student has gone off to art school and another student has gone off to engineering school and a third has gone to a trade school, they’ve been sorted and separated. That does not mean that one necessarily benefits more than others.
I knew it! I knew Foucault must have something to do with this gibberish.
But that’s a perfect teachable situation. When a selective school doesn’t admit a student because of the quantifications on her application, that doesn’t mean she’s of less value as a human being. Kids need to learn not to evaluate each other’s worth by quantifications, whether monetary wealth or test scores. That’s a key part of maturing and it’s our job to help them internalize that, not by getting rid of tests but helping them understand the proper role of such quantifications.
LikeLike
*sigh* Sorry, I messed up cutting and pasting. Ignore the above, I’ll try again.
LikeLike
Below is what I meant to post. (I wish we could edit here!)
LikeLike
O my God, it did it again!!! I give up. (For some reason it’s chopping off text I’m citing and responding to.)
LikeLike
Duane, these points are all absurd.
This doesn’t make sense. Since when did tests measure “quantity”?
Tests don’t measure the full aesthetics and qualities of those interactions. No one out there is arguing that they do. Even freakin’ Pearson and other nefarious test companies argue that they do.
Any given interaction has meaning beyond merely that interaction. Don’t think so? Would you have a doctor operate on you if you knew that doctor was guilty of gross negligence in a previous operation? If not, why not? Just because he messed up someone else’s liver doesn’t mean he’s going to mess up yours, right?
Of course it’s true that a test or a series of tests don’t define a person, but no one is arguing that.
Every test is imperfect. To argue that there should be no tests because tests aren’t perfect is like arguing there should be no cars because cars aren’t perfect.
Oh come on man, are you serious? What kind of desperate grad school jive is this? Are you sure he doesn’t mean that “error” is an area of opportunity for growth? That’s where I thought he was heading, but your conclusion there took the wind out of my sails.
Okay, so: if it has any errors in it it’s “invalid.” The only place where that’s absolutely true is logic and mathematics. This “if it’s not a perfect test then it’s a useless test” premise is just ridiculous. Something doesn’t have to be perfect to be meaningful.
You’re repeating the same absurdity over and over again. Again: because a test, or anything else, isn’t perfect, does not make it meaningless. Shakespeare’s plays aren’t perfect. Does that make them meaningless?
“Sorting and separating” does not necessarily mean some benefit more than others. If one student has gone off to art school and another student has gone off to engineering school and a third has gone to a trade school, they’ve been sorted and separated. That does not mean that one necessarily benefits more than others.
I knew it! I knew Foucault must have something to do with this gibberish.
But that’s a perfect teachable situation. When a selective school doesn’t admit a student because of the quantifications on her application, that doesn’t mean she’s of less value as a human being. Kids need to learn not to evaluate each other’s worth by quantifications, whether monetary wealth or test scores. That’s a key part of maturing and it’s our job to help them internalize that, not by getting rid of tests but helping them understand the proper role of such quantifications.
LikeLike
Jim,
I understand your/our frustration with editing here! Let me have at what you say.
First: “I knew it! I knew Foucault must have something to do with this gibberish.” Ad hominem argumentation is one of the weakest there is. What I argue is that students “internalize” what we the teachers, schools etc. . . tell them what they supposedly are through practices such as grades, standardized test scores, etc. . . We are taught from very early on to respect those in authority “over us”, that those in authority know of which they speak. How can a primary school age person begin to counteract what is be a completely false narrative about said person. Well, the vast majority can’t as they don’t have enough life experience to counteract this social fact. Perhaps you were one who was told during his early schooling that you were an “F” student, that you were “beginning” and not “proficient”. I doubt it. But by the time a student takes in all this labelling of him/herself, don’t you think they come to believe it because it comes from those in authority? That is internalization which is what Foucault points out as subjectivization. I try to give credit where credit is due and Foucault seems to be the one who explicitly pointed this out. Perhaps if I would have just said “internalization” you wouldn’t have have gotten yourself into a dither.
““Sorting and separating” does not necessarily mean some benefit more than others.” It sure does. Are there not rewards-scholarships, special treatment and awards given to those at the top of the academic heap while at the same time there are sanctions-no scholarships, no diploma etc. . . and condemnation/social ostracizing? Yes, sorting and separating does necessarily mean some benefit more than others. How can you deny that???
” What kind of desperate grad school jive is this?” Can you say ad hominem again??
“Okay, so: if it has any errors in it it’s “invalid.” The only place where that’s absolutely true is logic and mathematics. This “if it’s not a perfect test then it’s a useless test” premise is just ridiculous. Something doesn’t have to be perfect to be meaningful.”
You introduce a straw man argument here. I, and Wilson, never said anything about a “perfect test”. If I have please point out where. What Wilson is saying (and as I pointed out my version is like reading the Cliff Notes version of Wilson’s study with all the inherent limitations that has. It’s like reading the Cliff Notes of Moby Dick and claiming to know what Mehlville was trying to convey in that work.) is that using the test makers own logic (and he at one time was one of those test makers) one can rip apart their justifications of validity of the process, and Wilson does exactly that.
“When a selective school doesn’t admit a student. . . ” Again a straw man argument as I never referenced “selective schools”.
Four rhetorical points using two faulty argumentative devices does not a good argument make.
LikeLike
AP is not for every student, but if your son has learning differences and has NOT been accommodated by the school, that doesn’t mean that he should not have taken the AP courses. The school has to accommodate documented learning differences, and find children with those differences *regardless of intellectual ability/IQ/level of coursework*.
My son just got a D in an AP class this past quarter, and he had been refused a 504 plan this past spring. I do firmly believe that if he had a 504 plan, he would have been able to get a C or even a B as his medical problem relates to memory lapses especially regarding schedule changes and days off.
We are contemplating getting legal representation to force the school to accommodate his medical condition. Another problem we have is that he plays sports at a high level, and frankly the school does not believe that he is “ill in any way”. Two school personnel have said that “if his medical condition is such a problem, he should take easier courses” which is in violation of 504 rules.
LikeLike
How does a school have valedictorianS? I thought that word just meant the student with the top GPA. Ties are rare, no? [I graduated in 1984, so if things have changed let me know.]
I loved the AP classes I took in the early 80s (Fairfax County, pre-TJ). Got a 5 on chem and calc and had mastered the material. Had no problems doing calc 3/diffyQ and organic chemistry at the Naval Academy. Yeah it wasn’t MIT or Cal Tech, but it was pretty decent undergrad: standard approach to classes and not watered down from traditions going back decades.
I really felt the AP tests were pretty well done. Graders separate from teachers. National exam. Psychrometric calibration. Essay and MC.
Has Calc BC changed since the 80s? I still have my 4th ed TF Elements of Calculus book. And it is extremely similar to the content and problems and even style of my dad’s 1940s War Dept Granville text. It’s not like anyone has discovered a new derivative of sinx. The Granville and TF even had the exact same content in terms of series and the baby DiffyQs (2nd order with constant coefficients). And the TF was even designed to match the AP test.
I guess how much has AP gotten worse? Or always was bad and I didn’t realize it?
BTW, I do agree that APs should not be pushed on kids who don’t want them. And there should not be a GPA bonus.
LikeLike
Dual credit is a much better program, and the kids get credit for the course regardless of whether they pass an exam at the end. That makes the courses easier in most cases. Why spend a whole semester in the class and risk not doing well on the final? There’s also the rumor that many kids cheat their way through AP. Pointless. DC is regular college. You pass, you pass, you don’t, you don’t.
LikeLike