By now, almost everyone knows that the College Board offered a shortened version of AP exams–only 45 minutes–and that thousands of students took the exams at home, online, only to have their answers rejected. When asked about this phenomenon, which was so deeply upsetting to the affected students, the College Board responded that the problem was the students’ browsers. Some students (including one who commented on this blog) pointed out that they took more than one AP exam, and their answers were accepted for one exam, but rejected for the second or third.
Mercedes Schneider writes that something is wrong with the College Board, not the students’ browsers. This is not their first technical failure, nor is it likely to be the last. The College Board says that 99% of the students who took AP exams submitted their answers successfully, but we have to take their word for it.
Should we?
Diane Ravitch's blog 
Exactly what happened to one of my students. She had never had trouble submitting for any of the online practice tests, her earlier AP tests, or the first question on her AP Physics test. But for her second question on AP Physics tests, she was done with eight minutes to spare (students are told to submit with at least five minutes left)…but the submit link wasn’t lit up for her. She panicked as she watched the eight minutes run off as she could only helplessly sit there. Upon registering her complaint, she received a form letter from the College Board saying exactly what you’re saying here. Less than 1% of tests had this problem, which was likely due to her browser. She can re-take the test in 30 days, but she (and this is a top student) says that her motivation now is at an all-time low. She’s not psyched to study for AP Biology test this week or the make-up Physics test.
Thanks again, David Coleman.
What browser was she using? The fact that this happened with 8 minutes to go is strange, although I received a report yesterday from an AP Computer Science teacher that one of the “buttons” on a web page in that test was non-functional.
Most cases that I know personally stemmed from students doing what they tend to do in class on a very hard test – work right up to the last minute in panic. The precise instructions to students from the College Board are quoted in my blog article (probably below this response) and will be used as a CYA defense in these cases. However a freeze at 8 minutes to go sounds unusual as I highly doubt the majority of students would have been submitting at that time on an AP Physics 1 test which has the lowest pass rate on all AP tests. I had one of my AP Physics 1 students tell me that they submitted at 3 minutes to go without a problem, but friends who submitted later than that were hung out to dry. I would be grateful if you could talk to the student again and get precise details. Thanks in advance.
I just asked her teacher, and she didn’t know. Teacher says Physics 1 had more trouble than any other AP test to this point (having talked to other teachers). When I get the chance to talk to the student, I’ll get more details.
Thank you. Look forward to hearing from you.
As I mentioned above, AP Physics 1 has the lowest average score of any AP test, so one might expect more students working until the last minute on it.
So why didn’t the College Board warn that any submissions sent during the last minute were likely to be rejected?
I’m guessing that they probably weren’t completely sure when the capacity limits might be hit as this would be dependent upon the varying numbers of students taking each exam. I don’t know how much stress testing they did of the system; on the surface of it, it doesn’t look like much, but they would have probably had to pay consultants to develop an automated test system akin to a hacker’s “Denial of Service” attack against their website to get upload timing data. This approach could possibly give them a worst case scenario, but for less popular subject exams that amount of worst case time would have been overkill.
There are clearly a lot of problems implementing an online test like this in a short time frame as they have painfully learned. Unfortunately the cumulative stress levied on a million students was even worse.
Diane, students were warned to begin the submission process before the 5-minute remaining mark. Some of my students successfully turned in with under a minute left. This particular student was one of two who turned in with more than five minutes remaining.
Talking to the teacher, apparently the widespread issue with Physics 1 has led to a hypothesis that there was a bug in one of the many (10?) versions of the test preventing the second question answers from being submitted (where submission button was not even available).
That’s odd! Not sure how a defect in one version in ten would disable all buttons in the others?!?!! Most be some interesting software code 😉!
I may not have been clear. The teacher told me that it seemed like just about all teachers had about 10% of their students unable to submit answers for question #2 on the AP Physics 1 test (while having no trouble submitting answers for question #1). The thought was that every student who had that particular version of the test (like test #6 of 10 possible) faced the same bug. The problem was that there was no submit button to click.
OK, that makes better sense! Thanks!
David, my student just got back to me. She was using Chrome (“and, yes, it was updated”). Appreciate your interest.
Thanks for the info!
If Chrome is not acceptable for CB, what is?
” What browser was she using? ”
David, why engage in the narrative offered by the CB?
“Only 1% of the people couldn’t cast their votes during the 2020 elections due to some technical glitch.”
“Only 1% of the people who are executed are completely innocent.”
“Only 1% of the people are filthy rich, so why don’t we leave them be and focus on taxing the 99%?”
“Only 1% of the population lives in nursing homes, so why waste time with regulating their operation.”
“Only 1% of the population would use vouchers for their kids’ education, so why not let them do what they want?”
“Only 1% of the population are murderers so why waste resources and energy on gun laws?”
“Only 1% of the world’s population lives in Vietnam, so why worry about the war there?”
“Only 1% of the corona patients died because of incompetent handling of the pandamic by the Administration, so why waste time blaming them?”
There are correct and incorrect moral decisions to be made, and percentages do not measure their correctness level. In fact, there is no way to measure moral issues.
Why do we allow anybody address moral issues based on the same language as we evaluate correct functioning of a software, dishwasher, car, pesticide?
The reason that I asked was because the student submitted with 8 minutes to go, well before the uploading crush began, so I was simply curious about what might have gone wrong.
I understand, David.
Mate
You forgot an important one.
Less than 1% of the people at College Board actually give a #@&! about students.
No one makes me LOL (literally) more than you, SDP!
(David, my student read posts at your blog posts and agrees with most of what you wrote. And re-stating that her issue had nothing to do with bandwidth but with the fact that there was no submission button to click.)
Understood. I am not denying that there were other technical issues in addition to the capacity problem, but the capacity problem still seems to be the predominant problem from everything that I have seen, contrary to the CB’s initial reports.
However, if they thought that anything in the article was inaccurate, please have them contact me and I will make corrections.
Also agree with your points, Máté. Though it would break character, I would have liked some more admission of their own fault in the matter from the College Board. Asking for far too much, I know.
I had one student who successfully uploaded the first question and the second wouldn’t upload. That’s not a browser problem, and she had plenty of time to spare.
I had another student who couldn’t even get into the test. Supposedly, there was a rumor that the test just before ours could be checked after the test was over, so tens of thousands of students rushed back onto the AP site, colliding with my students trying to get in. One of my students tried getting in for 30 minutes and could never get in. She’s devastated.
Clearly we shouldn’t believe them, but they will next try to defend themselves in the manner that I explain at https://eduissues.com/2020/05/16/update-on-ap-exam-uploading-issues/, saying that they provided warnings to students about not waiting until the last minute to upload. As I point out, these warnings could have been much more explicit, so that defense also should be relegated to the “excuse” category.
David,
If the CB anticipated that papers submitted in the last minute would be rejected, they should have warned students. Insane deadline.
I just learned that students in Singapore took the exam at 4 am because of CB demand that all students everywhere in workd take it simultaneously.
Yes, I also mentioned both of those items in my blog article. The problem is that they should have added more server and network capacity after problems first started coming in early last week. If they are hosting the tests on an Amazon or Microsoft cloud service, for example, this could be possible, though it might be pricy. The fact that this problem was still occurring on Friday and looks like it may continue throughout next week is the real travesty.
If the College Board is hosting the tests on their own in-house computer systems, then this probably won’t be fixable in a short time period, so watch for the train wreck to continue all this coming week.
Finally, some tech people will defend them saying that they had to respond to the pandemic crisis in short order and mistakes were bound to happen, especially since this was their first attempt at an online test.
If the College Board was a small, underfunded start-up, one might accept such an excuse, but as a $3 billion in revenue organization (Mercedes documented the number in one of her articles a while back), capacity planning would have had to be part of setting up the system. Someone obviously would have been forced to provide a rationale for how many servers and how much network bandwidth would be needed. It is not possible for the online test system to have been set up without such a discussion.
Adequate planning might have cost money…
If this kind of mistake had been made in a Wall Street investment bank IT department, heads would have rolled…
I am also skeptical of the “Browser ate your homework ” claim.
But if it was indeed an inadequate bandwidth issue, the claim by college board that “over 99% of students successfully submitted their AP exam responses” (on first day of testing, may 12) raises a question.
How likely is that they just happened to end up on the “hairy edge” of having sufficient bandwidth? I’d have to say “not very.”
If they did not prepare adequately for bandwidth , I would not expect them to have come within 1% of 100%. While it is certainly possible that they just got lucky, it’s not very probable.
And if they did prepare, I would expect them to have built in a significant safety factor, say double or triple the estimated required bandwidth. Under the latter scenario, it would also have been possible but not probable that they would have come within 1% of 100%.
So, if the unuccessful uploads were not due to browsers or something else and were indeed primarily a bandwidth issue, id have to say I am skeptical of the “over 99%” claim. While certainly possible, it’s not very probable.
But the “over 99%” claim would be the easiest thing of all to verify.
But it would undoubtedly take a court order to get the data released to an unbiased third party for analysis.
There is no way the CB or anybody could control 100% of online tests and their submissions. The whole thing depends on thousands of hardware, infrastructure, millions of lines of computer code most of which are not under the control of CB. If any testing company claims that their online testing is foolproof, they are lying.
But if online testing cannot be done 100% correctly, then it cannot be allowed.
In general, no testing of kids should be allowed which doesn’t allow real time correction of errors. Hence we need to talk about tests which are completely controlled and evaluated by the test taking kids’ teachers.
Ironically, even Pearson’s test booklets have contained errors in the past. I like your reasoning though and we should apply it across the board, banning all these types of exams, paper or electronic. Let the teachers determine AP credits.
It’s almost unimaginable that such control would be handed over to teachers. That’s how far we’ve gone away from teacher autonomy. I think, if they interviewed parents now, they would trust a testing company more to run and evaluate AP tests than their kids’ teachers.
Well, the parents trust the teachers enough to write the AP review books, undoubtedly because they understand that the teachers know the AP material as well as (if not better than) anyone else.
One of my high school math teachers (Dave Bock, stellar teacher) co-authored the most popular AP Calculus review book, which has been used for decades.
Coleman … 🤮
🤬🤑🤯💰💸💵💣💩💩💩💩💩💩
🤬🤑🤯💰💸💵💣💩💩💩💩💩💩
Ha, ha–such a monumentous $t$***m that you had to post your Emoji parade **twice, flos56!
(I know it was done in error, but, as the White Stripes’ song goes, it “bears repeating!”)
The College Board royally screwed up. Then they lied about it. And they showed their utter callousness by treating the students’ having to take the tests over again as no big deal. They clearly don’t understand ANYTHING about how much emotional investment many kids put into that test administration, which has been a deadline they’ve worked for and been anxious about for an entire year.
Led by the idiotic perpetrator of the Common Coring of U.S. education, this vile organization publishes the SAT, which doesn’t even predict college success as well as high-school grades do. Another scam.
Dump the Core. Dump the SAT. Dump AP. Let rich parents in the suburbs find another way to go about claiming that their kids are better than those of ordinary mortals.
COLLECTIVE lawsuits needed to make the entire process invalid: testing is a shockingly elitist game in the best of times.
The nation was sold these tests, back in the early part of the 20th century, as ticket to meritocracy–a way to fulfill the Jeffersonian ideal of an aristocracy not of birth but of talent. And for some, indeed, they have been. I was lucky. I tested well, and this was a ticket for me. But I’ve come to understand that the tests are problematic on many levels and that they, in fact, perpetuate discrimination against kids who were brought up without advantages. They are great at predicting what ZIP Codes kids live in. And, of course, they were conceived of for use in education in the U.S. by a white supremacist eugenicist, Carl Brigham. They kept rich whites on top then, and they do now, and this is not because the kids of the rich whites are superior in inherent ability.
Steven Gould pointed out in his great book on this subject, The Mismeasurement of Man, that when IQ tests were first given to immigrants in the U.S., those who scored lowest on them were Hungarians and Jews from Eastern Europe–two groups that today have among the highest scores. IQ tests, the SAT, etc., are not culturally independent, a priori measures of intelligence and aptitude. People can be taught to take them well.
and that is why teaching kids how to take tests well is a huge and very profitable business
I just received this note from a parent a few minutes ago:
Looks like emailing if you get get an upload error message is now an option. Message from the College Board today.
Submitting Exam Responses Beginning Monday, May 18, and continuing through the makeup window, if your student is unable to upload their response due to a technical disruption, they’ll be able to email it to us immediately following the conclusion of their browser-based exam. If this happens during their exam, they will follow instructions on how to email their response on the page that says, “We Did Not Receive Your Response.” The email address that appears on this page will be unique to each student.
Good solution. Should have been in place from the beginning.
That would have required foresight or an ability to problem solve (Something that is required for those taking the AP exams not those administering them).
A teacher on a AP Facebook group I’m on for a specific test asked for we teachers to put in our numbers of how many couldn’t finish the test, vs. how many students we had that were able to finish the test. She got a sample of about 18,000 students, and the rate of students unable to get into and/or finish the test was about 6.5%.
My “favorite” was the condescending Tweet by one of the College Board head honchos that said that it was a “myth” that a lot of students had problems with the test (the test my students took was Tuesday, the 2nd day of the testing window).
I suspect their <1% number was their pre-established PR propaganda.
Don’t blame the students……..What is the alternative strategy? Peggy Funkhouser
Even if the submission was 99% successful and even if somehow the kids’ browsers misbehaved, how is this acceptable?
Percentages are meaningless here and we should not even consider weighing the correctness of the CB’s numbers.. This is not some kind of medical or drug test where statistics is the de facto tool to evaluate success, this is not an experiment where 99% agreement with theory is considered promising, it’s not voting which rely on percentages for decision making.
If the CB’s method of giving the test and collecting the answers don’t work for everyone, CB has a problem. And online testing will never work everyone. It’s can’t.
Indeed, the problem is generic and is present everywhere where technology is used where it shouldn’t be used. Technology, software, hardware work only a certain percentage of time. Hence it shouldn’t be used to cast votes, test and evaluate education, to determine who lives or dies. All these things should be done locally, supervised by a small group of people which can immediately step in and correct any mishaps and errors in real time.