Sue Legg used to be in charge of assessment for the state of Floridea, before the current reign of educational inanity took hold. She writes:

 

QUANTITY VERSUS QUALITY IN TESTING

Back in the late 90s, psychometricians were searching for ways to counter attacks against multiple choice testing. Opponents argued that essay exams were needed to measure higher order thinking skills. Too often multiple choice tests relied on fact based information or esoteric vocabulary as indirect measures of analytical reasoning. In response, we added essay components to state-wide assessments. Essay scoring, however, had its own problems. Readers are people–they have their own biases about essay scoring. I remember Mark Reckase, then at ACT, saying that it would take the average score of 7 readers to achieve the reliability of a multiple choice test. It was a conundrum. Do we trade the validity of the essay that could measure critical thinking for the reliability of a fact-based multiple choice test?

Instead, we looked for ways to combine and improve the two types of questions. At the same time, we worked to capitalize on the possibilities for using computers to improve testing and instruction. I remember the excitement over the possibilities of using computers to adapt multiple choice tests to better measure students’ abilities without spending endless hours in testing laboratories. Essay scoring could also be more efficient if computers could substitute for some, but not all human readers. Testing could become part of the learning process as well as less onerous.

Enter the law of unintended consequences. Instead of helping diagnose student learning problems, testing became the face of an accountability driven political agenda to reform rather than fund educational change. Even though testing has now become extremely expensive, it costs less than attracting and retaining high quality teachers and providing the support struggling students need to learn.

We do not have much to show for all the emphasis on accountability driven reform. Serous efforts to evaluate progress have come up short. Excessive testing does not make better teachers, better schools, or better students. Instead, legislators have created a huge profit driven testing industry, but even that industry cannot offer the advantages of new, more efficient ways to measure student learning for everyone. States are not willing to limit the frequency of testing to make it meaningful and affordable. Even the infrastructure needed to deliver these online, adaptive exams is limited.

Equally bad, the temptation to recreate the old paper and pencil drill and practice worksheets on the computer and call it innovative instruction is proving hard for online learning companies and legislators to resist. They are easy to develop and profitable. The question is how we can force reform minded legislators to recognize that there is a difference between quality and quantity. Online learning and testing has possibilities to improve teaching and learning that are beyond exciting. Unfortunately, they may be beyond the critical thinking and problem solving skills of the current wave of school reformers. One wonders if legislators are up to this test?