Russ Walsh, a literacy expert, is analyzing the reading passages on the PARCC test. In his first post on this topic, he reviewed sample questions from the test for readability levels; while the Lexile measure was aligned with the correct grade level, other measures showed the readability to be about two grade levels above the students’ actual grade. In the following post, Walsh looks at the kinds of questions that are asked.
He writes:
Readability, however, is about more than the level of difficulty of the text itself. It is also about the reading task (what the student is expected to do with the reading) and the characteristics of the reader (prior knowledge, vocabulary, reading strategies, motivation).
In this post I will look at the second aspect of readability that must be considered in any full assessment of readability: the task that the reader faces based on the reading. Since this is a testing environment, the task is answering reading comprehension questions and writing about what has been read.
In any readability situation the task matters. When students choose to read a story for pleasure, the task is straightforward. The task is more complex when we ask them to read something and answer questions that someone else has determined are important to an understanding of the text. Questions need to be carefully crafted to help the student focus on important aspects of the text and to allow them to demonstrate understanding or the lack thereof…..
Whenever a new test is rolled out, we know through past experience that test scores will go down. Over time schools, teachers, and students adjust and the trend then is for scores to go up. It will be no different with the PARCC tests. As the scores rise, some questions will arise like, “Have we been focused on the right things in these tests?” and “Have the tests led to better, more thoughtful readers?” Based on my analysis of these test questions, I am not confident.
Diane Ravitch's blog 
Reblogged this on David R. Taylor-Thoughts on Texas Education.
LikeLike
With nationwide, standardized testing, I keep seeing visions of lemmings going over a cliff. Or maybe it is the Midwest temperatures causing brain freeze.
LikeLike
“2.The questions indicate that the test designers are very focused on the idea of “citing textual evidence” as a dominant reading skill. Textual evidence is repeatedly cited in the CCSS and so I suppose it is unsurprising that it plays a prominent role in this CCSS aligned test. Citing textual evidence is a highly valuable and necessary reading ability, whether it deserves to be addressed in almost fifty percent of the questions on a test of reading comprehension is certainly open to debate.”
The notion that citing textual evidence is a necessary and important skill for elementary school students is ridiculous. At this level it simply requires unnatural and redundant thinking. Why should the CC place such a premium on the skill of restating the obvious?
Over half (18) of the 34 test items analyzed were classified as, “Author and You” Where the answer is not in the text. The reader needs to think about what s/he knows, what the author says, and how these two things fit together. Bloom would call this the inferential level.
53% of test questions require students to make “correct” inferences; items in which the answer is not found in the reading passage. That’s not reading comprehension – That’s asking kids to be mind readers.
Only 3 (out 0f 43) items asked about the “main idea” of a reading passage.
These tests are littered (as in garbage) with poorly written gotcha questions; many of which, as Russ points out, are double jeopardy items (16/34).
Russ does not address whether or not these tests are appropriate as minimal competency measures for ALL students – nor does he address the glaring fact that they are a completely invalid measure of teacher effectiveness. Most of the abstract skills being tested are nearly impossible to teach without turning ELA class into Test-Prep 401.
LikeLike
I imagine that the 53% that require inferences favor students from a higher socioeconomic level. A greater base of prior knowledge produces a greater the ability to infer.
LikeLike
NY Teacher says “Russ does not address whether or not these tests are appropriate as minimal competency measures for ALL students – nor does he address the glaring fact that they are a completely invalid measure of teacher effectiveness.”
Correct. None of the tests in use and designed to measure student achievement also provide evidence that these tests are “instructionally sensitive.” Instructional sensitivity is a concept that has simply been ruled out of discussions of testing (with the exception of a few researchers, notably James Popham).
Teachers are too often being judged on factors that influence test scores but the variance is these scores is not usually the direct result of their instruction, even if they are teaching to the test. That is one reason why the tests are really invalid for measuring teacher effectiveness.
An approximation of an instructionally sensitive test is one that has no content and no skills students are likely to have been learned in prior grades or subjects, or outside of school and preferably only those those taught by the teacher of record for the current course or grade.
Most tests of student achievement assume that the tested content has only been learned in school and particularly in current course (subject/grade level). Few policy makers or test-makers want to anything about designing instructionally sensitive tests because that would requite a reset button for almost all of the assumptions made about teaching, learning, testing, and whether we are really talking about education or Skinnerian training.
LikeLike