Parents in New York want to see the contents of the Pearson tests that are aligned with the Common Core but officials are adamantly opposed to releasing the tests.
Parents want to review the tests to see if the questions and answers are reasonable. That is not going to happen. Teachers have been warned that they may be disciplined if they reveal any questions. Unless students spill the beans, there will be no review of the test content. There will be no Pineapplegate this year, as there was last year.
It is odd that the state is so quick to defend Pearson’s right to privacy and yet so fast to release confidential student information to Bill Gates and Rupert Murdoch.
Diane Ravitch's blog 
Well of course The Pearson is “adamantly opposed to releasing the tests”. If they had to then they’d have to make all new ones every year and that would cost them more of their hard stolen-oops I meant earned monies. Not to mention the fact that the whole world would then see just what a total mess their tests are.
LikeLike
The errors and handful of poor questions in TN’s practice EOC book don’t exactly assuage my concerns about the actual test.
LikeLike
Would a FOIA request be appropriate here?
LikeLike
To me, as a tax payer, if I paid for it I should be able to see it.
LikeLike
FOIA is federal. Not sure that would apply here. Something similar at the state level, perhaps?
LikeLike
As per page 18 of the administrator’s testing manual, parents can view their child’s responses to the open ended questions on the ELA exam. This opportunity will be gone once the tests are mailed back to the state. Parents should and must take advantage of this opportunity now.
LikeLike
What a concession, only in the open ended questions. Absurd!
LikeLike
I tried the FOIL in the fall. http://atthechalkface.com/2012/10/19/i-foiled-the-test-seriously-are-we-violating-national-security-here/
LikeLike
A lawsuit will change their minds. A suit was won in Texas that forced the state to release the tests every other year.
LikeLike
In the meantime, ask a few NY children with photographic memories to tell you the questions.
LikeLike
I gave the ELA last week. The test wasn’t remarkable other than the fact that the readability is 2 or 3 grades higher than 5th grade. Not sure what the goal is here other than make 90% of the kids feel stupid.
LikeLike
Any response to this from Sunday’s NYT?
LikeLike
Murdoch = Phonegate.
Gates = Nerdgate
Pearson = CriminalSyndicateGate
LikeLike
I just finished reading the book “Making the Grades”. The author worked for Pearson for 15 years. People should demand to see the tests. We need an independent audit of the how these tests are scored.
LikeLike
Iheartdurham: I enthusiastically second your recommendation!
🙂
In all humility, I suggest you now buy and read Banesh Hoffman’s THE TYRANNY OF TESTING (1962). This book was published 47 years before Farley’s (2009) and anticipates (often in small details) the newer work. Shows that even after half a century the high-stakes standardized testing industry can’t get it right. Except, of course, for one thing: racking up lots of $tudent [?] $ucce$$.
As an added bonus: the two books are both inexpensive, slim, well-written, witty and meant for a broad public.
Thank you for posting.
LikeLike
Krazy TA – I went to my local library and read Todd Farley’s book for free. I’ll do the same for Hoffman’s book. Thanks:)
LikeLike
KrazyTA: Thanks for the recommendation. I look forward to reading Hoffman’s book. I happened to find Farley’s book in my library while in the education section. I don’t understand why this book (and apparently Hoffman’s) didn’t get more press (a la Unsafe at Any Speed).
Others – I didn’t know my link would show the amazon button on the bottom so sorry about that.
LikeLike
KrazyTA,
“Shows that even after half a century the high-stakes standardized testing industry can’t get it right”.
There’s a reason that they can’t get “it right”. Thats because the whole process of educational standards and standardized testing is so rife with error that it is completely invalid as shown by Noel Wilson in his 1997 dissertation “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700 .
LikeLike
My understanding is that, based on legal precedent, in NY, tests must be divulged publicly after a certain period of time. Moreover shouldnt one be zble to use the Freedon of Info act to force the release? Can someone check if this is correct.
LikeLike
Teachers should photograph the tests with their I-phones and upload them online, as a move towards a much needed EduWikileaks, and to break the spell of fear and repression surrounding them.
As Oliver Wendell Holmes said, “Sunlight is the best disinfectant.”
LikeLike
They should send them to a relative/friend to post…it is getting fairly scary for teachers out there right now.
LikeLike
Well understood, Cee Mor, I commented with the assumption that teachers would anonymously send the photos/ copies to sympathetic bloggers.
I have a suspicion Norm Scott at EdnotesOnline might be receptive to the idea.
LikeLike
As per page 18 of the administrator’s testing manual, parents can view their child’s responses to the open ended questions on the ELA exam. This opportunity will be gone once the tests are mailed back to the state. Parents should and must take advantage of this opportunity now.
LikeLike
Agreed. As the anti-reform conversation has been suggesting, this is a job for some old-school civil disobedience. This is a call to all teachers who have the fortitude to get their hands on any portion of the tests, make a couple scans or copies, and forward them to someone who can publicize the scans. This can be easily done!
LikeLike
For the sake of discussion – let’s follow this through. My claim: a public release of the tests will do more harm than good. (As always, I have to preface this with my staunch belief in public education and authentic assessments. I have nothing to gain or lose by testing. I don’t work for Pearson and am not a Pearson-apologist. They’ve dropped a lot of balls and should be held accountable. However, I am committed to supporting the profession of education and one of the ways we do that is by strengthening the science that supports the art of teaching.)
Supporting evidence #1 – Regents have been public for years. CTB was public for years for years and the field used the data to inform curriculum decisions. As a result, students were seeing lots of examples of inauthentic text and contrived reading. (I taught 5th grade. My students knew fables like the back of their hands, having spent week studying them in 4th grade because they were on the 4th grade test.)
Counterclaim: Are there instances from the previous era of non-security when the public challenged questions – and was right? (And the Pineapple doesn’t count. King pulled the passage because of security violations, not because it was inherently a bad item. Only item analysis data tells us what makes a good or bad item.)
Supporting evidence #2, #3, and #4: SED cant’ get much more transparent. They posted the item review materials here: http://www.engageny.org/resource/common-core-assessment-design. The Pearson/NYSED contract is now public. Technical reports are released here: http://www.p12.nysed.gov/assessment/reports/. NYS teachers reviewed the items before they went before students.
Request for discourse: Diane, please help me understand what the community will see in items that psychometrians and NYS teachers didn’t see? What will releasing items (that are being kept secure to save taxpayers money) reveal? How will it advance the conversation in any meaningful way?
LikeLike
Jennifer, how can Pearson be held accountable if no one can review their tests?
LikeLike
Your question implies accountability can be found in the tests. NYS teachers reviewed them prior to the operationalized tests. What will the public see to hold them accountable that the teachers didn’t?
What does that accountability look like? King has already warned Pearson about mistakes they made last year. What will public accountability do?
LikeLike
Jennifer, when I served on the NAGB board for seven years, I found many questions that had passed through review boards of experts that were WRONG. The questions were ambiguous, the answers were wrong.
What do you have against releasing the tests so the public can see if there are more Pineapples in there?
LikeLike
Diane – Because I’m not sure the public is necessarily the best judge when it comes to assessing the quality of a multiple choice item. I recognize the implications of that statement and I know it reeks of elitism (as one of the commentators) said. I cannot speak to your experiences. In mine, however, I’ve sat with teachers as they designed their own tests and done item analysis. A teacher wrote what she thought was a “hard” question and 95% of the students got the question right. Or she gave the students three hard choices and the right answer and students overwhelming ignored her distractors. The reality is – we cannot see with the eyes of an 8, 12 or 17-year-old. Test designers do everything in their power to eliminate measurement error during the design process but we both know it’s not until after the test is given and the reliability tests are done that one can really tell if an item is good or bad. Yes, DIF, p-values – all of that, are statistics, and part of the social sciences and are not infallible. But it is a field with peer-reviewed studies, transparent formulas, and a large amount of research, and critical thinking. If a parent says “This is a bad item, there’s more than one right answer.” Are they always right? Or is it possible they didn’t read the prompt correctly? (and the data would reveal the students read it fine and the majority got the question correct.)
Regarding the Pineapple. It was an odd question, no doubt. But I’m not sure it was necessarily a bad question, Knowing that trusting Pearson about this is a bit like trusting the fox in the henhouse, Pearson shared the item response data from other states (http://ideas.time.com/2012/05/04/pineapplegate-exclusive-memo-detailing-the-hare-and-the-pineapple-passage/2/) and the p-values and point bi-serials suggest the item wasn’t that hard for students. It was pulled because of security issues, not because it was a terrible, horrible, no good item.
All of this – everything I said – belies the fact that I don’t agree with the way standardized testing is being used. I do not advocate for multiple choice tests in schools and I think we should be moving towards portfolio and authentic assessment. Yet, teaching is a profession. NYS has documenting that teachers were engaged in multiple parts of the design process (Contract Mandate, they called it). Those professional educators looked at those items and gave them a green light.
The profession of education is not made on the back of standardized tests. These tests do not suggest an iota of what quality teaching and public education are. Yet, each time the public (or parents, most non-parent taxpayers don’t seem to care) demands to evaluate an aspect of the profession, I wonder about the long-term implications. I’m not suggesting the public should just trust designers, rather curious why now? Why these tests? Why not in the 10 years previously did no one speak of up and say “here’s a bad item?” Is it because of Pearson? Is it because of the policy? Are parents taking a critical lens to all tests their child brings home? I struggle with the absolutes in these conversations and the desire for many to label participants in these conversations as heroes or villains.
LikeLike
Jennifer, if standardized tests were so necessary for good schools, why do private schools seldom use them? Why do they trust their teachers to write their own tests?
Diane
LikeLike
Do AP and IB exams count as standardized tests?
LikeLike
First, I don’t believe they are necessary. Second, private schools have other ways to report to their customers about the quality of their program. Third, I’ve done assessment audits with private schools. Their assessment issues in many cases, are similar to what public schools wrestle with.
Right now, we’re using standardized tests to report to the public the impact of their tax dollars on public education. Alas, we’re using the easiest type of data and doing bad bad things based on the resulting data. As a publicly funded profession, it’s never going to an easy answer. We know public education is doing a herculean task but we also know not all students are walking across the stage, prepared to be the adult they want to be. Again, tests aren’t the answer. Not saying they are. But asking the pubic to vet just this one thing ignores all the rest of the assessment picture.
LikeLike
Jennifer, if you can point me to standardized tests that are not subject to random error, human error, and statistical error, I will trust them. Until then, I will remain skeptical.
Diane
LikeLike
Absolutely and you will never find one. However, it is interesting to note that all of your concerns are framed with in the language of psychometrics. Your insight into test design is not shared by Jane and John Public. Skepticism is healthy and warranted. And no one measure should ever be a determining factor. it’s written right into the Code of Fair Testing: Avoid using a single test score as the sole determinant of decisions about test takers. Interpret test scores in conjunction with other information
about individuals.
LikeLike
Ay, there is the dilemma. John and Jane Public think that tests are scientific instruments, that they are as accurate as yardsticks and thermometers. They are not.
They should be demystified, and the responsibility for that lies above all with the psychometricians who have elevated their own status by surrounding these tests with impenetrable jargon.
I thank Daniel Koretz for doing that, and also Todd Farley for his book “Making the Grades,” which shows how utterly arbitrary is the scoring of the constructed response answers to tests.
LikeLike
Another serious problem (that may not be immediately apparent) with K-12 standardized test secrecy is the inability of scholars and researchers to study the contents of the assessments. We must “trust” the test makers to report on the validity, reliability, and quality of their own products. What could go wrong? In the long term, this affects the field of education in general since the only ones who can provide research data about the effectiveness of these assessments will be the creators and purveyors – who also profit from the tests!
Making tests public does not narrow curriculum, making tests high-stakes narrows curriculum!
Test security (allowing only “insiders” access necessary to determine the legitimacy of assessments) will reinforce pseudo-scientific claims that cannot be verified.
It’s like letting pharmaceutical companies control medical research…or Monsanto study genetically modified food…oh – wait!!
-julie gorlewski
LikeLike
“My claim: a public release of the tests will do more harm than good.” Horse manure. Just as Nixon believed that the release of the Watergate tapes would do more harm than good your statement is based on nothing.
“. . . and one of the ways we do that is by strengthening the science. . . ” What you claim is “science”, i.e., standards and standardized testing, is not science at all. It’s psychometrics which is about as scientific as “scientific racism” and eugenics.
“. . . please help me understand what the community will see in items that psychometrians and NYS teachers didn’t see? What will releasing items (that are being kept secure to save taxpayers money) reveal? How will it advance the conversation in any meaningful way?
There is no “meaningful” conversation when one side has the information and no one else does.
Standards and standardized testing is so fraught with errors that the whole process is invalid as proven by Wilson (see above post for link). Only through vast obfuscation can the testing companies continue this travesty and educational malpractice.
LikeLike
“kept secure to save taxpayer money”
That may be what they say…
Perhaps kept secure to keep the tax payer from knowing they are being ripped off might be a more accurate statement.
Poor quality questions.
Questions “taken” from other sources (released AP exams, old text books, etc.)
Questions that are trivial.
It goes on and on.
How do I know?
There a lot of ESOL students at my school. There are teachers that are able to read the tests aloud to some of these students.
Some of those that have done so are now retired.
They spilled the beans.
In graphic detail.
LikeLike
As per page 18 of the administrator’s testing manual, parents can view their child’s responses to the open ended questions on the ELA exam. This opportunity will be gone once the tests are mailed back to the state. Parents should and must take advantage of this opportunity now.
LikeLike
Julie – I agree re: what narrows the curriculum. We cannot seperate testing and policy at this point in time and to do so is almost a losing battle.
Pearson needs to release the technical reports ASAP. Those should be public and transparent.
LikeLike
I find this ironic. My state (Missouri) and at least Washington state will be giving volunteer pilot testing for kids this summer. From DESE, my state educational agency:
“Due to the overwhelming demand for access to the Volunteer Pilot assessment from principals, teachers, parents, and the public in Missouri and across the Smarter Balanced Assessment Consortium, the Smarter Balanced Assessment Consortium has decided to release a more complete and more broadly available “practice test” in lieu of the Volunteer Pilot assessment.”
http://www.missourieducationwatchdog.com/2013/04/you-can-opt-your-child-out-of-pilot.html
I suppose I should write DESE and ask them since there is “overwhelming demand” to access the assessment, that I should be able to see the questions?
LikeLike
The statement by DESE sounds like pure educrat bovine excrement to me. Yeh, everyone is clamoring for these tests. And I have some great ocean front property down at Lake of the Ozarks in central MO to sell cheaply also! Hurry and put your money down now before it’s all gone and you have to get less desirable ocean front property over at Truman Lake.
LikeLike
Teachers,
Under no circumstances shoudl you photograph and post, or read and post test questions. This is likely a violation of state law and your professional contract. Do not give them an easy way to prosecute you and risk your certificate, reputation and career. Work through legal channels instead.
Please, please don’t give the reform movement any cause to call our professionalism into question.
LikeLike
Sorry, Matt. Our professionalism has been called into question for the past 13 years. Break a law for social justice? Been happening for a long time. The finesse part is not getting caught.
LikeLike
Hell, I’d take pics for a 12 pack of Keystone Light, a pack of Swisher Sweets and a bag of chips.
LikeLike
Segregated public facilities were once the law as well, and it was only the defiance of brave citizens, often told they were acting intemperately, that overturned them.
So-called education reformers have demonstrated time and again that following the law is only for the little people, disregarding it at will. Movements are not formed by the law, but the law can be changed and formed by movements. That’s what opposition to these tests must become, a social and political movement in defense of children’s right to a broad, humanistic education, of teachers and of public education itself.
I did not not make my suggestion lightly, and as a teacher I understand full well the dangers involved. However, we are talking about the literal abuse of millions of schoolchildren, and the destruction of public education, not in some distant future, but within a few short years, if this runaway train is not derailed.
I stand by my suggestion: teachers, let’s hygienically (for personal protection) upload those tests and let the world see how poorly written and invalid they are.
We might be surprised at the reaction. After all, my experience on the streets of NYC taught me that when you scratch a bully, you find a coward.
LikeLike
“Please, please don’t give the reform movement any cause to call our professionalism into question.”
Thanks for the laugh, I needed that!
LikeLike
Sounds like the test makers test doesn’t hold up to the test so they are getting testy.
LikeLike
It’s funny how the accountability people don’t like when their accountability is called into question. Students are just as good at taking pics…it is amazing the fear Fearson has put into people over something that is paid for by tax payers, given to children with no parental input or oversight, and will be used to share data with lord knows who. Stop being afraid and do something. If we wouldn’t have been afriad, oh say, ten years ago, we wouldn’t be in this mess now.
LikeLike
“It is odd that the state is so quick to defend Pearson’s right to privacy and yet so fast to release confidential student information to Bill Gates and Rupert Murdoch.”
What a fantasitc observation.
Reformer duplicity abounds.
LikeLike
Remember Steinbeck’s The Pearl: the priest, doctor, and pearl-sellers preferred that the common people remain illiterate so that only those in power could interpret texts. Educators, parents, and citizens must join forces and demand transparency and accountability. Corporations exist for profit. Schools exist for the public good.
LikeLike
Remember the assignment to debate Nazi’s & Jews? It was discussed on twitter a week or so ago.
Can we trust the exam won’t include sensitive questions or text? PPRA doesn’t apply here — or would it? It would depend on the questions & test narratives.
PPRA
http://www2.ed.gov/policy/gen/guid/fpco/ppra/index.html
LikeLike
Reblogged this on Transparent Christina.
LikeLike
Big Data = Big Brother
That’s the agenda of the educational-industrial complex to seize public funds to privatize and control all aspects of the workforce. From cradle to grave it seems. 1984 and Brave New World seem almost tame by comparison.
I hope we can challenge them in court and in the court of public opinion. The data doesn’t belong to them.
LikeLike
This is very common. I’m not aware of any test publisher or test where content of tests are made public. I do agree that tests should be subject to third party examination in terms of psychometrics, but I don’t think that translates into “parents can look at all of the test content.”
Even if there were a precedent for this, how would a parent be able to determine psychometric validity and reliability of a test? How is it important for parents to read test questions?
LikeLike
Is eded is in the “education business”?
LikeLike
Hi Linda. Hope you’ve had a good weekend. Any specific thoughts on my comments?
LikeLike
Great week – spring vacation, nothing super special, but I got caught up on grading.
You are a master of deflection. Telling us what grades and subject areas you taught would not give away your identity, which leaves me to believe you never taught a k-12 class yourself nor are you certified to teach. You still haven’t answered from over a week ago.
And you didn’t answer if ed ed is in the business of edededucation. Hmmm?
LikeLike
Bingo!
LikeLike
I hope spring break is relaxing. Linda, could you give me an example of how my number of years in various positions or origin of my username would change the meaning of my comments?
LikeLike
Obviously you are not nor have you ever been a classroom teacher. Good day. I think we got it figured out.
LikeLike
Linda, it sounds like you haven’t found any reason why your questions are relevant. If you can think of a reason why my username is relevant my comments, I’ll consider your question.
What if I asked you what your home phone number was? Would that be relevant to any comments you’ve made?
LikeLike
My home phone number is equivalent to “have you ever taught”?
I hope you’re not a test designer. You’re lacking some basic logic and critical thinking skills.
You appear a bit paranoid over a simple question.
Have you ever been a teacher? Yes or No.
That is the same as someone’s phone number? Phew…what a leap. Just forget it…I’ve lost interest and the answer is obvious.
LikeLike
Linda, I see you still haven’t indicated a reason why the number of years I’ve taught is relevant. Perhaps you can indicate why your home phone # and my resume are different?
LikeLike
You didn’t even state you are or were a teacher never mind the number of years teaching.
Maybe you are the reincarnation if Reinvent Ed, Al Meyers who had some ridiculous cyber city software eduflop and though he was all reformy. I don’t really care anymore.
Volunteer in a school. See and hear kids. Be real. Don’t spout until you walk the walk.
LikeLike
Linda, I’ve already told you I’ve held multiple positions in public schools, including as a teacher. If you want more information, let me know how it is relevant to my comments above. So, once again – could you indicate how the number of years I’ve taught would change your impression of my comments above?
LikeLike
Linda, I thought I’d offer you a bit more explanation in the hopes of settling this discussion. If I were to say 5+5=10, it wouldn’t matter how many years I’ve taught, right? Either way, I’m right. Being a teacher for 20 years wouldn’t make 5+5=10 any more true than if I had taught for 2 months.
On the other hand, if I said, “teaching gets easier the longer you do it,” the length of my teaching experience would be relevant. It wouldn’t make sense for me to make the comment if I had only taught for 2 months, or hadn’t taught at all. You would be right to question my experience regarding that statement.
My issue is that I can’t see how the questions you’re asking me are related to the comments I’m making. I’m not refusing to answer because I’m hiding that information, but because it seems that you are trying to discredit my comments based on irrelevant information. What if I were the owner of a company? A politician? How would my comments about Pearson disclosing the tests be any more right or wrong because I was a teacher as opposed to a politician?
So, here’s what I’m proposing: If, the course of discussion, you read something I write and feel that I’m not qualified to make such as statement, feel free to ask. I will consider what you’re asking, and if I can see the relevance and feel it’s not putting my privacy at risk, I’ll consider answering it.
I would say, though, that who really knows how many years any of teacher, or if we’re even teachers? If you’re participating on an internet forum, you can’t really expect to verify the identify of most folks participating. There is always the possibility that you and I are not who we say we are, and I’m not sure there’s anything you could do to change that. For example, if I said I had taught 10 years, would you believe me? What if I said 20? How would you prove I’m right? Because you can’t, why is it even relevant? All of this leads to the bottom line of my position in this particular discussion we’ve been having: Just consider the quality of my comments on their own merit. Rather than trying to attack or support me because of something you can’t even verify, just stick with addressing the content of the comments themselves.
LikeLike
Linda – remember why you shouldn’t roll in the dirt with a pig. The pig likes it.
LikeLike
Dienne, I hope you aren’t in a position where you are influencing children if you talk like that in your offline life.
LikeLike
“. . . to determine psychometric validity and reliability of a test?”
No need to as Wilson has already proven the psychometric (such a pure science, eh) invalidity and unreliability of standards and standardized testing more than once.
edededucation,
My challenge to you is to refute Wilson’s assertions that the whole process is so rife with error that it is completely invalid. (See above post for link). Email me at: dswacker@centurytel.net with any rebuttal.
Thanks,
Duane
LikeLike
Duane, I’ll respond here to keep the discussion public. I saw a link above but I didn’t see any particular evidence – just assertions that standardized testing in invalid and unreliable, which is a claim that can’t scientifically be made against all tests. Validity and reliability are determined for specific tests, and specific applications of tests. Which test in particular are you questioning, and what evidence are you citing?
LikeLike
Edededucation @ 3:42,
Which assertions of Wilson lack evidence for the claims he has made?
Read Wilson’s work mentioned above. For a shorter version on the invalidity as shown again by Wilson in “A Little Less than Valid: An Essay Review” found at: http://www.edrev.info/essays/v10n5.pdf In it he uses the testing bible’s own words to show the inherent contradictions in the bible itself to show the invalidities.
Without reading and understanding what Wilson (who by the way was in charge of developing standardized tests for the Australian state of New South Wales for a number of yeas and like Diane with her turn around changed his tune as to the effects of standardized testing regimes) has proven it is impossible for you to state that “is a claim that can’t scientifically be made against all tests.” Please explain how it can’t be “scientifically made”. The supposed vaunted “scientific” way cannot explain all. Wilson shows through a logical analysis (and logical analysis is really the basis of scientific thought processes) how the claims of the testing community is bunk through their own contradictory claims. The testing community can’t eat its cake and have it too which is what it attempts to do in its validity and reliability claims. And, yes, Wilson does prove that all these claims hold true for all tests including teacher made tests.
Sorry but your response is a non sequitor and begs my question/request for a rebuttal to Wilson. I have searched for over a decade for a rebuttal, have asked many a testing proponent to provide one or links to one and have come up empty handed. That is why I asked you send it to me personally so that I could have the time to read and analyze the rebuttal. Be that as it may your lack of rebuttal to Wilson leaves a lot to be desired.
Again, I look forward to a cogent rebuttal of Wilson’s claims, point by point and then the discussion can move forward.
Thanks,
Duane
LikeLike
Thanks for the link Duane. Perhaps I didn’t look close enough, but I didn’t see it before and didn’t really see any specific arguments against assessments as a whole. My sense is that I don’t really have a grasp on what Wilson is arguing, so I will read that and respond. For the sake of discussion on this forum, though, I think it might be helpful for other folks reading to make specific claims about why all tests are bad, rather than just making the general claim then referring to an outside paper. May lead to more interesting and accessible discussion.
So, I will go ahead and read the link, then send an email. If you think it might be helpful for others to follow along, feel free to respond with a specific rationale for why all tests are inherently invalid. My first response to that claim is that, while no test could likely ever be 100% valid and reliable, many tests have demonstrated sufficiently high level of reliability and validity in a number of contexts.
LikeLike
“I’m not aware of any test publisher or test where content of tests are made public”
AP exams (Mult. choice) are released regularly.
The free response used to be released immediately following the testing period.
Also, teachers grade the free response, so we are able to see the questions and rubrics for grading that summer.
So you don’t know a lot about school, do you?
“How is it important for parents to read test questions?”
It is their child.
These test can determine graduation. They have a right to know exactly why their child is being held back.
LikeLike
Ang, if you’re a professional educator I’d suggest you keep your comments professional, rather than asking questions such as, “So you don’t know a lot about school, do you?” In response, I’d say that education is a broad field and I don’t know everything about everything. Most of my experience is with elementary school, so I wasn’t aware of the AP results being shared. Thanks for that update.
In terms of being held back, there are multiple decisions that are made by school officials (often in collaboration with parents) when parents don’t have full access to all materials. For example, in special education parents do not have legal rights to view any of the test materials aside from test protocols. Parents may certainly ask for information regarding reliability and validity of tests being used, but do not have the right to view the tests themselves. This is standard practice.
Personally, I don’t necessarily think it would be wrong to share those tests with parents as the tests would likely be re-written the next year. I don’t see any harm in releasing them. However, I’m not sure there is a precedent for all tests to be released. You mention AP tests, but many other tests aren’t released. I’m also not sure how helpful it would be for parents. Could you give me an example of something a parent would do with a test once viewed that would be critical to a child’s education – something that wouldn’t be possible by having reliability/validity estimates by a third party?
LikeLike
There seem to be a number of advantages to being held back for high performing students. This has long been known for performance in sports where folks talk about red-shirting their children. There is mounting evidence that students who are old for their grade are more likely to be admitted to highly competitive universities, for example.
LikeLike
Edededuction @ 3:48,
You stated “Most of my experience is with elementary school. . . ” Please do not take this personally but what exactly is that experience?
I’ve taught public high school Spanish for over 19 years now, have a MAEd in education administration, was certified to be an administrator here in the Show Me State but chose to stay in the class room because I knew I couldn’t institute NCLB and other education malpractices. I had my doctoral funding pulled half way through my program, although I continued with the seminar meetings because my advisor was ok with it. My dissertation research dealt with standardized testing.
Experience and expertise can and does lend credence to what one says/writes. When one is not up front with his/her experience, expertise, or qualifications it tends to give the appearance of a hidden agenda. Others have asked this type of information of you. You don’t have to give it, I understand as perhaps your position dictates that you don’t. But then again that seems to me to be a lackey’s position.
Again, thanks for your responses,
Duane
LikeLike
Duane, thanks for your response. I have two reactions/thoughts for you. First, I’m currently in a position where I’m not sure my viewpoints and openness in expressing them would be completely welcome or helpful. My experiences are also pretty specific and unique (I have a fairly specific combination of positions I’ve held that would make it fairly easy for folks to figure out who I am). I’m not in public office, and certainly wouldn’t be known on a national level, but if anyone who knows me were to read my professional background, it would be a pretty quick indication of who I am. I’m proud of my positions and am not afraid to assert them, but my first goal is to be effective in working with the folks I work with – not establishing credibility on this blog – so I’d rather protect that.
Second, I hear you – background establishes credibility – but I’d refer you to my comments about relevance of background. Not sure if you saw my post in the matter, but I basically said my teaching experience is irrelevant if I’m trying to convince you that 2+2=4, because my years of teaching wouldn’t make that statement any more or less true. Contrast that with a statement such as, “Teaching is easier the longer you do it.” With that statement, by years of experience in teaching would be relevant.
So, my response has been: If you feel that you need some background information to consider the validity of any of my claims, ask and I’ll consider whether I can safely provide it. In this particular discussion, though, can you identify how any of my claims require me revealing exactly how many years I’ve taught, as opposed to me saying that I have? Another poster questioned whether I was part of an educational start-up – would that change the validity of my comments above? If so, how?
Basically, Duane, I’m open to being questioned about my experiences if it seems I’d have to have had them to make a particular statement. However, in this case, I’m getting the sense that folks are questioning back years teaching simply because they’ve run out of content-oriented material to address with me, and are hoping to discredit me through some other means. This seems to be what you indicated in your other post in response to me.
Whatever your response, I appreciate that you’ve been civil and professional in your comments, and look forward to continuing discussions.
LikeLike
You made a statement:
“I’m not aware of any test publisher or test where content of tests are made public.”
The apparent point of your statement seemed to be..No “high stakes” tests are released, so Pearson need not release.
You were informed that the NY regents exams are released and AP exams are released.
There is nothing unprofessional about calling out your lack of knowledge.
Perhaps when you don’t know you should ask.
You do seem to be a master of deflection, obsfugition and dissembling.
Consultant?
Start up?
Like Linda, I have lost interest.
LikeLike
Ang, there IS something wrong with calling out a lack of knowledge if you do so unprofessionally. I couldn’t imagine the tone of your comments being taken professionally during a staff meeting at school, at a workshop, or in the classroom. I’ve found several of your comments snarky, rude, aggressive, and off-topic. Is this the impression you’re wishing to give off, or do you really not see how others may take the tone of your language here?
Put another way, imagine if a casual reader stumbled across this post and read your comments. What conclusion do you think the average person would come to about the professionalism of teachers after reading your comments, and the comments of several others? If you are concerned with teachers being respected and treated professionally, is it not important for you to carry yourself respectfully and professionally in public places such as this forum?
I’m sure you mean well – I really do. I’m sure you think you are defending the honor of teaching, and that I am some outside force trying with all my might to slight the teaching profession. However, I’d really encourage you to consider the distinct possibility that your writing style on this blog appears to sound more like a bickering middle school student than an educated professional hoping to have meaningful conversation that moves folks closer to understanding just how great many teachers are. Instead, your tone, and the tone of others, contributes to the problem that I have routinely, which is that I (and others here) are seen as bitter, unprofessional individuals unable to focus on the true needs of our educational system.
A final comment, this time to Diane: I’m wondering if you find the comments made by folks like Ang to be helpful to your cause? Do you think these comments encourage folks to take your blog more or less seriously? If you are able to see some of these comments as unprofessional, I’m wondering if you’ve ever considered calling out anyone, even if they’re in the process of supporting you?
LikeLike
Edededucation @ 5:10,
I find the argument of “not being ‘professional'” to be quite specious and is in actuality an attempt to control the discourse when one doesn’t have the right to “control the discourse”.
Let’s all be polite and nice and go along to get along. Well that is one of the major background problems in these educational malpractice wars. One side, the deformers, play down and dirty and the other, those who personally experience the educational malpractices foisted upon the students on a daily basis, play nice. Sorry, but it’s been a long time gone (apologies to CSN, eh Joanna B) since true educators should have taken off the kid gloves and fought back against those who choose their own avaricious advantages over the innocent students.
LikeLike
Duane, thanks for your response – I completely agree, and find personal attacks to be a last resort when content-oriented discussion doesn’t seem to be working for someone. Looking forward to (hopefully) having content-relevant discussion in the future with all involved here.
LikeLike
Eded, there is definitely value in seeing tests. I’m not sure what world you live in or why you think a student should not be able to see model questions or past questions. Explain why there is a test industry devoted to the SAT and ACT? What about the LSAT and MCAT? The USMLE? Why is it acceptable for college level students to see test questions, but it is not acceptable for elementary or middle school parents to see them? Learning from mistakes is an important method of learning. And it’s not cheating. Pearson, in the name of intellectual property and profits, is cheating children and parents.
LikeLike
Sheery, I think you’re missing the point of why folks are calling for the release of tets. You are saying that the purpose is for “learning from mistakes,” but I’m not aware of any educator who really wants state test results released for instructional purposes. Rather, I’m interpreting the call for release as being a means of evaluating the test. So, I’d return to my previous comments – why is it important for an elementary student to view an assessment if you’re trying to determine the validity of that assessment? Allowing a third-party to conduct an independent validity study seems like a much better way. Would you disagree?
In terms of your comparisons with SAT/ACT, I’m not sure I see the point in releasing sample test questions for those assessments either. If the goal is assessing previous learning – not specific preparation for a specific test – it would make more sense to keep assessment content private, as making content public allows people to specifically prepare for the assessment, which – in a way – sort of invalidates it.
I’d also say that your example of SAT/ACT test prep material is an example of how making test material public is actually MORE aligned with profiting from the testing industry, as parents and students are sold classes and samples of test material to make money. In that case, making test content public is actually MORE business oriented rather than less.
LikeLike
Just saw this on HuffPostEd, 4-22-13; crossposted. WARNING: No context given but by the question and the handwriting of the answerer appear to be for a lower elementary grade.
“Problem Solving Real World
There are no horses in the stables. There are 3 stables in all. How many horses are in each stable?”
On the line provided for the answer, a young inhabitant of Planet Reality has written “really ????”
Link: http://www.huffingtonpost.com/2013/04/22/kids-calls-out-dumb-test-question_n_3132047.html?utm_hp_ref=@education123
reddit link: http://www.reddit.com/r/funny/comments/1cs3uf/raised_a_smart_one/
Of course, whoever produced, printed and distributed this test question employed the best psychometricians that $tudent $ucce$$ can buy.
“Laughter is poison to the pompous.”
🙂
**DISCLAIMER: the above does not apply to ethical numbers/stats folks like Mercedes Schneider [aka KrazyMathLady], Bruce Baker, J V Heilig, Bruce Baker, G F Brandenburg, Gary Rubinstein, Daniel Koretz, and [fortunately] too many others to mention.**
LikeLike
New York State Regents examinations were publicly available immediately after administration.
LikeLike
Thanks learningfirst – I wasn’t aware of that.
LikeLike
And they were WRITTEN by teachers. gasp.
LikeLike
I seriously doubt that the Pearson tests were written by teachers.
LikeLike
The 2005-2010 NYS ELA and math exams are available at http://www.nysedregents.org/.
LikeLike
NYC public school parents have a compelling case for seeing the scoring – especially after Pearson admitted and apologized for mis-scoring over 4,500 student Gifted & Talented assesments:
http://www.pearsoned.com/pearson-statement-regarding-scoring-errors-on-nyc-gifted-and-talented-assessments/#.UXWAq7-hDzI
It is time for our elected officials and mayoral candidates to get involved in this issue!
Parent, PS 58 Brooklyn (District 15)
LikeLike
As usual Krazy TA and Diane Ravich hit the nail on the head. The way to obtain the information is a public information request if you have it in your state.
LikeLike
Leakers please come forward!
Certainly there are people out there who have access to these awful Common Core tests. And those people know that these tests are unethical, inaccurate and designed only to profit the creators, managers and administrators.
And these people realize that these tests are all about political control of our schools. They’re a way to “prove” that our public schools and teachers are “bad”; that’s the reason they exist. There are numerous honest, decent people who have been taken in by these tests. But the people who will benefit by them are hostile to public education and to our educators.
So, we need a Daniel Ellsberg to come forward—privately or publicly—and provide this test to the taxpayers and parents who paid for it. We’re the ONLY ones that the Privatizers are afraid of. And we need to make our voices heard.
Come forward and LEAK this test, please. We’re waiting for you.
Do the right thing by we parents, teachers and students. Leak these tests!
LikeLike
Teachers and parents unite, subvert, be strong. There are many, many ways to get the truth out and spread the word. Keep at it. We have the numbers and we have the proof.
Never give up.
LikeLike
We wouldn’t want to put anyone’s career at risk, but even if test questions are not “leaked”, it’s not difficult to reconstruct them from students’ memories.
We set up a forum on Reddit for exactly that purpose.
http://www.reddit.com/r/CommonCoreMath/
LikeLike
I remain intrigued by a theme I keep seeing in the comments. Many of you haate the tests. Haaaate them. Your dislike and and well-earned distrust of what the tests represent comes through loud and clear. So given that, what good would come from releasing the items? SED releases the test maps so we know what the items focus on. Districts see success rates on items so they can see which concepts students struggled with. What would an angry public see in the items that psychometricians and NYS teachers didn’t see?
On a related note, what do we gain as a profession by disparaging a science that focuses on as accurately (as possible) capturing evidence of that profession? (BTW, if you’re interested in learning more about psychometrics, I recommend this organization – http://ncme.org/about/
LikeLike
Jennifer Borgioli,
You must admit that the testing has gotten out of control. Or maybe you would not. You need not hate testing to acknowledge that tests are being misused. Tests are supposed to be used only for the purpose for which they were designed, but now they are used to classify students, to deny them promotion or graduation, to reward and punish teachers, to fire teachers, and to close schools.
Do you think that is appropriate? Don’t you think the massive misuse of standardized testing has created the backlash?
Testing is NOT a science. The writing of test questions and answers is culturally conditioned. Standardized testing is a social construction. The grading of constructed response answers is subjective.
The only thing objective about the tests is machine scoring of multiple choice questions. Even then, the questions may be ambiguous, and the answers may be wrong.
LikeLike
Diane – I absolutely, without hesitation agree. Testing is out of control. Tests results are being misused. I would LOVE to see an end to high-stakes tests and a return to the original standardized tests, a la the Six Arts from the Sing Dynasty.
The backlash is absolutely warranted. My issue, and my need for perseverance in this thread, is wondering if the lash, as it were, is going where it will a difference. VAM – bad idea. Promotion based on test scores – TERRIBLE idea. Closing schools? Really, really dumb.
Yet, releasing test items won’t solve any of those problems.
Does it make a difference, though? If amazing, engaged members of the public like your readers rally behind the release of items and find concerns with items, then what? What will their concerns be based on? The cultural conditioning of parents to read state assessments will trump the teachers who read it before them?
Yes, testing is out of control. Our obsession (as a society) with numbers is causing us to focus on what’s easy to measure, not what is important to measure. I will not however, back from a defense of the science of our profession, even if it is a tenuous, new science. Some day, we’ll get it right. Yet, if the rally cry now is “show us the tests so we can judge the quality” (better than the test designers who studied the emerging science), I fear we’ll always be ten feet behind when it comes to the science of teaching and learning that the general public will always feel, that because they went to school, they know as much about our profession as those who study it.
LikeLike
“Yet, releasing test items won’t solve any of those problems.”
Really? You don’t think wide publicity of really bad test questions (a la “Pineapplegate”) won’t sour the public on standardized testing and increase pressure to end or at least scale back the tests?
LikeLike
Diane,
“Testing is NOT a science”
THANK YOU!
Like economics, testing seems to be attempting to equate itself with the natural sciences.
We must repeat often and forcefully that standardized testing is not and cannot be a science.
LikeLike
Jennifer,
Read and understand the two above mentioned Wilson writings on the total invalidity of educational standards and standardized testing to understand why some of us haaaate that bovine excrement or is it equine excrement. Some of us are sick and tired of seeing students abused by these educational malpractices.
Pyschometrics is not a science, it is a pseudo-science on the order of scientific racism and eugenics.
Duane
LikeLike
How many times are the testing companies audited by a non-stakes holder? According to the book I mentioned earlier, the execution of the reliability measurements used during the actual scoring of these tests is flawed.
A year ago, I question why people were so anti-testing, but I am slowly jumping on that bandwagon. I would be OK with a single day of multiple choice questions at the beginning and end of the school year to measure progress. I am not OK with random people scoring open-ended questions that create high stakes for my children and their teachers.
LikeLike
“Man is not what he thinks he is, he is what he hides.”
― André Malraux
LikeLike
“… What good would come from releasing the items … What would an angry public see in the items that psychometricians and NYS teachers didn’t see?”
The short answer is that, with these tests having a large impact on many people and institutions, we have a right to see what clothes the Emperor is wearing, or if he is just a buck naked fraud surrounded by a noisy, sycophantic entourage.
As the subjects of these exams, students, teachers and parents have the democratic right to see them. Political and social decisions of tremendous import are based on them, which trumps whatever preferences the creators and managers of tests may have.
Oh, and please spare us the professional disdain for the vulgar mob and its insufficient awe before the infallibility of psychometricians.
No matter how carefully crafted individual items and tests may be, they are now used aggressively as instruments of public policy, and it is not parents and teachers who made that choice. Their use as high stakes vehicles of social policy is a political decision, as is deciding to fight them. Your work necessarily takes a back seat to that, whether your choose to acknowledge it or not.
Professional standards and pride notwithstanding, psychic meter readers may wish to divorce their work from the political context in which it is produced, but the rest of us are hurtin’ out here, and we can’t, and won’t.
LikeLike
Now that was a good one.
LikeLike
Wow,
Super cool response of the day!
Well played, Michael.
LikeLike
I have always admired Michael. I save his responses.
They give me hope and Ang, your reply was so professional and mature :).
Three cheers to the both of you! Hip Hip Hooray!
LikeLike
Linda, Ang, and Michael, you are all amazing. As “a casual reader,” I find all of you, to be outstanding advocates for today’s children. I always look forward to your wisdom and professional understanding of teaching and children. Some others….I skim and move on.
LikeLike
Michael Fiorillo: simple powerful truths spoken in plain English.
IMHO, the criticisms of your stance boil down to “keep quiet” and “if you do the right thing you’ll be punished” and “be obedient to those who are your natural and/or social superiors.”
Banesh Hoffman (THE TYRANNY OF TESTING, 1962) got buried in silence under a mountain of dismissive contempt. Silence is compliance—and I love your refusal to be submissively quiet!
Keep on making sense out loud; I’ll keep on listening.
🙂
LikeLike
If defendants have the right to confront their accuser, and teachers can get fired for the test results, then teachers have the right to see the test and thoroughly scrutinize it.
LikeLike
Bravo! I love your succinct responses TC…I think I love you!
LikeLike
If David the bus driver starts with 2 riders, and picks up three at the next stop, and drops off four at the next stop… how did David get a license to drive a bus?
LikeLike
For a humorous vision of the DIBELS test check out the following post from blog.blurtso.com : http://blog.blurtso.com/2013/04/21/ditto-goes-to-school-xx.aspx
LikeLike
It’s amazing to me that anybody would say that parents, the ones who fund the schools, do not have a right to see what material is being presented to their children. No parent has to justify to Pearson or anybody else why they want to see the tests.
Why the secrecy? Maybe it’s because they’re giving children personality tests.
http://www.utahnsagainstcommoncore.com/common-core-a-mental-health-professional-parents-perspective/
Educational Testing
According to the U.S. Department of Education, CCSS will authorize the use of testing instruments that will measure the “attributes, dispositions, social skills, attitude’s and intra personal resources” of public school students under CCSS (USDOE Feb, 2013 Report). In a nutshell, CCSS simply states that it will develop highly effective assessments that measures….well….almost ”everything.”
LikeLike
Susan,
Brilliant exposure. It goes past even personality tests but into biometrics ( too scary to post. Most would find it inconceivable! )
LikeLike
Yes, many people I’ve discussed this with seem incredulous, especially fellow Democrats. (sigh) Here is the California longitudinal database called CALPADS.
http://www.cde.ca.gov/ds/sp/cl/
LikeLike
This article was shared in another thread and presents a pretty solid view of the big picture of large-scale assessments: http://www.minnpost.com/learning-curve/2012/06/student-testing-pioneer-angermeyr-skeptical-about-high-stakes-trends
My favorite quote: The tests we have, whether they’re state tests or commercially available tests, are by and large designed by modern psychometric theories, and they’re pretty sound.
LikeLike
Jennifer B.,
The people who design the tests say they are sound.
The cigarette manufacturers insist that smoking is healthy.
What else is new?
Psychometrics is NOT science. It is social science. Read Daniel
Koretz’ “Measuring Up.”
The best test experts warn about random error, statistical error, human error, etc.
The questions are written by humans. The answers are written by humans. The scoring of written responses is by fallible humans.
All are making judgments.
This is NOT science.
Tests are useful when used appropriately and with full knowledge of their flaws.
LikeLike
I agree – and am saddened you think I would disagree to any of those points. The quote was provided slightly tongue in cheek. I’ve read Measuring Up and recommend it to others. An interesting text from a different perspective is “Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies” by Wainer.
The next paragraph in that article I shared above is: “Where the distortion comes in is that you can only test a limited amount of the domain. Even if it’s a domain like mathematics, you can’t cover everything. And so you make assumptions about kids’ skills in that broader domain. Do we have eighth graders who are good readers based on a pretty small sample of questions and items?
Testing professionals know that you’re just sampling the domain and you don’t try to make inferences further than that. But nonprofessionals do that all the time. “American students are 51st in the world in reading.” There are a lot of assumptions that are made before you can get to that conclusion, but people leap right over that.”
LikeLike
Ha, ha! No Pinepplegate but–wait for it!–Advertisingate! All about the product placement in the tests as…reported by the middle school students taking them!
Come on, now, Pear$on–just how much $$$ is the Lego brand & Mug Root Beer paying you for including advertisements in…TESTS?! (Thought we wouldn’t find out now, did you?Our kids are smarter than you think– who’s practicing the “soft bias of lowered expectations” now? (If I have that Bush saying wrong, please correct me!)
LikeLike
I don’t work in this area so I’m not certain, but parents might want to try getting access to the tests under the Freedom of Information Act. Since it sounds like the Common Core stuff is Federal, the government would have to have good cause and prove *why* it’s good cause, if I understand it right. (Again, I do wills & estates so this is on the same level as if we met at a baseball game & it was mentioned.)
LikeLike
Needed: whistleblower teachers willing to handcopy test pages and leak them anonymously.
Public education is a civil rights issue!!
LikeLike