Laura H. Chapman gives more examples of the distortion and corruption of education practice amd policy by econometric language.
Students are performing on grade level if their scores on a standardized test are at or above the median on a percentile scale (1-99). On a large-scale test, a score at or near the 50th percentile (the median) will usually classify a student as proficient in the skills and subject matter on the test.
Expected growth means that gain-scores of students (on tests in a single subject, such as math or art) are staying in about the same location in a distribution from year to year—below average, average, or above average. For a large number of students, the distribution is likely to resemble a bell or normal curve.
Predicted growth is an inference about a student’s future gain-score, derived from a linear regression analysis of two or more years of that student’s gain-scores. This analysis assumes that past performance will predict future performance. Perhaps, but in education, this is a dismal assumption. It can become a self-fulfilling prophecy. The assumption is so risky that almost every corporate report begins with this caveat: Past performance does not predict future performance.
A student is said to have achieved a year’s worth of growth if his or her gain-score on a test of proficiency is equal to, or greater than, the gain-score made by a 50th percentile student. The same measure is applied to teachers. Teachers in some districts are rated highly effective only if all or most of their students have gain-scores of more than a year’s worth of growth.
References to a year’s worth of growth are fundamentally misleading because the common mental picture of a calendar year is different from a school year (typically 180 days); an instructional year (typically 172 days); and a typical accountability year (130 days from pre-test to post-test).
Academic peers are students whose test scores in a given year are the same or nearly the same. This concept permits comparisons of their gain-scores from the prior year to the current year. Students who make greater gains than their academic peers have an accelerated growth trajectory. Students who fall behind their academic peers need remedial work to keep up. The average of the gain-scores for academic peers in a teacher’s classes is typically used as a measure of the teacher’s productivity and effectiveness. This use requires a studied indifference to other influences on test scores.
A growth trajectory needs a target. Targets for learning need to be set using baseline data so the instruction offered to each student, during a known interval of time, is efficient and has a measurable impact on student learning. Meeting targets for learning is analogous to meeting a sales target or a production quota by a date certain.
Teachers and others who say they are “impacting the growth of their students” are not think-ing about the meaning of words. They are parroting econometric jargon.
Experts associated with Metametrics hope to set growth velocity standards. They describe their theoretical mapping of “aspirational trajectories toward graduation targets” in reading skills as analogous to “modifying the height, velocity, or acceleration respec-tively of a projectile launched in the physical world.” They seek greater precision in setting targets and cut scores for grade-to-grade progress in meeting the CCSS. (Williamson, G. L., Fitzgerald, J., & Stenner, A. J. (2013). The Common Core State Standards’ quanti-tative text complexity trajectory figuring out how much complexity is enough. Educational Researcher, 42(2), 59-69.).
Calibration refers to the quest for precision and consistency in measurement in the context of just-in-time delivery of a result, especially manufacturing.. In education, the term means that evaluators and other monitors have followed specifications in rating performances, presentations, processes, and products. Calibration events are training sessions intended to standardize how raters use or interpret language and to verify that rules for making judgments have been followed with fidelity. Such events are also called trainings or calibrations.
Audits are conducted to verify that calibrations are not needed, that rules have been followed, that data are free of ambiguity, and that low-inference definitions of performances and metrics are used consistently. Questions about the validity of the metrics may be ignored.
Bring to scale means that an educational policy, practice, or product is believed to merit replication in multiple locations, as in manufacturing and franchise systems for a mass market.
Aren’t percentile ranking routinely used in education? Several of my children have taken diagnostic exams for learning abilities and disabilities and were given IEPs based, in part, on percentile scores. Folks who post here routinely criticize the CCSS as being developmentally inappropriate for certain grades. Surely this does not mean it is inappropriate for every student in a given grade in the United States, but rather some high percent of the students.
“Surely this does not mean it is inappropriate for every student in a given grade in the United States, but rather some high percent of the students.”
It’s not the edumetric speak that renders the “it” (assuming you mean that “CCSS as being developmentally inappropriate”) as “inappropriate” but the fact that the whole basis of the edumetric is epistemologically and ontologically FALSE and therefore COMPLETELY INVALID.
Again, read Wilson’s work to understand why.
The there that is there in Wilson’s work may not be visible on first glance by economists as they are used to dealing with numbers and far flung “theories” of homo neo liberus economicus and not substantial words and discussions of the true nature of the “human” and its interactions with and within the realm in which it inhabits. Right, Tarun or is it Gautham???
“…read Wilson’s work…”
Relativistic drivel.
Have you read it, Tort? If so, how is it “relativistic drivel”? What about what Wilson has proven is “relativistic”?
It’s easy to call names but a lot harder to prove that the name calling has any substance.
Have I read Wilson?
No. I have read his disciple, Swacker.
You are extremely rude and offensive. Hardly one I would associate with being compassionate toward people.
As for grades: They are a necessary evil.
As for testing: American students, rich and poor, are given the opportunity to attend school. Nobody can provide equal outcomes.
Now, I asked a simple question and I am awaiting an answer to it. Can you answer my specific question about common core without suggesting (someone) should resign from teaching?
I’m waiting…
Or, you can simply apologize for being rude and offensive when people think differently than you do.
Okay Tort, I took the bait but your hook has not met flesh!
First of all my first response wasn’t directed at you but is one of a general statement about the inanities of the whole educational standards and standardized testing regime. Is your skin that thin? (and yes that one is a rhetorical one directed at you)
Second, your “question” wasn’t a question but a series of questions, I quote: “With CCSS in CA., the kids test in 8th and 11th grade. How is anybody going to measure me if I teach kids in 9th or 10 grade? I don’t see any linkage. In fact, didn’t Desay try to get the state to test kids in all grades so that lever would still exist?
Who can clarify this for me?”
I’m not sure what kind of clarification you are looking for. To what is “that lever” referring? Can you clarify your question so we here might have a chance to answer it?
Your statement that “grades are a necessary evil” shows quite shallow thinking and major GAGAness*. Your willingness to use invalid educational malpractices such as grading which causes many harms to students shows a lack of intellectual capacity.
Until then, to me the clarification is that anyone who believes that the teaching and learning process is amenable to “measuring” hasn’t spent as much time examining the epistemological and ontological underpinnings of “measurement” as they should, especially if they teach. Your statement that “grades are a necessary evil” shows quite shallow thinking and major GAGAness*. Your willingness to use invalid educational malpractices such as grading which causes many harms to students shows a lack of intellectual capacity.
Now since you admit to not having read Wilson are you willing to retract your statement that it is “relativistic drivel”?
As far as being rude and offensive, thanks for the compliment but you haven’t actually seen me “rude and offensive” so how the hell would you know!?!? Hell, if I were going to be rude I would just tell you to FOAD or ESAD! But I won’t as that wouldn’t be okay in this blog.
*Going Along to Get Along (GAGA): Nefarious practice of most educators who implement the edudeformers agenda even though the educators know that those educational malpractices will cause harm to the students and defile the teaching and learning process. The members of the GAGA gang are destined to be greeted by the Karmic Gods of Retribution upon their passing from this realm.
Karmic Gods of Retribution: Those ethereal beings specifically evolved to construct the 21st level in Dante’s Hell. The 21st level signifies the combination of the 4th (greed), 8th (fraud) and 9th (treachery) levels into one mega level reserved especially for the edudeformers and those, who, knowing the negative consequences of the edudeformers agenda, willing implemented it so as to go along to get along. The Karmic Gods of Retribution also personally escort these poor souls, upon their physical death, to the 21st level unless they enlighten themselves, a la one D. Ravitch, to the evil and harm they have caused so many innocent children, and repent and fight against their former fellow deformers. There the edudeformers and GAGAers will lie down on a floor of smashed and broken ipads and ebooks curled in a fetal position alternately sucking their thumbs to the bones while listening to two words-Educational Excellence-repeated without pause for eternity.
May have to respond to you, Tort, in the fashion that I now do with TE (or is that Tarun or Gautham?)-very limited.
Mr. S-wacko,
Try this: just don’t respond to me at all. You clearly haven’t kept up on your meds.
Tort,
I wouldn’t respond to your posts except I have this disorder over which I have no control (no meds can help with this one) and that is the lack of control to not respond to posts that have inanities and spout idiologies like “grades . . . are a necessary evil.”
But I will give credit to where it is due, Tort.
You have managed to come up with a new bastardization of my last name that I haven’t yet heard of (or at least remember hearing as my last name lends itself very well to churlish childish word games) in my 59 years on this planet. Felicitaciones.
To paraphrase KTA (sorry to bring you into this one KTA) “Keep flinging (sh)it and I may keep responding” (or I may not).
PS. Thanks for reminding me to take my meds!
The mania of testing based on the Common Core Standards to measure student growth while ranking and yanking teachers based on those scores and then closing schools to turn over to corporation owned Charters reminds of General Westmoreland’s White House supported VAM-like measures for winning the Vietnam War that was based on numbers too—enemy body counts.
The theory went like this: the more people U.S. troops killed, the happier Westmoreland and LBJ were and eventually Nixon—with his B-52 bombing of Cambodia, Laos and North Vietnam where the U.S. dropped more bombs than we dropped during all of World War II in both major theaters, The more U.S. troops killed the closer we were to winning the war.
About a quarter to a half-million Vietnamese civilians died, and a half million children have been born with birth defects since the war ended thanks to the use of Agent Orange that our troops were also exposed to. And that isn’t counting the half million to 1.1 million deaths of North Vietnamese and Viet Cong troops.
In the end, the U.S. lost the Vietnam War. I wonder how much suffering will be caused by NCLB, Race to the Top and the VAM driven Common Core agenda that is similar to the carpet bombing of Southeast Asia by Nixon, before the billionaires—for instance, Bill Gates—and Washington D.C. realize they lost this war too because using VAM like bombs wasn’t the way to improve public education to make it better than it already is.
I have a question about testing and how it is linked to teachers. Prior to CCSS, in Calif each teacher’s students took a grade level test. One could, therefore, measure (or try to measure) each teacher based upon those tests.
With CCSS in CA., the kids test in 8th and 11th grade. How is anybody going to measure me if I teach kids in 9th or 10 grade? I don’t see any linkage. In fact, didn’t Desay try to get the state to test kids in all grades so that lever would still exist?
Who can clarify this for me?
If I may “clarify this” for you, Tort.
Any teacher who doesn’t immediately recoil from and refute the concept of “measuring” the teaching and learning process IS NOT FIT, INTELLECTUALLY SPEAKING, TO BE A TEACHER AND SHOULD RESIGN FROM THE PROFESSION AS SOON AS POSSIBLE so that no more harm will be caused to the most innocent of society, the children who have no intellectual defenses built up to defend themselves.
Does that help???
Your reply tells me that I will not get an answer to my question. I have a question for you, however:
When you go to the doctor, when you drive on a bridge, when you visit an attorney after the police have wrongfully harmed you, do you want a qualified professional or do you want somebody who isn’t fit for practice?
In addition, no damage comes to student when they test. If the student bubbles the shape of a Christmas Tree on his answer sheet, that will only negatively impact the teacher or the school.
Students aren’t losing any sleep over taking tests that have no consequences on them.
It seems, therefore, that the CCSS testing (that I personally see no importance to) don’t link to any particular teacher. That is a far cry from the previous situation where kids from grade three through eleven all tested, and the results thereof were linked to teachers.
I get the fact, Mr. Swacker, that you don’t like testing. My question deserve a factual answer. I think that if I resigned, my students and former students would not be happy. How do I know? Feedback.
Tort, turn that around. Do standardized tests like the Boards or Bar guarantee good doctors or competent lawyers?
They don’t guarantee good lawyers or doctors (nothing does), but they help ferret out bad ones. Put differently, passing the bar doesn’t mean someone will be a good lawyer. But failing the bar (you get multiple chances) is a pretty good indicator that someone will not be a good lawyer.
In the end it’s about supply and demand. Are there enough prospective lawyers, doctors, and teachers that we can afford to cull the herd through entrance exams and other hurdles? And if not, is it worth risking shortages of these professionals to ensure that they all meet a baseline level of competence as defined by the exam?
Math Vale,
Thanks for replying respectfully.
I do not favor testing that measures teachers unless we have students sitting in the chairs that are all at grade level when they enter our/my door. This is not happening, and so testing for teacher competence makes no sense.
Your assertion is correct, but so is mine. I don’t care if the doctor, lawyer, engineer, carpenter, electrician, or cop is educated in public, private, or home schools–I just want them to be competent. How does eliminating grades, or testing for graduate school admission or professional licensure, help rectify the situation you presented?
By the way, if you are teaching high school mathematics, do you think it’s a little unfair to the kids that they’re sitting in your Trig class but they never learned the difference between 4 divided by 2 and 2 divided by 4?
My original question is still on the table.
Many people here are screaming about teachers being measured by CCSS. I ask how that will be done for the 9th and 10th grade math teacher if the kids only test in 11th grade. Do the kids put the names of those teachers on the test? Is that info pre-slugged? Are questions on the exam linked to the 9th and 10th grade teacher? What if the kids transferred from another school? Are his previous teachers linked to those 9th and 10th grade math classes? I doubt this; but who knows. Lots of talk today about “big brother.”
If nobody can answer this, then what is all the hub-bub about? Personally, my students don’t care about the test. They don’t go home scarred if or when they find out their scores. Give me a level playing field, and then measure me.
Oh, wait. They gave me an uneven playing field but people in my district said my kids did great. I still don’t like it. If you’re going to measure me, then dammit, give me kids that are at grade level when they walk into my room. I am tempted to believe that the CCSS is looking for data, but I don’t think it is data about teaching, but more about what’s been learned.
I simply wanted to know if I am right or wrong about this.
FLERP, failing the Bar may filter out some bad lawyers, but no question bad lawyers pass the Bar. Also you would probably agree the Bar represents only a small part of the realities of practicing law. Plus lawyers who possess those skills for success they do not teach in law school, fail the bar for whatever reason. So, the conclusion is, how useful is spending those grueling hours taking an exam that does not really serve a purpose? It is nearly impossible for a lawyer to permanently lose their license in Ohio for misconduct, let alone incompetence. Yet we are now destroying and blacklisting very good teachers based on flawed measures and systems. BTW, since the ABA did not allow African-Americans to join till 1943, were they correct in their standardized policy in that case? Of course not. Mistakes were corrected. So true with this obsession over testing as absolutes. (And yes, I have many friends who are lawyers. They are very good at what they do).
“FLERP, failing the Bar may filter out some bad lawyers, but no question bad lawyers pass the Bar.”
That’s correct. You can filter out some, but you can’t filter out all. That the Bar exam doesn’t filter out all bad lawyers isn’t necessarily a good reason to throw out Bar exams, any more than we should throw out teacher certification requirements because they don’t filter out all bad teachers (or med boards because they don’t filter out all bad doctors, etc.)
“It is nearly impossible for a lawyer to permanently lose their license in Ohio for misconduct, let alone incompetence.”
I have to say, this sounds a lot like the “it’s nearly impossible to fire a bad teacher” argument.
Mr. Swacker,
I have a B.A. in Political Science, and yet I have shown for decades that I can teach mathematics and have done so for many years. Students and parents, counselors and administrators, approve of my abilities in both content and in people skills with the kids.
But, do you want to know who threaten me?
Other math teachers. They want me out. My degree is not in mathematics. Your point might be well taken, but the damn rules say one must be highly qualified. To hell with highly effective. To speak as you do: bull shit. I am effective, sir! And, thus, I am not in good stead with those who don’t know how to adjust for the kids’ sake. I wonder, do you still scorn me?
But, the admin wants me to continue. So, who is trying to personally harm me?
My union colleagues. And they’re untouchable.
It seems the bullshit cuts both ways.
Tort. I’ll try. VAM assumes a certain initial level of measure, then takes a followup or series of measures to produce a difference The student, and teacher, effectiveness is based on mythical students considered equivalents. The exact formulas and methods are proprietary and secret, so you may never have a good answer.
I agree that prerequisite skills are important. Also the impact of other teachers and factors. I remain skeptical VAM can control for all the confounding factors. If I bring these up, the answer is usually “just because”. I want evidence and proof.
For example, what about HS students who possess 3-4th grade ELA skills but the VAM tests only begin measuring at 10th grade? You could work successfully to bring them up 5 grade levels and the VAM assessments fail to measure that – you as a teacher are still ineffective.
Teaching should use assessments for diagnostic puposes. But we’ve gone off the rail making VAM assessments the end game.
Yes, I do think it is unfair and the population I teach is often several grade levels below. I’ll go further, it breaks my heart to see them struggling and defeated. But destroying me and my career as being done in Ohio through VAM testing and publicly posting my ranking as a form of blacklisting does nothing to help my students.
Tort. I also come from an alternate background in mathematics (engineering) and have no doubt you have a good background in mathematics. But I spent a severance in the Great Recession to return and get licensed. I basically took the rest of my mathematics classes that a math B.S. would earn. No question, those abstract, analysis based courses helped me understand math at a deeper level.
Yes, I met jerks in a union but also not in the union. Mostly teachers who felt threatened or struggling themselves. In fact, I found the veteran union teachers very supportive and very concerned about improving my teaching, even though I was not a traditional teacher.
Thank you, Math Vale.
I don’t think my district uses VAM, and thus, I know little about it. We are not yet evaluated with test scores as one of the components. Still, I wonder, if that were to change, then how could the men and women downtown use a CCSS test score to evaluate me? I really don’t see how the results of my former students would be used to make a determination about what I did with them IN THE PAST unless they linked specific test questions to certain other courses of study beside the current course an eleventh grader might be enrolled in.
Evidently, there must be some cause for concern because people here are writing and commenting about CCSS testing.
I just don’t see it.
I hope you can rest at night. I don’t sleep long during the night knowing that I can’t get through to all of my students (because of the greater systemic problems beyond my control). You, being an engineer, are a great fit for the math classroom, and you can bring many rich experiences for your students.
You must also realize that engineers should be highly competent in their calculations. Public safety is in their hands. Even the smart men and women at NASA have failed, but for the most part, they are men and women who knew math and physics, and could prove it to others when they applied for work.
Tort. I’d say hang in there. I have resigned myself to the fact my kids enter my classroom many grade levels deficient. Since all us teachers, including the ones teaching prior grades, are operating under the same conditions, I see the problem as systemic. For the last 30 years, the classroom has been undermined and disrupted by outside interests more concerned about profits and elections than the well being of students. Get those influences out of the classroom and let teachers teach.
In Ohio, as in other reformy states, the obsession over metrics means my career teaching is limited. I work with the most challenged students. Most are on their last leg ready to drop out. Because of that, the state VAM system rates me at the lowest, if I can even get the students to take the VAM tests. I am simply a margin of error. If that is what Ohio wants, so be it. I’ve started over before and will do it again.
MathVale,
Do you think that students were not many grade levels deficient before the early 1980s?
Math Vale,
Now, I have compassion for you. You hang in there too. If you are low on the pay scale, they won’t use your students’ inability against you. If you are high on the pay scale, and, you tell bureaucrats the truth about why the system is failing, you will be a marked man/woman.
Tort. Hey, stuff happens. I look at Ohio as a state in severe decline, in spite of the slick rhetoric out of Columbus. Kasich and the GOP had a chance to rebuild, instead they destroyed. They remain fixated on trickle down economics that has never worked and isn’t working now. We rank 47th in job growth. Alot of people are insecure and struggling. Too many.
I am not the only teacher seeing a very dark time for students and education in Ohio. But we ride it out, hoping the public begins to realize the truth.
Tort.
I hope this helps answer your question. Between 2005 and 2011, the Gates Foundation spent at least $465 million to set up three interrelated entities: the Teacher-Student Data Link Project (TSDL), the Data Quality Campaign (DQC), and Center for Educational Leadership and Technology (CELT). Of this money, $75 million was to pump up “advocacy” for data-driven decisions.
The Data Quality Campaign, now operates with 82 “partners” (clearly a misuse of the term partner) including the Council of Chief State School Officers and Achieve owners of the Common Core State Standards. This campaign is working on details for standardized data gathering. The big categories are: A Statewide student identifier; Student-level enrollment data; Student-level test data; Information on untested students; Statewide teacher identifier with a teacher-student match; Student-level course completion (transcript) data; Student-level SAT, ACT, and Advanced Placement exam data; Student-level graduation and dropout data; Ability to match student-level P–12 and higher education data; and State data audit system.
The related Teacher-Student Data Link Project (TSDL) describes eight purposes in linking teacher and student data:
“1. Determine which teachers help students become college-ready and successful, 2. Determine characteristics of effective educators, 3. Identify programs that prepare highly qualified and effective teachers, 4. Assess the value of non-traditional teacher preparation programs, 5. Evaluate professional development programs, 6. Determine variables that help or hinder student learning, 7. Plan effective assistance for teachers early in their career, and 8. Inform policy makers of best value practices, including compensation.”
The TSDL project wants teachers to identify learning activities by the performance measures for a particular standard, by subject and grade level. The TSDL system goes further. It is intent of a period-by-period tracking of teachers and students every day; including “tests, quizzes, projects, homework, classroom participation, or other forms of day-to-day assessments and progress measures”—a level of monitoring said to be comparable to business practices. I call that surveillance.
The TSDL system will keep current and longitudinal data on the performance of teachers and individual students, as well schools, districts, states, and educators ranging from principals to higher education faculty. This data will then be used to determine the “best value” investments to make in education and to monitor improvements in outcomes, taking into account as many demographic factors as possible, including health records for preschoolers. http://www.tsdl.org/
This Gates-funded campaign works in tandem with USDE’s $700 million Statewide Longitudinal Data Systems (SLDS) Grant Program. Since 2006, more than forty states have received multi-year grants to standardize data. The program is “designed to aid state education agencies in developing and implementing longitudinal data systems. These systems are intended to enhance the ability of States to efficiently and accurately manage, analyze, and use education data, including individual student records…to help States, districts, schools, and teachers make data-driven decisions to improve student learning, as well as facilitate research to increase student achievement and close achievement gaps.” See http://nces.ed.gov/programs/slds/
Ms. Chapman,
I won’t waste your time with ‘hot air’ by commenting on what you’ve provided. However, you’ve given me a starting point to begin studying what lies ahead. Thank you for taking the time to do this!
You ask: “How is anybody going to measure me if I teach kids in 9th or 10 grade?” The short answer is “they can’t unless standardized exit examinations are developed to ensure that each CCSS outcome is mastered by each student”.
“Can this be done cheaply and easily?” you might ask, The answer is obviously “NO”. Consequently we are not measuring MASTERY of content in “Core” areas we are measuring COVERAGE… and we have no means of measuring ANYTHING in “non-Core” areas….
BUT… because we (ahem) have to measure every teacher with presumed precision we devise workarounds with spreadsheets. So an elementary Special Ed teacher’s VAM measure might be based on the aggregate VAM scores of teachers at his or her students’ grade level and an elementary PE teacher might use the reading scores of a particular teacher to determine their VAM score.
I’m sure some imaginative statistician in CA will devise a mathematically precise way to measure your effectiveness with 9th and 10th grade students because it’s a lot less expensive than to do that than to devise tests for content at each and every grade level… and a LOT less expensive than funding CA schools at an equitable level.
xxxxxxxxxxxxxxxxxxxxx Responses to you seem indirect. I would answer that, as a technicality, annual test scores can be tied directly to the teachers whose classes are tested. Scores on tests given at a 3-yr interval cannot, except by some suspect algorithm that would of necessity include factors outside the realm of that teacher’s classes. On the other hand, there are many documented instances around the country demonstrating that grading a teacher’s performance on his students annual test scores is a nonsensical process. Teachers teaching the identical curriculum in the identical manner have been graded highly efficient one year and inefficient the next based on annual test scores. It is the tying of teacher performance to student test scores itself which has no merit, as was pointed out in an April 2014 paper by the American Statistical Association.
I agree wholly with your observation that the process also lacks merit because the student test-takers have no direct stake in the game.
S and F,
These “high stakes” exams are often criticized for putting large amounts of stress on students. That would appear to be inconsistent with your concern ” that the process also lacks merit because the student test-takers have no direct stake in the game “. Do you disagree with the claim that these exams create stress for the students?
Those high achievers, TE, the ones who care about their grades and who strive to get into good universities, are the ones who are truly stressed out.
The underachievers could care less. Whether it’s because they can’t do the work or they are unwilling to try (there’s that old grit component) is irrelevant since the results tend to be the same. These are the ones who have no “skin in the game”.
Then there are those who have challenges which prevent them from reaching mastery, either due to language barriers, learning disabilities, emotional problems, health issues, etc. They, too, will feel stress just with the idea of taking a test, regardless of the outcome.
And all the above statements are based upon a fair and equitable assessment. The stakes are raised when the questions are of dubious validity – Either more stress or less motivation.
Add in the APPR and you spell TROUBLE – with a capital T – for both the students and the teachers (and even some parents).
Ellen,
No doubt high school students stress about everything connected to college, be it standardized exams or teacher assigned grades.
I know I did. I was an overachiever and worked very hard for my As, even through college. It got me into Phi Beta Kappa.
Now I’m grateful that I never have to take another test or write another paper for the rest of my days. I can finally relax.
Ahhhhh!
Ellen, your comment is intriguing. I can see the highly motivated student stressing out about ACT, SAT, or AP tests, but why would they stress out over high stakes state testing? The kids are 9, 10, or 11th grade and the results, as far as I know, have no impact on their college aspirations.
Please fill me in. Thank you.
Tort –
One quick comment – my oldest daughter drew a design on her bubble sheet for the first grade assessment in math. She ended up as the only child in remediation in second grade. PS: Now in her thirties, she makes more money than any teacher who ever taught her.
I have proctored exams at all grade levels. I have seen first and second graders burst into tears when they didn’t know the answer to a question on an assessment. I’ve also seen a temper tantrum when a child was told to put down their pencil before they were finished with a section. I’ve seen an eighth grade boy cry at the end of a math test which was filled with algebra – a unit not yet covered in the curriculum. I’ve seen children put their heads down in frustration in the middle of an exam. I’ve collected half completed test booklets or blank scan sheets from children who have given up.
Most students want to do well. They want to please the adults in your life – both parents and teachers. Nobody wants a notice sent to their homes with a two, or, horrors, a one, on it. Children don’t understand the political implications of the assessment. They don’t know what will happen if they “fail”. They are afraid and if they don’t understand the questions, it is psychologically devastating. And I even have questions about the implications inherent for those who don’t seem to care.
Now as for counting. If they don’t do well the students are put into a remediation program. This may mean that they can’t sing in the chorus, play in the band, or eat lunch in the cafeteria so they can get “extra help”. Or they might have to come in before or after school to meet with the teacher.
In the Buffalo Public Schools, a child has to apply to high school. One of the criteria for acceptance at a top notch site is their score on the 7th grade assessment. (I personally am glad that my future wasn’t based on anything I did in seventh grade). Other assessment scores from previous grades might also be considered. Even if a student wants to attend a private school – those scores are used to determine eligibility.
Finally let us consider high school. In NYS a student must pass five Regents Exams in order to graduate. Currently the NYS Education Department has created a new Common Core Regents in Algebra and English which must be passed prior to graduation. However, this is being phased in, so last year many students had the option to take both the regular Regents exam and the Common Core version. The state agreed to take the higher score of the two. Most kids took both. And I’ll throw this into the mix – advanced eighth graders who were taking Algebra also could (and most did) take the eighth grade math assessment.
Honestly, I couldn’t make this stuff up.
The state is currently creating more Common Core Regents Exams in other subject areas.
So in NYS, at least, the CC does mean a whole lot.
I’m sure other states have their own horror stories. I believe some states expect children to pass the assessment in order to advance to the next grade. Talk about high stakes. (This is illegal in NYS).
So, does this answer your question, Tort? There must be something behind our dismay to have created this much uproar. I’m retired and I’m still horrified. My experiences were with NCLB. I can only imagine what’s happening with the CC assessments. (I’ll just mention that last year one of the third grade exam booklets was missing numerous pages – talk about stress.)
Ellen, Yes, your post is very informative. I did not think of little children stressing out, only because I teach high school kids who don’t worry about this sort of thing; and, the post-NCLB tests are used primarily as a “gotcha” to assess teacher effectiveness (it seems).
Still, I would agree with you that kids are wonderful–they want to please while doing their best, even if their score has no meaning to them personally.
I don’t doubt for one minute that being successful in the real world sometimes happens in spite of what happens in the classroom. Thanks for the informative reply.
It is not just Westmoreland that looms large in your retrospective look at the misuse of data. Robert Strange McNamara was John F. Kennedy’s Secretary of Defense in that era. McNamara (Harvard graduate and Harvard professor at age 24) believed almost every problem could be solved by statistical analysis.
During WW II he had used statistical analyses to increase the efficiency of aircraft in hitting their bombing targets. He brought related skills to his work as President of Ford Motor Company in the post-war years, when systems theory was becoming a major tool for management.
As Secretary of Defense he relied on statistical analysis and relatively new systems for most of his recommendations on how to manage the Vietnam war. He relied on the “body counts” on the progress of that war. Easy data, like test scores, also misleading.
During his tenure as Secretary of Defense, McNamara also introduced John von Neumann’s mathematical theory of games as a tool for addressing options for cold-war strategic decisions—how to avoid mutually assured destruction, dubbed MAD. I doubt if Arne Duncan is smart enough to use this as a conscious strategy. Bill Gates is, but prefers to blast away at public education counting on not being deterred
Tort –
The answer is simple. In NYS a certain percentage of your evaluation is based on test scores (the percentage depends upon the “contract” created between the union and the administration, then approved by the state – it’s called APPR).
Now, if you don’t have a CC test or Regent’s Exam in your subject area, your test score is based on the overall test scores of your school. Thus, you may be evaluated on students you have never taught.
That’s what the fuss is about. Or at least some of it.
We can see how far removed the process is from reality. People who depend solely on these metrics want easy, turn-the-knob solutions. Sit in some far removed office, pore over speadsheets, change a few cells, and voilà! You have instant educational excellence!
I read “When Genius Failed” back in my past life studying business. It traced the rise and fall of LTCM. The verdict was this financial firm began to focus solely on metrics and ignored market realities. From stats, the “tails” destroyed them. You would think LTCM would be a canary in the mine. But then 2008 rolls in with systemic dependence solely on algorithms and metrics. Financal executives were making decisions and evaluating performance based on systems they did not understand. Talk to veteran, seasoned traders and they will tell you how critical thought, intuition from years of experience, and knowing the customer has been replaced with people cranking out numbers and graphs in front of a console of terminals.
Even without a PhD in statistics, I see obvious problems that cannot be controlled but show up in reality (i.e. practice). Too many to list here. And I need to move on the a few football games.
Ahhh, said while smoking a cigarette and sipping on some fine cognac. Doing “Edumetrics” is soooo satisfying that I just want to drift off into some dreamy fulfilled state of being knowing that all is well in the world.
Here is a recent question from an analogy section of one of the CCSS tests:
Edumetrics is to the teaching and learning process as ______________ is to the sexual reproduction process.
So much talk and writing, so little of substance in the edumetric realm. To understand why all this “edumetric talk” is so much mental masturbation (not meant to demean Laura as she is quite an astute writer just trying to turn the edudeformers’ language back onto itself) read and understand what Noel Wilson has shown of the educational malpractices that are educational standards and standardized testing in his never refuted nor rebutted “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
I much appreciate this second blog posting of the day on the uses and misuses of language.
While on several other occasions I have followed Señor Swacker’s serving of Noel Wilson with a topping of Banesh Hoffman [THE TYRANNY OF TESTING], I am going to add a dash of W. Edwards Deming today.
Words have connotations and some have customary usages particular to certain fields that may be at variance with general usage. For example, the following terms as used by psychometricians can—and often do—mislead the general public: bias, validity, statistical significance, achievement, and performance.
Think, for example, of what happens—including on this blog—when an entire discussion about one topic, learning gaps, is expressed as another, test score gaps. The two are very far from being synonymous.
A critical element common to the two blogs featuring Laura H. Chapman is defining terms. In this case, terms that supposedly enable us to measure, assess and judge with precision and accuracy.
IMHO, this is not a ‘technical’ matter of a dictionary-style definition but depends on such supposedly hazy matters as perspective, goals, aspirations and ideals. In other words, it is critical what one thinks “count” and what one thinks “doesn’t count.” And by how much!
From THE ESSENTIAL DEMING (2013, pp. 63-64)—
[start quote]
There’s no true value of any characteristic that’s defined in terms of measurement or observation. There’s no true value of the number of people in this room. There’s a number that you get by carrying out a procedure. Change the procedure, you get a new number. The question is whom do you count? Do you count people on the stage? Do you count people working on the sound and visual equipment? Sure they’re important. But do you count them? Do you count people who are not attending the seminar but are simply working here? Do you count somebody out there drinking coffee? Or somebody on the telephone? Do you count him or don’t you? There’s no true value for the number of people in this room? There’s a number that you get by carrying out a procedure. Change the procedure, you get a new number. Any two people have different ideas about what is important to include in the count.
[end quote]
It can be messy and lengthy and complicated but there is no substitute for flushing out the thinking behind the procedures being used to come up with numerical values, e.g., test scores and their use and misuse in VAM.
And that’s exactly the discussion the charterites/privatizers don’t want to have. They fare poorly when the thinking behind their numbers & stats is revealed. Revelation leads to “revile-ation.”
Just my dos centavitos worth…
😎
Duane, you are really off the wall today.
Any sane person will agree that most students advance about one year each year. If they advance two years, that is exceptional growth. If they only advance 6 months there are issues which need to be addressed.
Over the years, when I got my children’s report cards (and I had four kids), I did not care about the grades (at least not until high school when they counted for college admissions). What I looked for was whether they were above, below, or at grade level. Everything else was irrelevant.
Since the Common Core assessments do not seem to accurately measure whether or not the child has achieved a year’s growth and is or is not on grade level, the exam is worthless in my eyes.
In NYS the roving cut score is set to punish, not assist teachers or students in procuring a good education. In addition, the reading passages, vocabulary, and written responses do not match the expected abilities of the given grade. Plus many of the math problems also exceed the normal course of study. That’s why I advised my daughter to opt out my grand daughter from these assessments. Her IQ may be high, but her test scores do not reflect her ability.
And yes, Duane. In my grand daughter’s case, grades are a nuisance, not a help. And, over the years I have seen low grades based on the lack of turning in homework assignments vs the knowledge of the subject area. Ultimately, what do those grade really reflect? (It will be interesting to see how those underperforming students do in college as I follow them on Facebook. Already, the top student with straight As flunked out his Freshman year at UB.)
So in some ways I agree with you (and Wilson).
However, I also agree with Tort. At this time, colleges need some way to measure a student for possible admission. The GPA is a key measure and because of this fact, some students are more motivated to do their best work. I’ll agree that for others the grading system is a bust. I also admit that grades don’t always reflect ability and often only reflect effort. Potential is often a hidden attribute.
Unfortunately, this is the hand we’ve currently been dealt. The grading system is the least of our worries when assessments are rigged to fail the overwhelming majority of students (in NYS) and thus blame the teachers for being faulty educators.
Unfair? Hell yes! Unless you can afford a private school.
So let’s focus on the issues and put Wilson aside for the moment. We now have 1, 2, 3, and 4 not A, B, C, D, or F.
Unless we can find a way to increase a child’s IQ or change the shape of a bell curve, we must continue to fight this inaneness which passes for national and/or state educational policies.
“Any sane person will agree that most students advance about one year each year. If they advance two years, that is exceptional growth. If they only advance 6 months there are issues which need to be addressed. ”
Count me insane then as I find the concept of “growth” as you use it here, especially in regards to “grades” (no matter what type/kind/designation) to be quite lacking.
“At this time, colleges need some way to measure a student for possible admission.”
I have no doubt that colleges may “need some way. . . ” but that need shouldn’t be a guiding factor in the educational practices of K-12 public education. Their need is not our concern. Our concern is the teaching and learning process and how it interacts with each individual’s ability to learn the subject matter.
“Unfair? Hell yes!”
And yes, we should be fighting that “unfairness” which in reality is discrimination against certain people who don’t have the innate capabilities to score high on a “grading scale”. Should the public schools be in the business of discriminating against any portion of the population over factors of their being over which the individual has no control such as skin color, sexual orientation, gender and/or mental capabilities???
I contend that the government should not be doing that discrimination and it is in essence a breach of the individual’s constitutional rights.
Duane,
You use the word “learn” in your statement. How can you know how much another has learned if you refuse to attempt to measure learning. How can you even talk about anyone’s “ability to learn”?
Duane, the growth I was referring to was what is developmentally appropriate for a given age as per child psychologists such as Piaget. For example, most babies walk about age one, start speaking in short sentences by age two, and are toilet trained by age three – give or take a few months either way. Reading ability normally develops between the ages of five and eight. In this country we equate age with a specific grade level which does cause problems at times. For example, the learning disabled must take the assessment based upon their age and not on their ability. The same for a refugee from another country who does not speak English. There is a little more lee way in high school, but not in the earlier years. And this dictum usually comes from the government, not the school systems.
Perhaps this is not how they do it in your state. I know some states get around this issue of age/ability through retention. So if an eight year old has been held back, they don’t have to take the third grade assessment until they reach third grade. So much for using CC as a comparison between states when a child is retained year after year so their test results won’t bring down a school or a district’s ranking. Rumor has it that this is frequently done in states such as Texas and Florida.
Now Duane, we come to the issue of GPAs or the grading policies. I have seen numerous sorts of assessments over the years, including portfolios, presentations, projects, journaling, demonstrations, performances, recitations, etc. Whether it’s a number or a grade or a comment, some sort of reckoning must be made so the student gets the feedback necessary to make improvements for the next assignment.
We are not doing our children any favors by glossing over the results of their endeavors. Whether yay or nay, they need to know if their attempts were successful. In the real world they will be constantly assessed. Annual reviews determine whether an employee get a raise. The higher the score the higher the percentage. And bonuses are determined by their contributions to the organization.
Currently we use letter grades or percentages. If there was another system, the results would be the same. Just as a child knows that the Redbirds are the top reading group and the Robins are the bottom, a student recognizes the difference between the “A” and the “D” student. No letter grade is necessary – the “aura” is there. Call it what you’d like, there are some who strive for mastery and others who are simply biding their time (and each one knows who they are).
On the flip side, there are advantages to learning from ones mistakes and even smart kids at times fail and kids who normally do poorly excel on a certain topic. These experiences can enrich both. It also assists the teacher in developing curriculum and remediation (at least it did when lessons weren’t scripted but could be honed to meet the individual needs of a particular class).
Finally, those who work their butts off need to be rewarded. Those top 10% deserve their spot in the limelight. Why should they be lumped in with the bottom 10% who didn’t even complete the minimum requirements? Then again, who do you want to be your doctor, or lawyer, or physical therapist, or child’s teacher or head of the Department of Education?
I personally question the value of the SAT (and it will have even less value when it becomes a CC SAT), but it is still used as an admission ticket to most colleges. And they also look at the GPA. Other criteria, such as recommendations, extra curricular activities, and personal essays, help colleges narrow down their acceptances. Do you prefer a lottery system? Or should placement be solely based upon affordability? Perhaps location? Where does merit fit in? And if placement is random, are we doing our children any favors by possibly putting them in situations where they are likely to fail?Just because I did well at UB doesn’t mean that Harvard would have been a good fit for my abilities.
Grades are a tool. The GPA is a tool. Should other tools be used – yes! Should we eliminate those you personally don’t approve of (via Wilson)?
Whether you are right or wrong, I admire your fortitude, but Duane, old friend, I fear you are spitting in the wind.
One reason past performance can’t predict future performance is that the nature of English/Language Arts class changes between K-3 and 4-12. In K-3 it’s about teaching decoding; after that it’s about teaching (or trying to teach) comprehension, which is a very different ball of wax. So kids who excel at decoding may fall flat on their faces when it comes to comprehension. VAM would unjustly blame the teacher for the apparent regression.. But suppose the VAM folk got a bit more sophisticated in their models and controlled for this fourth grade slump (that Jeanne Chall and E.D. Hirsch write about). STILL ELA growth metrics would be unfair to ELA teachers because reading comprehension is not at all something that she alone is teaching. The history and science teachers, not to mention parents, are contributing as much, if not more, to reading comprehension ability by teaching the word and world knowledge that you need to comprehend a text. Only if you believe the false doctrine that reading comprehension is a matter of metacognitive operations can you hold the ELA teacher accountable for growth in reading comprehension. Now what the the VAM scientists do? Will they admit that scientism has found its limit and that it must not trespass any further into the hallowed halls of the liberal arts? “Growing” human minds is too complicated; it cannot be treated like agricultural science. Lloyd’s Vietnam analogy seems apt to me: faith in metrics can wreak havoc.
“Loonar Trajectories”
Econometrician, that am I
Inventing jargon on the fly
“Aspirational trajectories”
Newton with a bad disease
Alternate title: “Gaspirrational Trajectories”
SomeDAM Poet: does this appear in that slim volume put out by the Broad Academy called VAMAaniacal VECTORS?
Or am I getting this confused with one of the hidden mantras of THE LORD OF THE BLINGRING?
So many inanities, so little time to catalogue them all…
😏
TAGO!
“VAManiacal Vectors”
VAManiacal vectors
Duncans, Gates’ and Rhees
Inspirrational lectors
Spread rheeform disease
Diane, I feel like the language of the standards are polluting my son’s 5-year-old brain. He has been in kindergarten for 3 weeks and now insists that he can work on “reading to self” instead of “reading to himself/myself’. He is big into semantics and doesn’t believe me when I explain that “reading to self” doesn’t make any sense. He is quoting something exactly from school. I don’t know what to do about it.
Wonder if he will be saying writing to self, playing to self, and so on. Gather these examples and find an occasion to express your concern to the teacher who is likly to be suffering from an overdose of jargon and has missed the larger point that your child is picking up on that…a misunderstanding that could be difficult to modify since you say he does not think you are correct. Also, sometimes these glitches are really temporary. Beware of overreacting, but also alert to other indications of jargon creep. Lucky child to have such an astute and attentive mom.