Peter Smagorinsky, professor at the University of Grorgia, is one of our most astute critics of the current testing mania. This essay appeared in Maureen Downey’s blog in the Atlanta Journal-Constitution.
He writes:
“The Georgia Department of Education has introduced a new assessment vehicle, the “Student Growth Model,” to measure student and school progress. According to the DOE, it produces “[t]he metric that will help educators, parents, and other stakeholders better understand and analyze the progress students make year to year.”
Very enticing. Who wouldn’t want such an instrument to track students’ growth?
Georgia plans to assess teachers based on student growth, but are we clear on what growth really means?
The Student Growth Model relies on two measurements. One is based on the percentage of students who meet or exceed state standards on standardized tests. The second measurement is designed to assess year-to-year progress of each student, compared both to students in other Georgia schools and to students at the national level in “academically similar” schools in terms of demographic and socioeconomic statistics.
These measurements make up a major portion of the state’s new teacher assessment system. The model assumes that there is a one-to-one causal relationship between individual teachers and individual students in terms of their test scores, which serve as a proxy for learning, for growth, and for teacher effectiveness in all areas.
The Atlanta Journal-Constitution, whose coverage of education I respect, has provided very favorable exposure of this initiative, using the language of advancement to describe its (as yet untested) effects in terms of students’ “progress,” “learning,” “achievement,” and “growth.”
Damian Betebenner, the statistician who designed the model that Georgia has adapted, has said, “You may have a teacher that’s in a classroom and the kids aren’t growing. We’re not saying that you’re necessarily a bad teacher, but it’s just not working here.” Yet by factoring in “growth” in these measurements, the system does indeed conclude that teachers whose students do not improve their test scores relative to local and national peers are bad.
I would like to offer some alternative understandings of what human growth involves, and how to measure it. As one who is immersed in developmental psychology, I always ask of claims of growth, Development toward what? And thus by implication, Development by what means?
For a committed Southern Baptist, this growth might involve learning, through faith-based texts and adult guidance, Biblical precepts so as to walk a righteous path according to the church’s teachings. This path is, above all, going somewhere and might be measured by attendance at church, tithing, good works, and other indicators of devotion. Which would you find more valuable measures of growth within this community, a multiple choice test on the Holy Bible, or living a virtuous life led by worship?
Now, I am not a religious person, so this conception of growth would not suit me. I’m an old high school English teacher who now works in teacher education. There is great disagreement among English teachers about what it means to grow through engagement with this discipline and its texts, traditions, and means of expression. To some, growth through English involves learning canonical works of literature and the cultural traditions that they embody.
To others, growth involves becoming a more involved citizen through engagement with the values and beliefs available in literature. Others might see English as a vehicle through which personal reflection and maturation are available; or as a discipline that requires mastery of the conventions of formal English…..”
Growth, progress, achievement, learning: We all want these attributes in our children and expect our teachers to promote them. But the new Student Growth Model measures do not measure up to what most people hope for in their child’s developmental course: their development into good human beings according to some cultural definition of a quality life.
So, what does it mean to conceive of a curriculum and assessment package in terms of human growth? I don’t think it’s the same for everyone, because people are headed in different directions.
Even those headed in the same direction often take different pathways, follow different paces, integrate that pathway with different goals, and otherwise follow Henry David Thoreau’s wisdom: “If a man does not keep pace with his companions, perhaps it is because he hears a different drummer. Let him step to the music which he hears, however measured or far away…..”
Statisticians’ solutions are admirable in their ability to reduce assessments to single numbers, and thus are prized in the policy world. Teachers’ solutions tend to be much knottier, because they work with kids of delightful variety and hope to help each one realize his or her potential in an appropriate way.
If you agree that Georgia’s Student Growth Model does not rely on measures that encapsulate either student growth or teacher effectiveness, and if you agree that making students and teachers accountable for growth is a good idea, what might be a better alternative in terms of developing teacher effectiveness measures? If you believe that test scores constitute valid measures of student growth, toward what end are they growing, and in what manner do these scores demonstrate that growth conclusively?
In prior essays in this forum, I’ve made points I needn’t recapitulate here in detail. I oppose the standardization of diverse people, and believe that teachers should be entrusted to know their disciplines and how to teach them. I think that standardization is conceived especially poorly when it is measured by people who have never taught. I think that factory-style schooling is more likely to set back authentic human growth than to promote it in ways that lead to satisfying and productive lives. I think that single-iteration test scores are unreliable measures of performance. I think that most conceptions of curriculum and assessment provided by today’s policymakers are misguided and harmful to teaching and learning.
Diane Ravitch's blog 
I sort of feel like the “growth” aspect is the buzz word traditionalists figured would allow them to be compliant with VAM, but without really giving in to VAM (so they thought). “Well we don’t mind that they base decisions on test scores as long as they are looking at growth,” because most assumed growth is easy to show. Maybe it is; maybe it isn’t. But I don’t think it entails the wiggle room assumed. In fact, in some buildings it is a problem because if a child already is scoring above grade level, what growth will they show unless they are bumped up?
Also, doesn’t data mislead? Isn’t that why we got into the financial crisis, because the data did not indicate a problem, and so common sense was ignored in favor of data?
LikeLike
Growth is a concept that comes from the business world. This is another way that refomers try to make schools work like a business.
If you are the company that owns M&M’s you want to see growth in your business segment from year to year. So say M&M’s have a 45% marketshare (I am making this up). A growth goal for next year could be 50% then 55% the year after. If you are Starbucks and you have 1000 stores open, your growth goal could be to have 1100 next year, then 1200 the following year, and so on. If you are a toy company and you make a 33% margin your growth goal could be for next year to make 35%, then 38% the year after. The idea that schools can be treated the same and have similar growth goals is just not feasible beucase schools and children are not products and profit margin.
And even if they were, growth goals eventually cannot be attained at a steady continuous rate like this, becuase M&M’s can keep growing their marketshare but they will never have 100%, because there will always be other types of candy. Starbucks store openings will cap out because they will saturate the market. Profit margins plateau becuase you get to a point where you just can’t cheapen the product any more or you just can’t raise the price any higher. Growth in standardized test scores assumes every year you will keep going up and this is the flawed logic behind NCLB that we began with. You cannot get everyone to 100%, it is just not possible. You cannot make everyone exceed the average year after year becuase then it is no longer the average. It is just not logical. Looking at student growth this way is just going about education and learning all the wrong way.
LikeLike
And schools fell for it hook, line and sinker, eh?
Shame on us.
LikeLike
exactly right
LikeLike
“Measuring growth” sounds like something an oncologist should be doing, not a teacher.
LikeLike
Alan,
What do teachers base grades on if it is not a student’s growth in the class? Is it race or gender or wealth? These are just guesses, so I hope that you can help me refine my understanding of how K-12 teachers decide how to pass or fail the students in their classes.
LikeLike
“delightful variety and hope to help each one realize his or her potential in an appropriate way.”
yes.
LikeLike
Growth is an issue that takes a small step in the right direction. However, growth toward what becomes a bigger issue. The only way a test can be an indicator of growth is that it is confirmed by classroom progress or the lack of it. That issue I discuss in my upcoming book at http://www.wholechildreform.com.
And then it is simply an indicator of second class achievement that must be confirmed by first class achievement which is the whole child? These first class assessments in the classroom allow demonstrations of learning which show what a child can do.
In addition for teacher assessment there is a wide variety of skills a professional educator will demonstrate that should be a part of the total assessment.
Other questions would be are low scoring students attending school? Are they in a chaotic environment? Do they have childhood stress? and on and on. With this information, the “whole village” must be involved to effectively teach the child.
There is no simple answer but i thank those who are looking.
LikeLike
Control the language of the debate and victory is assured!
How could anything called a Student Growth measure only measure Math and Reading?
LikeLike
For me the key statement of the article is “the yet-to-be-proven but soon-be-used way of assessing how much teachers enriched each of their students.” Once again a state is jumping on the bandwagon without validating their assessment. Students and teachers are at the mercy of whatever “brain fart” the politicians and their corporate masters deem “the great idea du jour.” They just toss it out to see what hits the fan like VAM. Forget research and due diligence.
LikeLike
I so agree. But there’s more to it. The growth models & their definitions of growth are part of a wave of ed reform which pit students against teachers & teachers against teachers & students against students.
It’s not just another fad. All of the ed reform policies are based on pitting individuals against each other, & define achievement in terms of individual effort, oblivious to the true nature of education, which involves a social interchange which supports achievement of a social benefit to the society.
LikeLike
p.s. this is me, Spanish & French Freelancer, now reviealed through digital serendipity as Virginia Bucci
LikeLike
“When I use a word [eg “growth”], Humpty Dumpty said in rather a scornful tone, ‘it means just what I choose it to mean — neither more nor less.”
This is a case where the people doing the “measuring” have no idea what they are supposedly measuring means (or if it means anything at all).
FAIL! FAIL!! FAIL!!!
These people (like Damian Betebenner, the statistician who designed Georgia’s model) need to go back to school and take “Intro to Problem Solving” because understanding and defining the problem is key. Without fist doing that, what follows is simply garbage.
LikeLike
I don’t see what’s so hard about measuring growth – I do it for my own kids every six months or so. Just have them stand straight against the back of their door and make a line just at the top of their head. Now, I can see why schools wouldn’t want to go making marks on doors or walls, but it seems like a large piece of butcher paper would do.
Oh, wait, that’s not the kind of growth they’re talking about, is it? Although I’m baffled why you’d want to measure any other kind.
LikeLike
Dienne. The statistician William Sanders, known as the father on all of this statistical ranking of teachers (although not the first really) gave us the EVASS and other variants of VAM. He used your example of marking the wall to record the changing height of child explain the statistical version…but got nailed but the late great Gerald Bracey, expert in testing, for the misrepresentation.
Then came along the infamous “oak tree analogy” complete with powerpoints from folks who knew that they were taking advantage of really complex and established ideas about human growth and development in education in order to market the “new” idea of “growth metrics.”
They called the double heritage for the word “growth” fortuitous. I will try to find the reference. It was in train-the-trainer materials. So the process of conflating an ample set of educational meanings of growth and the economic meaning of growth is deliberate.
Consider how “growth is set up as measure within RttT legislation. The first part requires a definition of “achievement.”
“Student achievement means (a) For tested grades and subjects: A student’s score on the State’s assessments under the ESEA; and, as appropriate, other measures of student learning…provided they are rigorous and comparable across classrooms. (b) For non-tested grades and subjects: Alternative measures of student learning and performance such as student scores on pre-tests and end-of-course tests; student performance on English language proficiency assessments; and other measures of student achievement that are rigorous and comparable across classrooms.” (Tests that teachers create for use in their own classrooms are not acceptable).
Then comes the next part.
“Student growth means the change in student achievement for an individual student between two or more points in time.” (Fed. Reg., 2009, p. 59806).
A line graph can easily be constructed to illustrate how a trajectory of learning from a pretest toward a posttest “growth target” is implicitly envisioned as a tidy and continuous upward slope and the slopes are steeper for students who do poorly on the pretest. Growth trajectories of that kind are also an essential feature of rating teachers “effective,” “highly effective,” or “ineffective.”
Then comes the judgment of a teacher.
“Effective teacher means a teacher whose students achieve acceptable rates (e.g., at least one grade level in an academic year) of student growth” “
Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half grade levels in an academic year) of student growth…” (Fed. Reg., 2009, p. 59805).
The focus on rates of growth in the test scores is not different from asking if profits in a business are increasing, decreasing, or about the same as they were before. An effective teacher is like a sales person who produces higher than average sales or profits.
LikeLike
Your post reminds me of my days in the business world. Your summaries are straight from the advertising/marketing world, which those of us in other fields viewed as ‘cutthroat’– our acquaintances in that field were held to projected sales; if you failed to meet the projection you were fired– if you met the projection you were automatically held to a higher projection the following year. The people I knew there burned out within a few years.
I was in the engrg/constr field, in a procurement capacity. There were many nuances, & common sense prevailed. For example, you had to be careful how you treated your bidders, because your goal was to keep them on board for future competitive bids. The idea was to save money for your company while obtaining a quality product.
Whereas in the advertising world, the idea was to invent a market– convince consumers they needed the product, & sell it to them. That’s the paradigm operating in education today.
LikeLike
Laura Chapman.. I cannot help myself on this one… so in one state (NC?) you have the EV …ASS for evaluating teachers and then you have the PARCC (Crap spelled backwards) for “evaluating” students. Now what acronyms will there be for administrators … Oh wait.. those who impose top-down assessments on students and teachers do not get evaluated!
LikeLike
To answer the question: NOTHING!
Assessment regimes in education are not measuring devices, they are counting devices and counting is not the same as measuring.
LikeLike
Duane Swacker: “counting is not the same as measuring.”
As good as it gets. One most fabulously krazy TAGO!
😀
And that doesn’t even include such questions as what are we counting and why are we counting it and does it even matter how much or little we count it and is it worthwhile or useful to be busy counting it.
“It is not the answer that enlightens, but the question.” [Ionesco]
😎
LikeLike
Because it is well-established that factors such as parental discord, hidden health issues, household financial problems, and myriad behavioral factors can heavily influence a student’s achievement in school and performance on standardized tests, then in accord with the Due Process rights that teachers have, a teacher whose job is on the line because of students’ poor test scores should request detailed information about students’ home life, parental relations, family finances, and other pertinent information. That would raise such a backlash against using test scores to evaluate teachers that use of test scores to evaluate teachers would be swiftly dumped.
LikeLike
From whom should the teacher in your hypothetical be requesting this information?
LikeLike
From parents. And if they refuse to furnish it, under Due Process that’s legal grounds for invalidating the evaluation and for pursuing a law suit if the district persists in using the test results to evaluate the teacher. Due Process is a constitutional right under the 14th Amendment and can’t be taken away by any mere action of a school board or by any local, state or even federal law or by any contract.
LikeLike
I don’t think you’ll find that a parent’s refusal to give you their personal, private information constitutes a deprivation of due process by the state. On the other hand, demanding personal, private information from parents may be independent grounds to fire you, depending on how annoyed the parents are by the request and how tolerant your employer is.
LikeLike
In fire-at-will employment, your shoe size is grounds for dismissal. But if a private company can demand my personal health history and lipid panel results, it is a small leap for the State to demand personal information from students in the name of reform and world class education.
LikeLike
Perhaps, but it would still be a leap. And this commenter is saying that teachers, not the state, should do the demanding.
LikeLike
To most far right conservatives, there is no distinction. It is fair, if we are to base hiring, firing, and compensation on metrics, to ensure the statistical models include these factors, not just adjust for them. The poster had a valid point in that teachers should bring up the horrendous flaws and insanity of VAM as a voice from the classroom.
LikeLike
AMEN, Duane.
LikeLike
Frankly, I like the way you think, Scisne. Fight fire with fire. If the gov ed institution takes the provocative position that teachers can be fired for cause– where that cause is claimed to be their students’ sub-par grades– file a lawsuit citing studies which show that sub-par student grades are caused in the major part by other factors than quality of teaching.
LikeLike
What I continue to not understand, is how humans throughout history, schooled or not, managed through life’s challenges without all of these reductionist metrics …….
LikeLike
That’s just it, they didn’t manage the challenges and we are all just bits of data in a macrocosmic miasma of metrics.
LikeLike
Huh?
LikeLike
My district has struggled with this since they decided 50% of every teacher’s evaluation would be based on “growth” strictly on student scores from one test. I teach Kindergarten, so we have no standardized test scores from Pre-K (yet) to compare their end of year scores to. They came up with their own test that is basically just testing sounds and letters. My students knew most of these in the beginning of the year, so when they take the same test in April-they will show no growth. Not because they didn’t learn anything, but because of the measure they decided to use. Oh and by the way, no one can define for me what percentage of growth they are looking for to count as “growth”. We still don’t have our results from the evaluation last year-I have no idea how they will calculate the growth or if my kids were successful or not…and it’s November.
LikeLike
Tex… That’s the game. You look better on growth if the kids do poorly on the pre-test and do better on the posttest. Also, the administators should know that using the exact test for pre and post is not legitimate.. kids have encountered the items before, and it is clear the test is not a great choice for your students. The test sets you up for not having much room for growth. This is a bum deal and you are not alone.
LikeLike
Have any of these statistical gurus noted that a child who falls just short of the fourth grade standards, who then goes on to fall just short of the fifth grade standards has learned as much as the student who just makes the fourth grade standards, who then goes on to just make the fifth grade standards. Like the basketball team which was behind five points at the half and who loses the game by five points, scored exactly as many points as their opponent in the second half, they just started that half five points behind.
So, that first student showed one whole grade’s worth of growth … and that counts against his teacher? Whatever happened to finding out where they were and then measuring how much they grew from there?
LikeLike
“. . . then measuring how much they grew from there?”
Because there are no legitimate “measuring devices”, no legitimate “standards” for those devices, no measuring whatsoever unless you mean height weight, etc. . . .
All the “measures” are pure hogwash.
LikeLike
Steve,
I believe the measures look at the annual gains, so if a student fails just short of the fourth grade standards and just short of the fifth grade standards the next year, the student is credited with a full year of gains.
LikeLike
Depends on the counterfactuals.
LikeLike
MathVale,
I am a little unsure about the meaning of your post. It would seem that a one year increase is a one year increase.
LikeLike
When is a year increase not a year increase? When the definition of a years growth is contrived based on nebulous and mythical students magically linked to the real students in my classroom.
What if a student scores LOWER on the post test than the pretest? Does that mean they lost knowledge I never taught them? Did I take away learning during the year? The insanity of VAM.
LikeLike
MathVale,
Do you make a determination of a student’s ability to take the down stream class when assigning a grade? If so, it seems to me you are using your own notion of what is a year of learning.
If your student scores lower on your final exam than the pretest, you might want to consider the possibility that your class was a waste of time for that student.
LikeLike
TE, I think the problem here is with annual measure/ assessment, particularly in the PreK/primary grades’ acquisition of reading, writing, & number sense leading to math skills. These are skills which tend to phase in, & overlap, in periods of several years. [Anecdotally, we K-moms noted how closely reading skills tracked to the acquisition of adult front teeth! (which obviously varies over a 2 – 3 yr period among various kids)].
LikeLike
MathVale: given the frail nature of such things as definitions of “grade levels” and the slippery nature of VAManiacal schemes and other metrics built on test scores—
Perhaps, to borrow a phrase from Shakespeare, it is all just “a tale told by an idiot, full of sound and fury, signifying nothing.”
Or perhaps those clinging to the thin reed of the inherently imprecise and narrow EduMetrics of the self-styled “education reformers—dressed up as mathematical certainty—are just here at the bidding of “Dr.” Charlie Chaplin [honest! a PhD in Laughology with a specialization in Smile-esthenics]:
“A day without laughter is a day wasted.”
My day was not wasted.
Keep writing. I’ll keep reading.
😎
LikeLike
TE, you did not answer my question, as usual. You appear confused. The way VAM and SLOs work is to set a baseline entering the classroom, then establish some measure of growth based on a final measure and virtual students. “Downstream” classes are not under my control, but in reality, affect my final rating.
And your veiled insult shows a chink in your normally passive aggressive posting demeaner. Glad I got under your skin.
But your did not answer why the illogical notion of a student scoring LOWER on a post test is considered valid data. This means I am measured on learning the student lost that I never taught them. Try to keep up, please.
LikeLike
MthVale,
I have no idea what insult you are talking about. I take pains to address arguments and idea presented on their merits and not make any personal remarks at all.
If a student’s score on the post test is lower than the score on the pretest, you might want to think about why the student is in that class. This seems like a reasonable position to take. Here is a real life example. My middle son maxed out the math MAP exam as a high school sophomore, so it was in fact very likely that future MAP scores would be lower than his sophomore year score. My spouse and I, along with the high school principal and mathematics professors at the local university all came to the conclusion that further study of mathematics in the high school was pointless, so my middle son began taking mathematics classes at the university.
LikeLike
One thing I’ve kind of wondered is if most kids stay get about the same scaled score from year to year? In Massachusetts I think that does that not count as “growth.”
LikeLike
Education reform is a garden path.
LikeLike
Maybe it’s a garden path in the Garden of Eden where temptation and avarice prevail.
LikeLike
It doesn’t matter because the odds are stacked against teachers. They might as well put a goat into your classroom and have it eat the desks, and based on how many desks it finds delicious, is your effectiveness as a teacher. And know, no matter if it eats a few or a lot, you suck, you will be labeled ineffective, put in a rubber room or fired, your school closed, turned over to a charter or the neighborhood will be vouchered out to a charter owned by and Imam in recluse in the Poconos, NJ. The end.
LikeLike
TAGO!
LikeLike
Thank you, Donna! You made my tea come out my nose – best laugh in a while!
LikeLike
No, Donna, no! Stand up & fight it in whatever way you’re able! We parents are behind you, & any who aren’t just haven’t understood yet what’s going on. It’s a nutty phase we’re going through, & it will not prevail.
LikeLike
Student and school “growth” are abused in many places and sometimes by well-meaning folks. A good source to understand what is reasonable or not about growth can be obtained in pdf form at
http://scholar.harvard.edu/andrewho/publications/practitioners-guide-growth-models .
LikeLike
I find the publication that you have recommended an example of the problem. The whole discussion is aimed at comparing systems of number crunching and equating statistical leaps through thin air with the many and often inexplicable influences on human growth and development, including but not limited to formal schooling.
If all you put into a statistical growth model is test scores all you get out is more information about test scores and statistical estimates, not insight into teaching and learning. It does not help that this publication is not much different from publications from USDE and that the statistical models under discussion do not apply to an estimated 70% of teachers who have job assignments for which there are not statewide tests.
LikeLike
” A good source to understand what is reasonable or not about growth can be obtained in pdf form at. . . ”
A good source to understand what is reasonable or not about “growth” and all the inherent errors involved in the process of the supposed measurement of that growth rendering it completely invalid can be obtained at:
“Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
Brief outline of Wilson’s “Educational Standards and the Problem of Error” and some comments of mine. (updated 6/24/13 per Wilson email)
1. A description of a quality can only be partially quantified. Quantity is almost always a very small aspect of quality. It is illogical to judge/assess a whole category only by a part of the whole. The assessment is, by definition, lacking in the sense that “assessments are always of multidimensional qualities. To quantify them as unidimensional quantities (numbers or grades) is to perpetuate a fundamental logical error” (per Wilson). The teaching and learning process falls in the logical realm of aesthetics/qualities of human interactions. In attempting to quantify educational standards and standardized testing the descriptive information about said interactions is inadequate, insufficient and inferior to the point of invalidity and unacceptability.
2. A major epistemological mistake is that we attach, with great importance, the “score” of the student, not only onto the student but also, by extension, the teacher, school and district. Any description of a testing event is only a description of an interaction, that of the student and the testing device at a given time and place. The only correct logical thing that we can attempt to do is to describe that interaction (how accurately or not is a whole other story). That description cannot, by logical thought, be “assigned/attached” to the student as it cannot be a description of the student but the interaction. And this error is probably one of the most egregious “errors” that occur with standardized testing (and even the “grading” of students by a teacher).
3. Wilson identifies four “frames of reference” each with distinct assumptions (epistemological basis) about the assessment process from which the “assessor” views the interactions of the teaching and learning process: the Judge (think college professor who “knows” the students capabilities and grades them accordingly), the General Frame-think standardized testing that claims to have a “scientific” basis, the Specific Frame-think of learning by objective like computer based learning, getting a correct answer before moving on to the next screen, and the Responsive Frame-think of an apprenticeship in a trade or a medical residency program where the learner interacts with the “teacher” with constant feedback. Each category has its own sources of error and more error in the process is caused when the assessor confuses and conflates the categories.
4. Wilson elucidates the notion of “error”: “Error is predicated on a notion of perfection; to allocate error is to imply what is without error; to know error it is necessary to determine what is true. And what is true is determined by what we define as true, theoretically by the assumptions of our epistemology, practically by the events and non-events, the discourses and silences, the world of surfaces and their interactions and interpretations; in short, the practices that permeate the field. . . Error is the uncertainty dimension of the statement; error is the band within which chaos reigns, in which anything can happen. Error comprises all of those eventful circumstances which make the assessment statement less than perfectly precise, the measure less than perfectly accurate, the rank order less than perfectly stable, the standard and its measurement less than absolute, and the communication of its truth less than impeccable.”
In other word all the logical errors involved in the process render any conclusions invalid.
5. The test makers/psychometricians, through all sorts of mathematical machinations attempt to “prove” that these tests (based on standards) are valid-errorless or supposedly at least with minimal error [they aren’t]. Wilson turns the concept of validity on its head and focuses on just how invalid the machinations and the test and results are. He is an advocate for the test taker not the test maker. In doing so he identifies thirteen sources of “error”, any one of which renders the test making/giving/disseminating of results invalid. And a basic logical premise is that once something is shown to be invalid it is just that, invalid, and no amount of “fudging” by the psychometricians/test makers can alleviate that invalidity.
6. Having shown the invalidity, and therefore the unreliability, of the whole process Wilson concludes, rightly so, that any result/information gleaned from the process is “vain and illusory”. In other words start with an invalidity, end with an invalidity (except by sheer chance every once in a while, like a blind and anosmic squirrel who finds the occasional acorn, a result may be “true”) or to put in more mundane terms crap in-crap out.
7. And so what does this all mean? I’ll let Wilson have the second to last word: “So what does a test measure in our world? It measures what the person with the power to pay for the test says it measures. And the person who sets the test will name the test what the person who pays for the test wants the test to be named.”
In other words it attempts to measure “’something’ and we can specify some of the ‘errors’ in that ‘something’ but still don’t know [precisely] what the ‘something’ is.” The whole process harms many students as the social rewards for some are not available to others who “don’t make the grade (sic)” Should American public education have the function of sorting and separating students so that some may receive greater benefits than others, especially considering that the sorting and separating devices, educational standards and standardized testing, are so flawed not only in concept but in execution?
My answer is NO!!!!!
One final note with Wilson channeling Foucault and his concept of subjectivization:
“So the mark [grade/test score] becomes part of the story about yourself and with sufficient repetitions becomes true: true because those who know, those in authority, say it is true; true because the society in which you live legitimates this authority; true because your cultural habitus makes it difficult for you to perceive, conceive and integrate those aspects of your experience that contradict the story; true because in acting out your story, which now includes the mark and its meaning, the social truth that created it is confirmed; true because if your mark is high you are consistently rewarded, so that your voice becomes a voice of authority in the power-knowledge discourses that reproduce the structure that helped to produce you; true because if your mark is low your voice becomes muted and confirms your lower position in the social hierarchy; true finally because that success or failure confirms that mark that implicitly predicted the now self evident consequences. And so the circle is complete.”
In other words students “internalize” what those “marks” (grades/test scores) mean, and since the vast majority of the students have not developed the mental skills to counteract what the “authorities” say, they accept as “natural and normal” that “story/description” of them. Although paradoxical in a sense, the “I’m an “A” student” is almost as harmful as “I’m an ‘F’ student” in hindering students becoming independent, critical and free thinkers. And having independent, critical and free thinkers is a threat to the current socio-economic structure of society.
By Duane E. Swacker
LikeLike
The problem seems to be a very basic confusion of growth with development.
In biology, growth and development are two very different things. Growth refers to an increase in size (weight, for example). Development involves much more.
Take the example of a caterpillar turning into a butterfly. It actually involves a decrease in overall weight (negative growth), but wholesale rearrangement of the makeup of the living thing — indeed transformation into something entirely new and different.
Human learning is much more akin to the latter than to simple growth.
Most people would agree that trying to gauge the changes from caterpillar to butterfly with a simple measure of growth like weight would be ridiculous — unlikely to tell us anything meaningful.
So why do some folks think it is OK to do a similar thing in the case of human learning?
LikeLike
Post,
Do teachers grade on growth, development, or a mixture of the two when deciding if a student will pass a required class? The answer might begin an interesting discution about what a grade in a class means about the student and if there should be required classes in education.
LikeLike
These discussions are not about grades in any conventional sense.
LikeLike
Laura,
This discussion is about doing high stakes evaluation of students. There is no higher stakes evaluation of students than a teacher assigned grade in a required class. If teachers are going to fail students and prevent them from graduating from high school, it would seem important for teachers to think about why they fail the students.
LikeLike
In most all cases I’ve seen, teachers do not fail students, those “F”s are earned. Most teachers thirst for ANY effort and interest from students to recognize. The objectives are set and students work towards those. You have to look beyond grades as a sole descriprion and see the individual.
LikeLike
MathVale,
How are theses Fs earned? In terms of Poet’s distinction, does it require a lack of growth or a lack of development? Do teachers actually distingiuish between the two when deciding that students have earned an F?
No doubt people are more than thier teacher assigned grades. In my state though, it is the teacher assigned grades that determine if the student is a high school graduate.
LikeLike
TE, I would venture to suggest that SomeDAM Poet’s post is an attempt to get an aerial view of the ‘big picture’ of why the SGO approach isn’t valid. But let’s talk in down-to-earth terms.
It makes great sense to me to define my objectives for the year’s course in terms of what I want my students to achieve. Then it is my job to assess progress toward those goals on a regular basis. The results of my assessments will help me understand where I need to tweak my own pedagogy. Those assessments will also give me a reality check on where my students are actually at, & I may end up adjusting my achievement bar higher or lower over the course of the year.
SGO’s are an attempt to generalize & codify this process so as to be able to quantify metrics for administrative purposes. As a bureaucratic tool, they are a blunt instrument. The main weakness is in assuming that incoming students are at the same learning-level. The downfall is in tying the year’s goal-achievements to yeacher ‘efficiency’, The obvious outcome, built in & incentivized by the process, is that teachers will lowball the initial assessment so as to maximize the appearance of years-‘end ‘growth.’
What fries me about the SGO ‘growth-model now being substituted for NCLB’s crude measurements is that it’s based on the “Management by Objectives” business model– which I myself was required to use in the late ’70’s in engrg/constr field, an ‘accountability’ tool easily gamed– which is now obsolete & roundly criticized (in economic circles) .
LikeLike
Vandykel@michigan.gov
LikeLike
The only “growth” that truly matters here is that of the profits and power of the Overclass, and their so-called reformer proxies.
From that perspective, things are looking really, really good.
LikeLike
I thought a growth was cancer. My bad.
LikeLike
LHP,
I thought growth is what was required for a tudent to graduate from high school. Perhaps I have unrealistic beliefs about high school.
LikeLike
The key to stopping this high stakes testing insanity is parents–OPT OUT of testing so there is no data to be used against teachers and students.
LikeLike