Archives for category: Teachers and Teaching
« Older Entries
Newer Entries »

Why VAM Is a Sham

By dianeravitch
October 1, 2013 //
8

Audrey Amrein-Beardsley of Arizona State University is one
of the nation’s leading authorities on teacher evaluation. She has
the advantage of having taught middle school math for several
years. She understands better than almost any other researcher just
how flawed value-added measurement is.

Next year, her book on the
limitations of test-based accountability will be published.

I invited her to contribute to the blog so you would become familiar
with her valuable work.

She writes:

Stock Your Bunkers with VAMmunition

While “Top Ten Lists” have become a recurrent trend in
periodicals, magazines, blogs, and the like, one “Top Ten List,”
presented here, should satisfy readers’ of this blog and hopefully
other educators’ needs for VAMmunition, or rather, ammunition
practitioners need to protect themselves against the unfair
implementation and use of VAMs (i.e., value-added models).

Likewise, as “Top Ten Lists” typically serve reductionistic
purposes, in the sense that they often reduce highly complex
phenomenon into easy-to-understand, easy-to-interpret, and
easy-to-use strings of information, this approach is more than
suitable here whereas those who are trying to ward off the unfair
implementation and use of VAMs do not have the VAMmunition they
need to defend themselves in research-based ways.

Hopefully this list will satisfy at least some of these needs. Accordingly, I
present here the “Top Ten Bits of VAMmunition” research-based
reasons, listed in no particular order, that all public school
educators should be able to use to defend themselves against VAMs.

1. VAM estimates should not be used to assess teacher
effectiveness. The standardized achievement tests on which VAM
estimates are based, have always been, and continue to be,
developed to assess levels of student achievement and not levels
growth in student achievement nor growth in achievement that can be
attributed to teacher effectiveness. The tests on which VAM
estimates are based (among other issues) were never designed to
estimate teachers’ causal effects.

2. VAM estimates are often
unreliable. Teachers who should be (more or less) consistently
effective are being classified in sometimes highly inconsistent
ways over time. A teacher classified as “adding value” has a 25 to
50% chance of being classified as “subtracting value” the following
year(s), and vice versa. This sometimes makes the probability of a
teacher being identified as effective no different than the flip of
a coin.

3. VAM estimates are often invalid. Without adequate
reliability, as reliability is a qualifying condition for validity,
valid VAM-based interpretations are even more difficult to defend.
Likewise, very limited evidence exists to support that teachers who
post high- or low-value added scores are effective using at least
one other correlated criterion (e.g., teacher observational scores,
teacher satisfaction surveys). The correlations being demonstrated
across studies are not nearly high enough to support valid
interpretation or use.

4. VAM estimates can be biased. Teachers of
certain students who are almost never randomly assigned to
classrooms have more difficulties demonstrating value-added than
their comparably effective peers. Estimates for teachers who teach
inordinate proportions of English Language Learners (ELLs), special
education students, students who receive free or reduced lunches,
and students retained in grade, are more adversely impacted by
bias. While bias can present itself in terms of reliability (e.g.,
when teachers post consistently high or low levels of value-added
over time), the illusion of consistency can sometimes be due,
rather, to teachers being consistently assigned more homogenous
sets of students.

5. Related, VAM estimates are fraught with
measurement errors that negate their levels of reliability and
validity, and contribute to issues of bias. These errors are caused
by inordinate amounts of inaccurate or missing data that cannot be
easily replaced or disregarded; variables that cannot be
statistically “controlled for;” differential summer learning gains
and losses and prior teachers’ residual effects that also cannot be
“controlled for;” the effects of teaching in non-traditional,
non-isolated, and non-insular classrooms; and the like.

6. VAM estimates are unfair. Issues of fairness arise when test-based
indicators and their inference-based uses impact some more than
others in consequential ways. With VAMs, only teachers of
mathematics and reading/language arts with pre and post-test data
in certain grade levels (e.g., grades 3-8) are typically being held
accountable. Across the nation, this is leaving approximately
60-70% of teachers, including entire campuses of teachers (e.g.,
early elementary and high school teachers), as VAM-ineligible.

7. VAM estimates are non-transparent. Estimates must be made
transparent in order to be understood, so that they can ultimately
be used to “inform” change and progress in “[in]formative” ways.
However, the teachers and administrators who are to use VAM
estimates accordingly do not typically understand the VAMs or VAM
estimates being used to evaluate them, particularly enough so to
promote such change.

8. Related, VAM estimates are typically of no
informative, formative, or instructional value. No research to date
suggests that VAM-use has improved teachers’ instruction or student
learning and achievement.

9. VAM estimates are being used inappropriately to make consequential decisions. VAM estimates do not have enough consistency, accuracy, or depth to satisfy that
which VAMs are increasingly being tasked, for example, to help make
high-stakes decisions about whether teachers receive merit pay, are
rewarded/denied tenure, or are retained or inversely terminated.
While proponents argue that because of VAMs’ imperfections, VAM
estimates should not be used in isolation of other indicators, the
fact of the matter is that VAMs are so imperfect they should not be
used for much of anything unless largely imperfect decisions are
desired.

10. The unintended consequences of VAM use are
continuously going unrecognized, although research suggests they
continue to exist. For example, teachers are choosing not to teach
certain students, including those who teachers deem as the most
likely to hinder their potentials to demonstrate value-added.
Principals are stacking classes to make sure certain teachers are
more likely to demonstrate “value-added,” or vice versa, to protect
or penalize certain teachers, respectively. Teachers are
leaving/refusing assignments to grades in which VAM-based estimates
matter most, and some teachers are leaving teaching altogether out
of discontent or in protest.

About the seriousness of these and
other unintended consequences, weighed against VAMs’ intended
consequences or the lack thereof, proponents and others simply do
not seem to give a VAM.

Categories Teacher Evaluations, Teachers and Teaching, Testing

Patrick Welsh: Four Decades of Failed Reforms

By dianeravitch
September 30, 2013 //
16

Patrick Welsh is an experienced teacher who writes often in the Washington Post about the real problems of schools. If only the editorial board of the Post–besotted with the failed strategies of corporate reform–would heed the wisdom of Patrick Welsh!

In this article, he describes the many reforms that have been imposed in teachers in his high school since he started teaching forty years ago. The article refers to “four decades of failed school reforms.”

Now we are in the worst of all reforms, where the “reforms” are devised by non-educators or people who taught for a year or two. Where non-educators or those with minimal experience are made state commissioner of education and use their power to demoralize teachers and destroy the teaching profession.

This era will end. But how many excellent teachers will we lose before the reform industry admits its failure and goes away?

Categories Corporate Reformers, Education Industry, Education Reform, Humor, Teachers and Teaching

Philly Teacher: May 100% of Your Students Score Proficient by 2014!

By dianeravitch
September 28, 2013 //
40

This was written by Raniel Guzman, who is a teacher in the School District of Philadelphia and an adjunct professor at Esperanza College of Eastern University:

May 100 % of your students score proficient or above on standardized tests by 2014.            

An attributed Chinese proverb is often wished upon one’s enemies by asserting, may you live in interesting times… This understated “curse” levied upon one’s enemies has a restrained Buddhist sensibility even as one wishes ill toward others. Educators today are indeed living in interesting times. Students and parents are certainly living in interesting times as well. However, the curse placed upon us all is not restrained but rather overt.

The curse is well known to educators and asserts the following, may 100 % of your students score proficient or above on standardized tests by 2014. So then, who has placed this “curse” upon us all? I am certain we can think of obvious enemies. Nonetheless, I am not certain many of us are thinking about the less obvious and thus more lethal enemies. They give politically correct speeches, and radiate a fatherly presence. Their threat resides precisely in the proximity to their victims, namely us. We often develop a blind spot for such figures and hope that they will protect us from sorcerers and things that go bump in the night. Unfortunately, simple examination of deeds, rather than speech, proves otherwise.

Their “curse” is characterized by dilution and diminution. Teachers, students, administrators and parents are diluted in endless paper chases disguised as tests, assessments and reports. Conversely, the edification of concepts and individual free will are systematically diminished. The combined effects may elicit the maddening image of a hamster running endlessly in a caged wheel. However, a more accurate image of these effects is more akin to desperate victims racing to the top of an inextinguishable inferno.

May 100 % of your students score proficient or above on standardized tests by 2014 is particularly stressful when father figures emphasize the deadline, by 2014. This has been the curse that has dominated the bulk of my teaching career. A curse so powerful it has decimated all attempts to render it inoperative. This “super curse” has brought forward other conjurers, who with wands in hand have temporarily waved away some provisions yet, have not been able or willing to undo this “super curse”. What would motivate someone to place such a curse?  Let’s entertain some thoughts.

Many profess that a goal as noble as ensuring that all students score proficient at the same time and place is beyond reproach. Well, it is simply reprehensible. This cynical goal suffers from a pernicious pathology, which advocates forgive as a delusion for perfection. Shamefully, these apologists hide the correct diagnosis. It is not a delusion of perfection that motivates the jinxers, but rather a pathology of exclusion. We all know the consequences of not scoring proficient and obtaining AYP, -they close your (our) school. However, 2014 can’t seem to arrive soon enough for some hexers. Hence, the rush by officials –this year-, to close as many public schools throughout our country under the convenient excuses of “austerity” and “scores” is well afoot. The unprecedented shuttering of dozens of public schools particularly in largely African American communities, as in Chicago, Philadelphia, Washington D.C. etc. are justified under the rationale of budgetary challenges.

This rationale anchors itself on the operational premise of right sizing. Irrespective of the fact that we are talking about children and their development -this thinking may have legitimate administrative basis in the private sector. However, the current juxtaposition of private sector practices and public sector commitments, such as providing an adequate and free education as stipulated in the constitution of the Commonwealth of Pennsylvania, have dissimilar agendas and unequal strengths. Why then the rush if the “super curse” will effectively close thousands of public schools all across our country within a year? We will surely have the right size and number of schools soon enough. Could it be that the proponents’ zeal for the “super curse” is waning? Hardly.

In fact the “super curse” is going very well. The political party, pardon, -the political parties, both of them, have put away their “profound pedagogical partitions” to unite in this effort. Likewise, the industrial testing complex continues to redact tests, assessments, practice workbooks, study programs, while hiring consultants, garnering ever greater budget allocations, and accumulating data all aimed to ensure that scores comply with targets of soon to be imposed common core standards. But above all of these machinations, one supersedes them all, the president’s silence on these matters is the best indicator that all is truly going well among the jinxers and hexers. Conversely, hope for a change is sequestered. So again I ask, why the rush? Could it be that the “super curse” itself is waning? Hardly.

In fact, the jinxers are emboldened by their powers and lack of meaningful restraints, -why wait? “Let’s exclude NOW!” they demand. What we have in front of us now is a turn to Kronos, but an inverted Kronos. We have all seen the depictions of Kronos devouring his children. His filicides are motivated out of fear, a fear of competition from his children. Thus, he eats them. In our case, our Kronos is not committing filicide, but rather devouring the children of others. One can imagine a repented titan with a renewed paternal instinct and displaced fear of competition in the presence of other children asking another titan, “How do we devour the potential competitors of our children?” The other titan may respond, “Well, one way is to be proactive. For example, you offer your children greater pedagogical and assertive experiences and opportunities. In addition, you enroll your children in a school that is free of stigma, for example, one that does not administer standardized tests, say a private school or a divinity school. In other words, enroll your child in a school that does not partake in omens or curses.”

Nonetheless, there is still another way, a more reactive way, to devour your children’s potential competitors, -you close their schools and in doing so -eliminate the competition. Once shuttered, you place stigma upon the displaced children and adults. Moreover, you place a scarlet tattoo on both and you never lift the curse. The calculation is the following; the failure of some children will ensure that mine will thrive. This charter is an open collusion devoid of formalities.

I am well aware that we live in highly secular times, which are dominated by facts and figures. But for some of us who recognize the evil that lurk in men’s hearts, we cannot ignore the immutable. Those who conspire against children are devoid of judgment. Consequently, their motivations are drawn from an irredeemable well. They practice technical numerology, model apparitions, and consult conjurers. Some of them believe in curses. Others still are weary of omens. Some even fear children, often their own. I believe in God. Evil may cause great pain and destruction, but evil never prevails. Evil’s harvest never mature and eventually in its rage devours its own seed.

Categories NCLB (No Child Left Behind), Philadelphia, Teachers and Teaching, Testing

Why Teachers Should Write the Tests, not Corporations

By dianeravitch
September 27, 2013 //
47

While I was traveling to Denver, I received a request from The New Republic to respond to an article by Dr. Ezekiel Emanuel, the provost of the University of Pennsylvania, who asserted that we need more tests, because studying for tests makes kids smarter. I wrote this article, responding to Dr. Ezekiel that what matters most about tests is who writes them, how quickly results are reported, and what they are used for. The best tests, I argue, are made by the teachers. They know what they taught. They get. Almost instant feedback. They can see which students need extra help. And they can learn how to improve their teaching.

Yes, Dr. Emanuel is the brother of the mayor of a large US city.

Categories Teachers and Teaching, Testing

Enrollments in Teacher Preparation Program Plummets

By dianeravitch
September 25, 2013 //
30

According to a new report by Edsource in California, enrollment in teacher preparation programs in that state continues to plummet.

Teacher layoffs and budget cuts combine to make teaching a bad bet as a career.

The attacks on teachers by prominent reformers no doubt add to the diminishing prestige of teaching as a profession.

The reformers’ insistence that a “great” teacher needs only five weeks of intensive training is no doubt another contributing factor.

So, while “reformers” insist that the teacher is the most important factor in closing the achievement gap and producing students prepared for college and careers, their ideas are destroying the teaching profession.

This is not the lesson to be learned from other nations that the reformers claim to admire, like Finland, where entry into the profession is highly selective and the course of preparation lasts five years.

What is happening is criminal.

Categories Accountability, Teachers and Teaching

Teacher Education in Finland

By dianeravitch
September 15, 2013 //
114

Finland is generally recognized as one of the
world’s highest performing nations. Over the past decade, Finnish
students have been high performers on the international PISA exams.
In Finnish schools, students never take a standardized test. How is
their progress assessed? By their
teachers.

Finnish educators say that the key to
their success is the high quality of their teachers. Not just a
star here and there, but the profession as a whole has high
standards for entry and for preparation. There are no shortcuts to
becoming a teacher in Finland. Teachers are highly respected, just
as much as other professions.

Finland believes in
high-quality teacher education. Students apply to enter teacher
colleges at the end of high school. The small nation’s eight
teacher preparation institutions are highly selective. Only one of
ten applicants is accepted, based on multiple measures, including
an essay, an entry test, an interview, and evidence of a high
motivation to teach. In addition to studying liberal arts subjects
and the subjects they will teach, future teachers study pedagogy,
theory, and conduct research about education. They learn how to
teach students with disabilities.Tthey take the study of education
seriously. They practice teaching. Preparing to become a teacher
takes five years. Then and only then may they become
teachers.

Higher education is completely free.
Finland views education as a basic human right, and as such, free
of cost to students. Thus, graduates of higher education in Finland
have no student debt to pay off. They can get as much education as
they want at no cost to them, because it is good for
society.

There are no alternative routes into
teaching. There is no Teach for Finland. Nor would anyone be
accepted as a teacher with an online degree. Nor would someone who
had a degree in physics or history be allowed to teach in a Finnish
school unless they had the required pedagogical
preparation.

Once graduates of the pedagogical
institutions become teachers, they have wide latitude about their
daily work in the classroom. Within each school, the principal and
teachers together make many decisions about what and how to teach
The national curriculum provides guidelines, but does not intrude
upon the professionalism of teachers. Teachers are trusted to make
the right decisions about and for their
students.

Finland has a NAEP-style national
assessment, but it is (like NAEP) based on sampling and has no
consequences for students, teachers, or
schools.

Because there is no standardized
testing, teachers are never evaluated by the rise or fall of their
students’ test scores. There is no value-added assessment in
Finland.

Finnish schools have small classes (I
visited three schools and never saw a class with more than 20
students). Finnish teachers use technology as a matter of course.
The arts are very important in Finnish schools, as are recess and
physical education.

Almost every Finnish teacher
and principal belongs to a union. They belong to the same union.
The union represents the interests of the profession in discussions
of national policy. Once a person becomes a teacher, they have
lifetime tenure. Few people leave the profession for which they
have trained so rigorously. The working conditions are good. They
are held in high esteem by their fellow citizens. Why would anyone
want to leave?

In Pasi Sahlberg’s award-winning
book, “Finnish Lessons,” he says that the crucial reforms in
Finnish education were drawn in large part from American educators
like John Dewey. That is why the teaching profession is highly
valued, and the classrooms are student-centered, test-free, and
devoted to the full development of each child’s full
humanity.

Categories International, Teachers and Teaching

Art Teacher: Just Say “No” to Teaching to the Test

By dianeravitch
September 14, 2013 //
29

A retired teacher is happy that parents are opting their children out of state testing

She writes:

“I am a former elementary art teacher and I am thrilled parents are taking matters into their own hands. The testing culture is madness. I had to sit in too many staff meetings, and watch as the administrators devote all professional development time to students passing the standardized tests. It did not matter that students did not take tests in art, the specialists still had to sit there and listen for hours about how to teach to the test. All resources were put toward classroom teachers teaching to the test. All professional development days were used for classroom teachers teaching to the test. The specialists were told to find something to do on those days. I thought to myself, we as educators need to just say no. I am glad someone finally is saying no!”

Categories Teachers and Teaching, Testing

How Many Teachers Leave the Profession and Why?

By dianeravitch
September 12, 2013 //
71

A regular reader has posted several comments that seem to
imply that not enough teachers are being fired. Or that a system
with a small number of teachers fired was not up to par, assuming
that there are many “bad” teachers who have not been found out yet.
This seems to be the assumption behind Race to the Top and the
Gates’ approach to evaluation: stack ranking, from top to bottom.
Fire the bottom. I responded that about 40% of teachers leave
within the first five years of starting their job. He asked for
evidence. Good question. Here are two good sources. Ken Futernick
of Wested in Sn Francisco wrote an excellent article called “Incompetent
Teachers or Dysfunctional Systems?” Matt Di Carlo wrote
a good overview of
the research here. In no other profession do so many
people exit so rapidly. This suggests to me that states and
districts should have high standards for hiring teachers and then
should mentor new teachers, build a collegial culture, and make
sure that retention is a goal. We make a huge mistake with the new
evaluation systems, which seem intended to find and fire weak
teachers. The goal should be to make teachers better, if they are
willing to be helped. Churn is bad.

Categories Teacher Evaluations, Teachers and Teaching

Are TFA and TNTP Teachers Better than Traditional Teachers?

By dianeravitch
September 10, 2013 //
48

This question comes up again and again, and different studies reach different conclusions. Typically, TFA teachers get better than usual results in math, but not in reading, which is less susceptible to test prep and more influenced by home life.

Mathematica Policy Research released a new study today, saying that TFA and TNTP teachers get better results in math than traditionally prepared teachers. But Dana Goldstein analyzes the findings and learns that the headline oversimplifies.

For one thing, the gains were modest: “For the average child in this study, who scored in just the 27th percentile in math compared to her peers across the country, having a TFA teacher will help her move up to the 30th percentile–still a long way off from grade-level math proficiency.”

For another, the study shows that experience matters: “The bias against first-year teachers is borne out in the data. The students of second-year teachers outperformed the students of first-year teachers by .08 standard deviations–a larger gap than the one between the students of TFA and non-TFA teachers. And even though TFA recruits did well in this study, that doesn’t mean teachers reach their pinnacle after two years on the job. To the contrary, the researchers found that for teachers with at least five years of experience, each additional year of work was associated with an increase of .005 standard deviations in student achievement. ”

And Goldstein notes that 89% of the TFA teachers in the study were white, which causes concern because there are many reports of urban districts losing teachers of color, especially African Americans. That may be as big or bigger a problem in the long run that a few percentage points up or down.

Categories Teach for America TFA, Teachers and Teaching, Testing

Jack Schneider: What Do Students Remember about Their Favorite Teacher?

By dianeravitch
September 9, 2013 //
13

This is an excellent article by historian and former high school teacher Jack Schneider.

He writes about the students who remember him fondly many years after graduation.

What do students remember?

Here is a sample:

My best teachers taught me how to read, write, and cipher.  But they also treated me with kindness and humanity.  They made me feel like I was welcome in their classrooms.  They instilled in me the sense that I mattered.  They inspired me to grow up and be like them.  Where, I wonder, are those kinds of characteristics in our current policy discussions about teacher recruitment?  Where is that in talk among so-called reformers about overhauling teacher training?  Where are those traits in our evaluations of the “value” added by teachers?

Categories Teachers and Teaching
« Older Entries
Newer Entries »