I asked Audrey Amrein-Beardsley to compile a list of the most important VAM research.
Here are her recommendations for the top 13 research articles about Value-Added Measurement:
Amrein-Beardsley, A. (2008). Methodological concerns about the Education Value-Added Assessment System (EVAAS). Educational Researcher, 37(2), 65-75. doi: 10.3102/0013189X08316420.
Amrein-Beardsley, A., & Collins, C. (2012). The SAS Education Value-Added Assessment System (SAS® EVAAS®) in the Houston Independent School District (HISD): Intended and Unintended Consequences. Education Policy Analysis Archives, 20(12), 1-36. Retrieved from http://epaa.asu.edu/ojs/article/view/1096
Berliner, D. C. (2014). Exogenous variables and value-added assessments: A fatal flaw. Teachers College Record, 116(1). Retrieved from http://www.tcrecord.org/content.asp?contentid=17293
Baker, E. L., Barton, P. E., Darling-Hammond, L., Haertel, E., Ladd, H. F., Linn, R. L., Ravitch, D., Rothstein, R., Shavelson, R. J., & Shepard, L. A. (2010). Problems with the use of student test scores to evaluate teachers. Washington, D.C.: Economic Policy Institute. Retrieved from http://www.epi.org/publications/entry/bp278
Darling-Hammond, L., Amrein-Beardsley, A., Haertel, E., & Rothstein, J. (2012). Evaluating teacher evaluation. Phi Delta Kappan, 93(6), 8-15. Retrieved from http://www.kappanmagazine.org/content/93/6/8.full.pdf+html
Haertel, E. H. (2013). Reliability and validity of inferences about teachers based on student test scores. Princeton, NJ: Education Testing Service. Retrieved from http://www.ets.org/Media/Research/pdf/PICANG14.pdf
Haut, M. & Elliott, S. W (Eds.). (2011). Incentives and test-based accountability in education. Committee on Incentives and Test-based Accountability in Public Education, National Research Council. Washington, DC: National Academies Press. Retrieved from http://nfpcar.org/Archive/Education_Evaluation_12521.pdf
Hill, H. C., Kapitula, L, & Umlan, K. (2011, June). A validity argument approach to evaluating teacher value-added scores. American Educational Research Journal, 48(3), 794-831. doi:10.3102/0002831210387916
Newton, X., Darling-Hammond, L., Haertel, E., & Thomas, E. (2010) Value-Added Modeling of Teacher Effectiveness: An exploration of stability across models and contexts. Educational Policy Analysis Archives, 18(23), 1-27. Retrieved from http://epaa.asu.edu/ojs/article/view/810
Papay, J. P. (2010). Different tests, different answers: The stability of teacher value-added estimates across outcome measures. American Educational Research Journal. doi: 10.3102/0002831210362589
Paufler, N. A. & Amrein-Beardsley, A. (2013, October). The random assignment of students into elementary classrooms: Implications for value-added analyses and interpretations. American Educational Research Journal. doi: 10.3102/0002831213508299
Rothstein, J. (2009). Student sorting and bias in value-added estimation: Selection on observables and unobservables. Education Finance and Policy, 4(4), 537-571. doi: http://dx.doi.org/10.1162/edfp.2009.4.4.537
Schochet, P. Z. & Chiang, H. S. (2010, July). Error rates in measuring teacher and school performance based on student test score gains. U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/pubs/20104004/
When asked for the top 25 (including the above), here is her list.
If your curiosity is not sated, here is an even longer list, thanks to Professor Amrein-Beardsley.
You too can be an expert.
Diane Ravitch's blog 
Reblogged this on David R. Taylor-Thoughts on Texas Education and commented:
This is going to be big issue in the next several years. Learn as much as you can so you can fight back against it.
Please give at least equal attention to the 70% of teachers who have job assignments without VAMs (no state-wide tests). For this majority, USDE promotes Student Learning Objectives (SLOs) or Student Growth Objectives (SGOs), a version of 1950s management-by-objectives on steroids.
Teachers who have job-alike assignments fill in a template to describe an extended unit or course they will teach. A trained evaluator rates the SLO/SGO (e.g. “high quality” to “unacceptable” or “incomplete”).
The template requires the teacher to meet about 25 criteria, including a prediction of the pre-test to post-test gains in test scores of their students on an approved district-wide test. Districts may specify a minimum threshold for these gains.
Teachers use the same template to enter the pre-and post-test scores. An algorithm determines if the gain meets the district threshold for expectations, then stack ranks teachers as average, above or below average, or exceeding expectations.
1. The Denver SLO/SGO template is used in many states. This example is for art teachers—-Denver Public Schools. (2013). Welcome to student growth objectives: New rubrics with ratings. http://sgoinfo.dpsk12.org/
2. One of the first attempts to justify the use of SLOs/SGOs for RttT—-Southwest Comprehensive Center at WestEd (n.d.). Measuring student growth in non-tested grades and subjects: A primer. Phoenix, AZ: Author. http://nassauboces.org/cms/lib5/NY18000988/Centricity/Domain/156/NTS__PRIMER_FINAL.pdf
3. This USDE review shows that SLOs/SGOs have no solid research to support their use—-Gill, B., Bruch, J., & Booker, K. (2013). Using alternative student growth measures for evaluating teacher performance: What the literature says. (REL 2013–002). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, Regional Educational Laboratory Mid-Atlantic. http://ies.ed.gov/ncee/edlabs.
4. The USDE marketing program on behalf of SLOs/SGOs—-Reform Support Network. (2012, December). A quality control toolkit for student learning objectives. http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/slo-toolkit.pd
5. The USDE marketing campaign for RttT teacher evaluation and need for district “communication SWAT teams” (p. 9) —- Reform Support Network. (2012, December). Engaging educators, Toward a New grammar and framework for educator engagement. Author. http://www2.ed.gov/about/inits/ed/implementation-support-unit/tech-assist/engaging-educators.pdf
6. Current uses of SLOs/SGOs by state—-Lacireno-Paquet, N., Morgan, C., & Mello, D. (2014). How states use student learning objectives in teacher evaluation systems: a review of state websites. Washington, DC: US Department of Education, Institute of Education Sciences. http://ies.ed.gov/ncee/edlabs/regions/northeast/pdf/REL_2014013.pdf
7. Flaws in the concepts of “grade-level expectation” and “a year’s worth of growth” —-Ligon, G. D. (2009). The optimal reference guide: Performing on grade level and making a year’s growth: Muddled definitions and expectations, growth model series, Part III. Austin, TX: ESP Solutions http://www.espsolutionsgroup.com/espweb/assets/files/ESP_Performing_on_Grade_Level_ORG.pdf
I’m guessing many if not most of these have been peer-reviewed. It could bechelpful to know specifically which ones, since most VAM support is based on sketchy, unreviewed reports and opinion pieces.
Closely related to VAM is the question of standardized testing because in our present circumstances, in the words of the old Sammy Cahn song, “you can’t have one without the other.”
I humbly submit the following for the viewers of this blog: Daniel Koretz, MEASURING UP: WHAT EDUCATIONAL TESTING REALLY TELLS US, 2009.
😎
As you say KTA, VAM is as intimately tied to standardized testing as standardized testing is to educational standards (CCSS) and as head is tied to tail on a coin. Be that as it may, I definitely would be remiss if I didn’t submit Noel Wilson’s “Educational Standards and the Problem of Error” found at: http://epaa.asu.edu/ojs/article/view/577/700
According to my own research the test scores of students we don’t teach, or the scores of subjects we don’t teach have ZERO correlation with our effectiveness as teachers.
My study had a validity and reliability rating of 100%
TAGO!
VAM: The Scarlet Letter
http://youth.be/dfMymU86Bjo
Here’s Arne Duncan’s favorite state, TN.
Memphis won’t have any public schools left when they’re done.
I’m sort of mildly curious. Did they tell the people there the goal was 90% charters when they came in? Seems like public schools didn’t have much a chance of lasting if that was the predetermined goal.
Is “90%” privatized the goal everywhere they land, and if so, shouldn’t they say so at the outset?
http://tn.chalkbeat.org/2014/04/04/charter-schools-continue-to-grow-in-memphis-and-shelby-county/
I saw they had some guy from Memphis peddling the bs charter line to people. It is amazing how easy it is to roll into communities, buy off people, and then sell these garbage schools to people.
It is amazing how they keep replicating this junk all across the country as if has worked somewhere. What a joke.
I remain curious why the goal is always “90% privatized”
What’s that 10% for? Safety net schools? Is that the role of public schools in privatized districts and cities? If so, how do reformers treat public schools when they’re winding them down and replacing them?
10% is for “THOSE” students.
http://www.courier-journal.com/story/news/local/indiana/2014/04/07/experts-question-indiana-teacher-evaluations/7427243/
Indiana starts to question their elaborate teacher evaluation metric along with their elaborate school evaluation metric.
Solution? Remove more pesky human beings from evaluation process!
“Behning said too much district control in determining the evaluations might have skewed the results.
Districts can choose how to conduct the evaluations so long as a “significant” percentage is based on standardized tests in an effort to eliminate biases. Behning said that might be too vague to get comparable results from each district.”
I think they should literally weigh and measure teachers, using a scale and yardstick 🙂
This couldn’t get any more ludicrous, and at least that would give us a reliable measure of….. something or other.
How much do you think they’ve paid to consultants to develop teacher evaluations by now? How much does it cost a district to conduct these experiments?
I bet we could have hired every unemployed twenty-something with a student loan they can’t pay to work with those kids who need extra help by now, and the other kids would have benefitted too, because they’d have more teacher to go around.
Win/win!
It’s a win/win/win because the formerly unemployed teacher-assistants would be paying back their students loans! 🙂
I know a book is not an article, but one book stands out on our shelves… A lot of Jim Horn and Denise Wilburns’ book The Mismeasure of Education debunks the Tennessee “value added” nonsense. Their critique flows in a delightully factual and historically exhaustive way. It is one of the major works we read here in Chicago in CORE and at CTU about the nonsense of utilizing VAM. I suspect the book will also now be catching on among the parents who are leading Opt Out.
George Schmidt: I heartily endorse your pick.
Among other gems is this one re a close sibling of VAM, merit pay, on p. 60:
[start quote]
One of the longest lasting merit pay systems involved extra pay for better test scores in England (Wilms & Chapleau, 1999), and it lasted from 1862 to the mid-1890s:
As historical accounts go, English teachers and administrators became obsessed with the system’s financial rewards and punishments. It was dubbed the “cult of the [cash] register.” Schools’ curricula were narrowed to include just the easily measured basics. Drawing, science, singing, and even school gardening simply disappeared. Teaching became increasingly mechanical, as teachers found that drill and rote repetition produced the “best” results. One schools inspector wrote an account of children reading flawlessly for him while holding their books upside down. [para. 4]
[end quote]
Thank you for the recommendation.
😎