Pasi Sahlberg and Jonathan Hasak wrote a post about the failure of Big Data to introduce effective reforms into education. Big Data are the kind of massive globs of information that cannot be analyzed by one or several people; they require a computer to seek the meaning in the numbers. Big Data are supposed to change everything, and indeed they have proved useful in many areas of modern life in understanding large patterns of activity. Traffic patterns, disease outbreaks, criminal behavior, and so on. But those who try to understand children and teaching and learning through Big Data have failed to produce useful insights. They have produced correlations but not revealed causation. In reading their article, I am reminded of the sense of frustration I felt when I was a member of the National Assessment Governing Board, which oversees the National Assessment of Educational Progress (NAEP). In the early years of my seven-year stint, I was excited by the data. About the fourth or fifth year, I began to be disillusioned when I realized that we got virtually the same results every time. Scores went up or down a point or two. The basic patterns were the same. We learned nothing about what was causing the patterns.
Sahlberg and Hasak argue on behalf of “small data,” the information about interactions and events that happen in the classroom, where learning does or does not take place:
We believe that it is becoming evident that big data alone won’t be able to fix education systems. Decision-makers need to gain a better understanding of what good teaching is and how it leads to better learning in schools. This is where information about details, relationships and narratives in schools become important. These are what Martin Lindstrom calls “small data”: small clues that uncover huge trends. In education, these small clues are often hidden in the invisible fabric of schools. Understanding this fabric must become a priority for improving education.
To be sure, there is not one right way to gather small data in education. Perhaps the most important next step is to realize the limitations of current big data-driven policies and practices. Too strong reliance on externally collected data may be misleading in policy-making. This is an example of what small data look like in practice:
*It reduces census-based national student assessments to the necessary minimum and transfer saved resources to enhance the quality of formative assessments in schools and teacher education on other alternative assessment methods. Evidence shows that formative and other school-based assessments are much more likely to improve quality of education than conventional standardized tests.
*It strengthens collective autonomy of schools by giving teachers more independence from bureaucracy and investing in teamwork in schools. This would enhance social capital that is proved to be critical aspects of building trust within education and enhancing student learning.
*It empowers students by involving them in assessing and reflecting their own learning and then incorporating that information into collective human judgment about teaching and learning (supported by national big data). Because there are different ways students can be smart in schools, no one way of measuring student achievement will reveal success. Students’ voices about their own growth may be those tiny clues that can uncover important trends of improving learning.
“This is where information about details, relationships and narratives in schools become important. These are what Martin Lindstrom calls “small data”: small clues that uncover huge trends.”
Please define “details, relationships and narratives.”
exactement
I think by relationships, they mean the human relationships between kids and teachers, kids and kids, teachers and teachers.
The reason big data doesn’t work is the same as big data wouldn’t work to evaluate, manage and fix family relationships. There are no sweeping rules that work for all. The rules depend on the given family.
Big data works where sweeping, stable patterns are possible. This is why they work in science. In fact, this is how we could define the realm of science: description of stable, verifiable patterns.
I think, the connection between science and relationships (hence big data and small data) can be described when we consider psychology. Psychology identifies common behavioral trends in people. Knowing about these can help manage my own family’s relationships. But ordering the whole US population to follow certain given psychological recommendations in their families would be ridiculous, since the variability in families is so great.
So when they talk about VAM, and they say “the variability was 26%”, what they mean is exactly what I said about the applicability of psychological rules and families: variability of the subjects is too great to establish general rules.
It’s good to know about big data, it’s good to know about statistics, but we cannot be governed by them.
Suggest big data never worked – invariably it collects too much garbage. Businesses have known this
for more than half a century; hence, the focus on information. Those who
count on collecting everything, also count on an expensive marketing campaign to define success.
TESTING kids to death and determining teacher effective via test scores JUST DOESN’T WORK … period. But all this testing sure does generate $$$$$ for the FEW.
We modern “enlightened” human beings have become convinced that “data” only means the type of data that can be used by a computer – things that can be measured/scored, labeled and recorded. Because that’s all a computer is capable of handling. But the human brain can handle so much more – and does from the minute of birth onwards. Facial expressions, tone of voice, body language, sensations, perceptions, emotions, subtle feelings and communications we may not even be consciously aware of. “Big data” forces us to exclude all of that. Those “small data” are what enable relationships to happen, which is what enables teaching and learning to happen. Imparting information, especially in ways that a computer can “measure” (sic) is such a small part of education, but it’s all that tests and computers can do, so people get stuck on “well, we have to measure (sic) something, this is the best we can do”. When in fact this measuring/scoring/labeling/recording is actually interfering with real understanding. It’s the worst we can do, not the best.
“Small data,” parsed and interpreted in the classroom by an experienced teacher in relation to individual children, cannot be readily turned into a commodity and monetized.
Big data, on the other hand…
I think Peter Greene’s post today is rather relevant to the idea of “big” vs. “small” data: http://curmudgucation.blogspot.com/2016/05/improvement.html
Small, often informal data, are more useful than standardized test results. Having taught long before our testing obsession, I learned more about students from informal tests, observations, running records in reading, writing samples, other examples of student work, and games. Yes, even games! The information I gathered was immediately diagnostic and applicable to helping the students learn more and better. Students teach us all the time how to serve them better! We have to be observant, vigilant, pro-active, and have the insights to develop a strategy or plan. When we see the improvement in student performance as a result of our efforts, that is the rewarding, sweet spot, of teaching. Too many teachers have lost this joy due to our misguided policies.
When standardized tests entered the picture, I considered them a necessary evil. Without all the high stakes attached to them, they could be dispensed with efficiently. It was wasted time, and I never learned anything new from the results! Today testing is holding our schools, students and teachers hostage. We need to get back to gathering information that helps teachers help their students rather than feeding the Big Data machine, for a political purpose.
Retired teacher!
Shhhhhh….don’t tell that to anyone! Someone will come up with some horrible rubric we will have to follow and be evaluated on!!
“Students’ voices about their growth may be those tiny clues that can uncover important trends of improving learning.”
This is what Japanese educators have been working on for over a century. The Japanese have essentially created a professional development system that honors teacher learning in conjunction with student achievement. They call this system jugyou kenkyuu or lesson study. One of its main features is that it relies on student thinking in the context of a “live” lesson (the centerpiece of our profession) to inform teachers about the effectiveness of the lesson. In this way, student voices (discussions & artifacts) are heard and studied to improve teaching.
Lesson study is a flexible system that can test existing curricula or innovations. It is a bottom-up model with top-down supports. In Japan, teachers are honored as intellectuals and researchers. In fact, the results from research lessons inform national educational policy. Accountability changes to a verb and is beneficial. Lesson study offers educators a clear vision coupled with an adaptable, systematic process that empowers TEACHERS with an approach to drive reforms slowly & incrementally.
At this time, there are approximately 500 lesson study groups functioning in the United States.
Before “reform” had choke hold on teachers, many school districts had mentoring programs. Veteran teachers worked with novice teachers and guided them without being evaluative. Novice teachers often lack awareness of some things they may or may not be doing. I would have appreciated this type of help when I first started. Instead, it was trial by fire or trial and error. It is helpful to have another person to bounce ideas off, or even to offer encouragement.
IMHO, good education will produce a conscientious citizen for community and society.
All conscientious human beings are responsible for their own well being and are sensible and thoughtful to the welfare of the unfortunate.
Good education will motivate all learners to be creative and to pursuit their life long and boundless learning.
The bottom line for having good education is that teachers, workers and parents need to have a stable income that pay off the basic needs like: rent, food, utilities and transportation cost PLUS tuition fee. People cannot think if they are homeless, sick and hungry.
In conclusion, American corporate, bankers, Pharmaceutical/Technological manufacturers outsource jobs and use temporary visa workers to grip a profit gain. This idiotic/imbecile greed will lead them to be a hunted, NOT a hunter to global business endeavor. Please see automobile and cell phone makers in global market to understand the concept of hunter and hunted manufacturers. Who are owners and who are debtors? Who threatens the stability of global economy? Who invents the nuclear and who becomes fearful of their own invention? Who creates internet and who becomes victims from its own invention due to GREED? Back2basic
And teachers utilize “small data” to make hundreds, if not thousands of decisions every day in the classroom!
I doubt that the Whitney Tilson’s of the world have any clue to what that statement means.
Teachers’ tools : observations and relationships. Works with teacher assessments as well. IMHO
@ Dienne “Facial expressions, tone of voice, body language, sensations, perceptions, emotions, subtle feelings and communications we may not even be consciously aware of. “Big data” forces us to exclude all of that. ”
Big Data is looking to collect this information as well. In a USDE “Grit report”
“”Affective computing is the study and
development of systems and devices
that can recognize, interpret, process,
and simulate aspects of human affect.
Emotional or physiological variables can
be used to enrich the understanding and
usefulness of behavioral indicators.
Discrete emotions particularly relevant
to reactions to challenge—such as
interest, frustration, anxiety, and
boredom—may be measured through
analysis of facial expressions, EEG
brain wave patterns, skin conductance,
heart rate variability, posture, and eyetracking.”
Children used as lab rats
Click to access OET-Draft-Grit-Report-2-17-13.pdf
Big Data is tracking everything through our kids, right down to voter status, Bus Stop ID’s, and beneficiaries of wills. “Research”skirts FERPA law.
1984
Scroll down to see “Attributes” and look at all the options to the left and click on through
http://nces.ed.gov/forum/datamodel/eiebrowser/techview.aspx?instance=studentElementarySecondary