This is a terrific article that appeared on the New York Times blog.
Written by Abigail Zuger, M.D., it is titled “The Real World Is Not an Exam.”
Dr. Zuger explains what happens to the brilliant young doctors who aced every standardized test (there were so many of them!), but were flummoxed when it came time to diagnose a complicated real-life problem presented by a patient.
She gives examples of how these hotshots dealt with new situations: Badly.
They looked for the right answer, but there was none. What was needed was judgment and experience, and they didn’t have enough of either.
Dr. Zuger sees these young doctors as victims of linear thinking, a very bad habit caused by taking too many standardized tests with a single right answer.
She writes:
In 2009, a Supreme Court decision upheld the validity of multiple-choice testing for evaluating firefighters for promotion, prompting a heated nationwide discussion. Critics pointed out that test-taking savvy may have little to do with job performance.
Medical educators have been contemplating this possibility for years. But the problem particular to medicine may be the sheer volume of these tests, and the standard hysterical preparation they engender, which constitute a form of training in itself. Educators may not actually teach to the test, but students think to the test, in linear multiple choice.
They learn to recognize key phrases (neck pain) and stock situations (older woman), and they live in dread of unlikely worst-case scenarios. (Dies from heart attack while buying new pillow. You are sued.) Sometimes the actual, three-dimensional patient is not real enough to eradicate all her paper iterations.
Fear of uncertainty is expensive, and my young friend seemed poised to become single-handedly responsible for the high cost of health care in his ZIP code.
Like all victims of the single best answer syndrome, he ordered tests in wild profusion because, in his experience, every question had an answer and a test that would get you there. Options never included “You decide to keep an eye on it for a little while,” “You tell the patient to come back if it gets worse” or “You must make peace with the fact that you are never going to figure this one out.”
Some medical educators have tried to bend the linear algorithms of the multiple-choice experience to the nuances of the actual clinical world. One set, from the Mayo Clinic, specifically set out to teach against the test. Instead of the usual call-and-response questions, the educators took their students through complicated, contradictory cases for which there were no clear “best” strategies, but many reasonably acceptable ones.
Some students appreciated the point; others complained that all the ambiguity took up too much time. One bemoaned “cognitive overload,” and another wrote, “We know that there are many ‘correct’ ways to do things, but I prefer to be taught one way, rather than many.” (A linear thinker, that one, heading for an unhappy collision with the nonlinear world.)
My young friend never veered from his own well-honed habits. I could tell he thought I was remarkably ill trained, and I’d rather not imagine what he muttered about me to his peers. To my own, I muttered about his remarkable lack of sense and judgment until their eyes began to glaze.
From more than one, I got exactly the same response. “How bad could he be?” they said with finality. “He passed his boards, didn’t he?” Of course, that final hurdle in the path out of training consists of nailing a gigantic quantity of single best answer questions.
And it is partly for this reason that medical students now have to go through simulation testing as part of their certification. Because the practice of medicine is MUCH more than just getting a multiple choice answer correct.
Thanks for the link. Tacit knowledge gained through experience, curiosity, and real-world practice is resistant to easy measurement and so has been devalued in this test-crazed world. This is one reason Teach for America is allowed to exist. And it’s why the Gates Foundation and other “reform” groups can get away with saying that teaching experience (beyond the first few years) doesn’t matter.
In reality, tacit knowledge can be more important than explicit factual knowledge. Opportunities for tacit learning need to be woven into any plan for improving school learning. (See Douglas Thomas and John Seely Brown, A New Culture of Learning, among other titles.)
“Tacit knowledge gained through experience, curiosity, and real-world practice is resistant to easy measurement and so has been devalued in this test-crazed world.”
All of this came out of business management – “you can’t manage what you can’t measure”. It was never intended to apply to every single possible situation, sector or set of circumstances.
Like a lot of perfectly reasonable ideas of limited application and utility it’s been taken to ridiculous lengths and jammed into places where it doesn’t fit.
The first time I heard “continuous improvement” was from people who work in a manufacturing facility where they make tires. Fifteen years later they were grading Ohio public schools with that phrase.
Thank You Chiara…Wise comment..
‘ “Tacit knowledge gained through experience, curiosity, and real-world practice is resistant to easy measurement and so has been devalued in this test-crazed world.” ‘
Who are you quoting? I love it! From a recent experience with the medical profession, it became clear to me that if my doctors were bound in their thinking by a multiple choice format, I might not be here today.
“Tacit knowledge gained through experience, curiosity, and real-world practice is resistant to easy measurement and so has been devalued in this test-crazed world.”
It’s not that those things are “resistant to easy measurement”. They can’t be measured no matter what. There are many things in this world that can’t be measured even though people attempt to “numerize” them in clumsy attempts to “scientize” their poor logic-the magical ability of using numbers to appear authoritative and substantive. It never ceases to amaze me just how gullible the vast majority of folks are to this “numerization” aspect
Yes, there are people who are amazingly good at synthesizing observation but, there are more that suck at it. Consider the synthesis of the anti-vaxxers and the number of dead kids that that chimera has resulted in, or the folks who just knew that the nursery school around the corner was a satanic cult and that the devil was possessing their kids because of it. Lots of innocent people had their lives ruined by that one. The cool thing about doing stuff by the numbers is that it gives useful results even from average physicians. And yes, progress is being made in IT with expert systems.
Not all people are suited to spend their time “Institutionalized”, where a “Degree” is the
point of entry. A “Degree” can be a disability, or an impediment to actual function.
A degree based on “Testing Well” may distort the perspective of achievment WITHOUT
experience.
If institutionalized function could be determined BEFORE a Degree or Certification
is given, the institutionalized could “hit the ground running”. Some might find they
don’t want to spend their time institutionalized and take another path.
What the young doctor is doing is not linear thinking but following a flow chart with decision points (yes/no) based on observation and testing. If the condition is not in the chart she is sunk and flounders.
Next time you are at the quack’s observe how they nod through the diagnosis chart. Head bobs are a good clue
This article should be the poster child for obliterating all the high stakes testing in public education and the accompanying data frenzy based on this testing. Testing does not develop critical thinking skills for real life … only strategy skills for answering something correct. I knew someone who presented himself to a doctor seeped in tunnel vision. He had all the classic symptoms of colon cancer that began to show symptoms. The doctor told him he did not need a colonoscopy because he was too young for colon cancer and gave him medicine for digestion problems. I guess the doctor’s check list told him it was not colon cancer so he did not question my friend about diet and lifestyle etc… The ending was not good. The checklist style rubriks teachers must create or rigidly follow lead to this tunnel thinking in addition to all the bubble in responses students must fill in. Horrible!
There IS a science of medicine
but
the healing ARTS must be the bottom line.
Too – not necessarily applicable to the above but still important to me.
I just finished watching [for the second time in two days] the movie entitled
“Patch Adams”. There is a kind of similarity here in that people are people, with names, not just “sore throats” or cancer problems etc. AND when they are treated as people good things happen.
I remember several years ago when Bill Moyers had a series of programs on “Healing and the Mind”, the importance of one’s outlook, how they perceived themselves and their “illness”.
Obviously, this kind of mentality, as shown above in other writings has a direct bearing on how children learn. It may/may not be important to this discussion but homo sapiens have two halves to their brain. It use only one half is to be a halfwit. In my day this was not a complimentary term.
In my day this was not a complimentary term.
Oh, that’s wonderful!!!
Reblogged this on David R. Taylor-Thoughts on Texas Education and commented:
This is an interesting read. You have new doctors that are allowed to make patients go through unnecessary tests and keep their license but, if a teacher fails more than twenty percent of their students then they get a corrective action plan. There is something drastically wrong here.
Reminds me on the wonderful and terrifying movie “Wit” starring Emma Thompson, where she, a college professor of John Dunne, is dying of cancer, and she is treated by one of her students, a young doctor.
Agree.
And BTW, The play is even better! Well worth reading if you can’t catch a production.
Also, the author….Margaret Edson, is a teacher!
This is one of the reasons why I like to have the oldest doctor around. They have seen everything, and they know how to think outside the box. The younger ones seem to treat the symptoms and not the patient.
It’s just like the current dichotomy in education where the system wants to teach the curriculum and not the children. Patients and students are first of all, HUMAN beings, with all the non-linear messiness that entails. One of the factors that comes into play is non-compliance on the part of the subject. The issue is addressed in medical school, and few fault the doctor when the patient refuses to follow doctor’s orders. The education reformers, on the other hand, don’t seem to understand that it is also an issue with children, who by their nature don’t understand the long-term value of what it is that we are trying to do to them in school.
That’s a quaint idea, LHP, but children are no longer human beings: they are “assets,” meaning they are fungible, as are their teachers.
Michelle Rhee said so herself, and she puts StudentsFirst, so it must be true.
Lol
kids as fungibles
that just about sums up the view that has given rise to the standards-and-testing machine
I was discussing this very topic with my “pain management” doctor the day The New York Times “Science Times” printed part of this analysis. The best specialists know that the ability to actually treat patients’ problems — as opposed to filling in computer sheets after having spent decades bubbling the “correct” answers — is becoming a critical problem. In addition to the problem outlined in this blog post, we are facing more and more “data driven” insanity from the Obama administration, which is using medical doctors as higher paid gerbils in the same kind of cage they want to force teachers into. For decades, most of us knew out “doctor,” who in turn knew not only out “data” but us. There is a HUGE danger today being foisted on the people of the USA by the current administration’s attempt to substitute a culture of insane efficiency — computerized medical charts — for the professional wisdom of trained and experienced medical doctors and nurses. Same insanity, different profession.
Yes and no. The flow charters will spot 90-99% of the problems faster and with fewer false negatives and positives than the “impressionists” (what you gonna do when the impressionist’s image of you is as a hypochondriac) but the question is what happens at the bottom of the flow chart when you hit the WTF bubble.
To a large extent being able to spot what is going on in such cases is a question of experience, e.g. seen it before, so expecting a new physician to be able to see such things is sorta like our sainted sixth grade teachers who told every class that they were the worst she ever had.
The standardized LSAT exam puts a great premium on reading and thinking quickly. In our lawyer-ridden days, there is great competition to get into law school, and thus it is no exaggeration to say that how quickly one works is the major factor determining who gets into the top-tier schools, ranking second only to the size of one’s parents’ bank accounts overseas. Now, working QUICKLY is great for law school itself.
However, not so for the actually practice of law. There, the most important factor is having a grasp of the details. The lawyer who wins is the one who can grab that detail and pull it until the whole case, as put together by the other side, unravels.
And there is a lesson to be learned there: You get what you measure. Well, yeah, I suppose. But the chances are that you are not measuring what you think you’re measuring and that what you are measuring will not be what you really want to get.
cx: for the actual practice of law
If American engineers back in the 1800’s learned through the CC, they would not have been able to build the railroads West. Our engineers outdid others because they thought outside the box.
Interesting..sounds familiar …
This blog topic became hauntingly personal at 10:00 last night when we had to take our daughter to the ER. Despite the obvious excruciating pain that she was in, the attending physician order two tests, both of which were inconclusive, and sent her home at 1:30 AM without even ordering a drip to help re-hydrate her. He fluffed away all her concerns and left without answering any questions, except for mentioning that the blood pressure reading of 92/46 was normal for a young woman her age. He never put his hands on her to help determine what was causing the pain. He just treated her according to the algorithm. We were left wondering why we bothered at all, considering that this non-treatment will end up costing her and her husband several thousand dollars. As I said above, the doctor treated the symptoms and not the patient.
Hope your daughter is better, but what exactly was the physician going to learn by a laying on of hands?
What is more disturbing to me is that the doctor left this family with no understanding of what was going on and no reassurance or direction. You do not dismiss someone who is in extreme pain with inconclusive test results. Whether she is fully recovered or not, the doctor was negligent in not explaining why he was releasing her without further treatment.