As hard as it is to believe, I've been a physician for 23 years now and a fully trained surgeon for over 15 years. If there's one thing I've learned in that time, it's most doctors really, really don't like to be told what to do. I don't know if part of it comes from all the long years of medical school and residency, with fellowship tacked on for many, during which we're relentlessly told what to do by more senior residents, fellows, and attendings or if it has something to do with the personality traits that lead young people to go into medicine, particularly surgery. It's probably a little of both.
Be that as it may, one negative consequence of this particular personality trait is that many physicians are surprisingly--shockingly, even--resistant to the very concept of evidence-based medicine. For purposes of this post, regular readers should be aware that, just this once, I'm not going to worry too much about the distinction between evidence-based medicine (EBM) and science-based medicine (SBM), mainly because in the case of plausible treatments based on sound science the differences between EBM and SBM tend to disappear because the results of clinical trials usually represent the culmination of preclinical investigations that establish the science behind the treatment being tested. Just the other day I saw a post by a blogging orthopedic surgeon who goes by the 'nym The Angry Orthopod (AO) entitled Evidence-based medicine removes a physician's autonomy. It's an annoying little screed chock full of straw men attacks against EBM/SBM slathered with copious quantities of arrogance and disdain for scientists and an inflated sense of the author's own ability to synthesize the medical literature into his approach to patient care. You'll see what I mean in a second.
After sarcastic rejoinders about EBM, the AO starts in earnest:
Evidence based medicine, or EBM, may be just another way to remove a physician's autonomy. This trend has marched on for years, castrating us bit by bit. EBM is nothing more than the old process of peer-reviewed journal articles, but now there's a classification systems that grades according to the article's strengths or weaknesses. In other words, it's to help the non-academic dummies tell the difference between crap and quality. In the U.S., a five-level scale is favored, while the U.K. prefers a four-stage system, and there are others.
AO says that as though it were a bad thing.
A little Orac-ian sarcasm aside, though, this is a common viewpoint, namely that somehow physicians should have near absolute autonomy to do whatever the heck they want treating patients (constrained, apparently, only by malpractice laws and the weak enforcers that are state medical boards), that they are somehow all-powerful, all-knowing gods of medicine whose judgment must not be questioned. Indeed, what I sense more than anything else from AO is resentment and suspicion at anything that might constrain his autonomy. To a certain extent this is not a bad thing (after all, part of being a profession is to have a degree of autonomy to set standards of performance), but this is ridiculous. What is apparently pissing AO off so much is an attempt to systematize how evidence is evaluated and applied when it comes to clinical questions, in essence using that system to provide both information and guidelines to physicians about what the latest evidence shows. AO's apparent confidence in his own abilities notwithstanding, a lot of doctors could actually use the help. Many, myself included, even welcome the assistance. But not AO;
My interest in this is truly for the end recipient: you, the patient. I think EBM at its core is a good thing, but its ultimate use must be questioned. The obvious objective for EBM is to arrive at the best care for the patient with a certain diagnosis. The subversive goal of EBM is to "mechanize" the whole medical delivery system and put the decision-making process on a prescribed pick list.
I've heard this sort of complaint many, many times before. In particular, I've heard it with respect to checklists, even though there is quite a bit of evidence that judiciously used checklists improve patient care and patient outcomes. There's a paranoid component to these sorts of objections to EBM/SBM as well. Often physicians ascribe to the motives of supporters of EBM as a desire for "control" or claim that it's insurance companies trying to save money. In fact, AO makes exactly that argument near the end of his post. It's very predictable; any time you see broadsides against EBM like this inevitably the person writing the screed will imply that insurance companies and government payers like Medicare are behind it all. They're not, but not surprisingly third party payers do like and encourage EBM because it does have the potential to improve outcomes and (possibly) save money. What's often forgotten, though, by doctors like the AO is that EBM can also cost money. What if the literature finds that a a very expensive intervention is the most effective? Or that, for example, screening for cancer with breast MRI is far more effective than mammography? When it comes to cost, EBM can cut both ways; yet its critics (and sometimes its proponents) always seem to assume that it will only ever cut costs.
Financial considerations aside, though, as I've said before, it all appears to boil down to not wanting to be told what to do.
In fact, "cookbook medicine," as AO so disparagingly calls EBM and evidence-based guidelines is very important in improving patient outcomes; yet all too often physicians don't follow them. One of the commenters took AO to task by pointing out so aptly that, even after many unassailable studies have shown the benefits of aspirin, beta blockers, ACE inhibitors, and other interventions that there are still quite a few patients whose outcomes are jeopardized because these modalities are not appropriately prescribed by their doctors. My favorite example is the fact that it's been shown time and time again that antibiotics do not do any good for viral infections. Yet large numbers of physicians continue to prescribe them for obvious viral illnesses. As commenters pointed out, there is no "subversive" goal behind EBM. The goal of EBM is quite overt, and that's to standardize the parts of medicine that can be standardized based on evidence, leaving the more complex problems and questions that are either not well addressed by clinical trials or for which evidence is conflicting for physicians to exercise their professional judgment upon. Think of it as triage, in which guidelines for clinical situations on which the literature is clear are established and a framework for evaluating the literature for clinical situations that aren't so clear is created in order to assist physicians make decisions.
It's rather interesting to look at one of the article that AO references as a criticism of EBM. Among other complaints, the authors of the article criticize EBM based on philosophy:
The first type of criticism involves the philoso- phical underpinnings of EBM, which is based on empiricism. In its rawest form, EBM elevates experimental evidence to primary importance over pathophysiological and other forms of knowledge, and implicitly assumes that scientific observations can be made independent of the theories and biases of the observer.
What I found hilarious about this critique is that it's exactly the opposite of the criticism of EBM that many of us make, namely that it undervalues basic science and knowledge of pathophysiology that is well-established and overvalues forms of clinical trials that, in the case of highly implausible therapies like homeopathy, easily produce false positives. The rest of their complaint boils down to a load of postmodernist nonsense and complaints that science is "biased." The authors also complain that EBM doesn't incorporate "professional experience" into it. Of course, professional experience shouldn't be part of EBM. That's rather the point, to minimize reliance on subjective standards like "professional experience," which is why it is entirely appropriate to relegate expert opinion to the lower rungs of the evidence hierarchy. Professional experience is what a physician uses to evaluate EBM guidelines, not a part of those guidelines.
Not that this stops AO from going further into hostility against EBM:
Of course the randomized, triple-blinded, placebo-controlled study is the research crown jewel, but those studies are far and few between, especially in our paranoid, liability-fearing world. Who decides the assignment of a level? Does a higher-level study render all lower ones irrelevant by default? Can't I decide which articles are accurate and relevant, to me?
Apparently AO can't. After all, he can't figure out whether a higher level study renders all the lower level ones irrelevant by default. I know, I know, he's asking rhetorical questions, but, really, his hostility towards EBM makes me wonder whether he's equally hostile towards evidence that doesn't fit his preconceptions about how orthopedics should be practiced. It's often been said that there are two kinds of medicine, EBM/SBM and "experience-based" or "dogma-based" medicine. AO's protestations to the contrary notwithstanding, his utter confidence in his own ability to evaluate the literature without assistance (and that's all EBM really is when it comes down to it, assistance in synthesizing the medical literature) leads me to think that AO falls closer to the dogma-based, rather than evidence-based, medicine camp.
As I said before, I'll say again: AO's objection to EBM seems to be far more rooted in not wanting to be told what to do, even going so far as to compare doctors as "lambs to the slaughter":
Doctors, the ultimate holders of the key to patient care, are being lead to slaughter in years to come, and EBM is part of the puzzle.
And to conclude:
If allowed, EBM will change medicine from a practice of individual-based, case-by-case care to cookie-cutter cookbook recipes. Maybe some docs need a cookbook but I don't. The docs I respect don't either.
Comparing EBM to "cookbook medicine" betrays a total lack of understanding of what EBM is. it's not cookbook medicine, and I say that as someone who practices by evidence-based guidelines. Most of the physicians at our cancer center practice based on the National Comprehensive Cancer Network (NCCN) guidelines. These guidelines are extensive, comprehensive, and detailed, but even with a high level of detail there's still a hell of a lot of clinical judgment that goes into our practice because we often encounter situations where it is not clear which guidelines apply or for which there is no real guideline. Moreover, the guidelines themselves often have caveats and appeals to clinical judgment or a list of evidence-based choices. One example that we encounter a lot now is the question of whether in breast cancer a surgeon should remove the rest of the lymph nodes under the arm after one or two of them are found to be positive for metastatic cancer. In the past, we used to do it routinely, but then there was a major study earlier this year that says it's not necessary in most cases. What to do? It's not likely that there will be another study to answer the question because the study (ACOSOG Z0011) never managed to complete accrual because of increasing resistance among patients to being randomized to axillary dissection (removing all the lymph nodes under the arm). Consequently, we have to use our clinical judgment about how much credence we place in this one study in the context of all the studies that went before because Z0011, for all its flaws, is currently the best study we have about this question, and for practical reasons probably the last.
It's sometimes been said that those who complain the most about EBM and show the most hostility towards the very concept are the ones in most dire need of EBM. I would tend to agree, because hostility towards the concept to EBM in particular to me implies hostility towards scientific evidence in general when it doesn't fit with one's current practice and an overweening arrogance that lends far more faith to one's own "clinical judgment" than is warranted. Humility is in order, and unfortunately it's in short supply among many physicians like AO.
It's tempting to conclude by recalling a very famous physician from 150 years ago by the name if Ignaz Semmelweis, so tempting that I am going to do it. Pretty much every doctor knows who Semmelweis was. Basically, he was an Austrian physician who noticed a much higher death rate from puerperal fever in the obstetrical wards administered by physicians and medical students than in the wards administered by midwives and midwife students. To make a long story short, Semmelweis noticed a link between cadavers and a puerperal fever-like disease and hypothesized that the source of the higher rate of puerperal fever in the physician-administered obstetrical ward was material from the cadavers that physicians and medical students brought with them after doing autopsies and not washing their hands. (Disgusting, I know.) To test his hypothesis, Semmelweis mandated that anyone doing autopsies had to wash their hands with a solution of chlorinated lime (basically, bleach) before examining or working on patients. As a result the death rate from puerperal fever plummeted dramatically.
Unfortunately, the reason why the story of Semmelweis is a cautionary tale in medicine is that Semmelweis had difficulty convincing his fellow physicians of his results. Doctors had a hard time accepting the evidence, as blindingly clear cut as it was, and had a hard time accepting that they might be harming patients. In essence, many (but not all) of them refused to change their practice based on this evidence. In reading AO's little broadside against EBM/SBM, I couldn't help but hear echoes of Semmelweis's critics from that long ago time. I like to think we as a profession had advanced beyond that, but every so often I'm reminded that our time is not as different from that of Semmelweis as I would like to think. I also can't help but wonder if AO is a member of the AAPS, whose Ayn Randian belief that they are "supermen" (and women) whose egoism and genius will inevitably prevail over timid traditionalism and social conformism--not to mention pesky things like EBM that seek to constrain their creativity and genius with mundane things like evidence. Such thinking leads to some very unfortunate occurrences, such as the acceptance of cranks as "brave maverick doctors." It is not a coincidence that the AAPS as an organization is very hostile towards the very concept of EBM and uses arguments very much like those of AO, arguments based mainly on how EBM is "castrating us, bit by bit." The analogy reveals a lot about AO's mindset and that of those who show similar hostility towards EBM.
But then what do I know? I'm just one of those pointy-headed academics who think that the systematization of evaluating scientific evidence in medicine is in general a good thing in the vast majority of cases.
And here I was wondering what to write about today. Thanks for this, Orac, you've made a few things clearer in my mind. You raise some very interesting issues that my inner psychologist is currently salivating over.
AO doesn't understand cookery either, by the way.
He needs to replace his descriptions with the word 'evidence'
"If allowed, EBM will change medicine from a practice of individual-based, case-by-case care to cookie-cutter cookbook recipes. Maybe some docs need [evidence] but I don't. The docs I respect don't either.
And admit that he just wants to make things up:
"Can't I decide which articles are accurate and relevant, to me?"
No, the evidence decides which are accurate and relevant - and it doesn't matter who you are... what he's really saying is:
Can't I decide which articles are accurate and relevant, to me? Regardless of the evidence.
The answer is no. No you can't. Nobody can just 'decide' what's accurate and relevant without looking at the evidence.
Funny you should mention checklists, Orac. I just read a news story. Doctors at a Croatian Hospital removed the healthy kidney of a 56 year old woman. She was in for a back operation, and another woman with a similar first name and the same surname was in to have her kidney removed. The two were mixed up.
AO strikes me as incredibly arrogant. Even the best of us can make mistakes.
I think it should be noted that in the Semmelweiss case, one important detail that often gets omitted is that he was a raging asshole. Seriously, go read up on it - he didn't just tell people they should wash their hands, he used to barge in on other departments in the hospital and ridicule the staff (sometimes in front of their patients) who didn't wash their hands. Rather than just present his evidence, and let people draw their own conclusions, he basically demanded the hospitals he worked at adhere to his guidelines without any further consideration.
Yes, his story is very sad, and yes, there were definitely doctors who just covered their ears and said "Lalalalala, I can't hear your evidence", but certainly Ignaz didn't help his own case any.
I'm aware of Semmelweis' difficulties with interpersonal relationships; I've even mentioned it myself in previous posts in years past. I'm also aware that he wasn't too great at presenting his work at professional conferences. It only partially explains the virulent resistance to his results. A lot of physicians who didn't know Semmelweis rejected his results too. Part of the problem was that Semmelweis' discovery occurred about 20 years before germ theory began to be established. At the time, doctors still thought disease was due to imbalances in the humors or to miasmas, and Semmelweis' results didn't fit into the established medical framework of the times, except perhaps in England, where contagion theory was fairly well accepted. One of Semmelweis' primary opponents, for example, was convinced that puerperal fever was due to uncleanliness of the bowels and recommendedâof courseâpurges as a preventive measure.
Sometimes, it's also exaggerated just how rejected Semmelweis was. There were groups of physicians, even prominent physicians, who accepted his results and agreed that hand washing could lower the incidence of puerperal fever. Even if they didn't understand how hand washing could accomplish such an amazing decrease in the death rat, they couldn't argue with the data. They're often not mentioned, or they're downplayed.
The Wikipedia entry on Semmelweis is actually quite good in explaining this aspect:
Only belatedly did his observational evidence gain wide acceptance; more than twenty years later, Louis Pasteur's work offered a theoretical explanation for Semmelweis's observationsâthe germ theory of disease. As such, the Semmelweis story is often used in university courses with epistemology content, e.g. philosophy of science coursesâdemonstrating the virtues of empiricism or positivism and providing a historical account of which types of knowledge count as scientific (and thus accepted) knowledge, and which do not. It has been seen as an irony that Semmelweis's critics considered themselves positivists, but even positivism suffers problems in the face of theories which seem magical or superstitious, such as the idea that "corpse particles" might turn a person into a corpse, with no causal mechanism being stipulated, after a simple contact. They could not accept Semmelweis' ideas of minuscule and largely invisible amounts of decaying organic matter as a cause of every case of childbed feverâ ideas which in the absence of a replicative biological mechanism, must have seemed no more chemically likely than homeopathy. To his contemporaries, Semmelweis seemed to be reverting to the speculative theories of earlier decades that were so repugnant to his positivist contemporaries.:45
And in another article:
Today it is well known that Semmelweis was wrong about the theory of cadaveric contamination. What Semmelweis did not know is that chlorinated lime not only destroys the stench on contaminated hands, but also the bacteria there â the germ theory of disease had yet to be discovered. Many of the epidemics of childbed fever were probably caused by streptococcus infectionsâ either type A, which is commonly found in the throat and nasopharynx of otherwise healthy carriers, or type B, which lives on the skin. Type B is also found in the female genitals of about 5-30% of pregnant women. It is therefore necessary for the physician to disinfect their hands before every examination and not, as Semmelweis thought, only after visits to the morgue.
The difference, of course, is that if someone showed me evidence that a homeopathic remedy could have as dramatic an effect that is as compelling as Semmelweis' evidence was for hand washing and puerperal fever, I'd start to reconsider my assessment of homeopathy.
The bottom line is that the Semmelweis story, as is true for so many historical stories, was far more complicated than it's often represented. The core of it is still largely true, though, namely the part about how physicians have a tendency to resist change and evidence.
"The first type of criticism involves the philoso- phical underpinnings of EBM, which is based on empiricism. In its rawest form, EBM elevates experimental evidence to primary importance over pathophysiological and other forms of knowledge"
"Other forms of knowledge"? This reminds me of when homeopaths and other wooists deride research trials in favor of "other ways of knowing".
I wouldn't be surprised if The Angry Orthopod* had lots of company in the woo crowd in denouncing checklists and recommended/mandated diagnostic tests. Not only do these degrade "physician autonomy", they don't fit comfortably into a "treat the patient, not the disease" paradigm where workups and therapy are individualized according to the practitioner's unscientific convictions.
*when I picture an angry orthopod, it makes me want to get out the slug and snail bait. :)
AO's attitude strikes me as comparable to everyone's favorite pediatrician to the stars, Dr Jay Gordon. His whims around vaccine schedule trump all evidence. He has a feeling based on years of experience (and a few endorsements no doubt).
Dr Jay seems unable to evaluate risk/benefit curves for vaccines and rather than admit he might have slept through the stats class, he makes shit up. Same bad and potentially dangerous attitude as AO exhibits.
AO's biggest strawman is in implying that EBM is being rigidly applied with no regard for clinical judgement. That this is completely untrue is illustrated by this article (http://www.independent.co.uk/opinion/commentators/michael-rawlins-stati…) by Michael Rawlins, head of NICE.
What is an emerging NHS problem is that managers are abusing EBM to produce illness management protocols which are then applied by non medical staff to do the doctor's job on the cheap.
Is AO *already* a woo? Or definitely "on the path" ** to becoming one? So much of the nonsense I survey involves this theme: whether they're talking about vaccination, cancer treatment, or SMI, they are wont to ridicule the idea of rules and SB data-based information-"accepting the consensus is for followers, not leaders"- they only like rules or data when they make them up for themselves. They urge their followers to think for themselves and then tell them what to think: "Rebel against Authoritarianism: follow me!"
In a similar vein, AO uses unusually vivid figures of speech: "castration","lambs to the slaughter"? Really! This type of belongs-in-the-trashcan prose would be right at home @ NaturalNews or the ProgressiveRadioNetwork. Perhaps by keeping an eye on his effusions we might be able to witness first-hand his metamorphosis into woo-meistery.
** Pink Floyd reference
I like the connection w/ Semmelweis, Orac.
Nice timing on the article, Orac. The story of Semmelweiss came up on a recent Skeptics' Guide to the Universe (episode 326). The segment "This Day in Skepticism" was about World Handwashing Day. Semmelweiss' discovery (and attitude) were discussed.
A few observations:
1) Another example of the value of evidence is the story of Dr. John Snow and Rev. Henry Whitehead and the Broad St. pump in the 1854 London cholera epidemic. Disease was attributed to miasmas or the constitutional weakness of the lower classes. Absent the germ theory, they had to pile up a towering mass of evidence before they could get the pump shut down. It was the first major epidemiological investigation and everyone in medicine since has wanted to be the next John Snow, having the revolutionary insight that everyone else had missed since the dawn of time.
2) Orthopedics is possibly the most purely surgical of all the surgical disciplines. Surgeons see their principal work as over the patient on the table, less as practicing medicine (in terms of the divide between physicians and surgeons). After residency and fellowship, no one stands across the table from them and tells them what to do. They must rely on their own skill, judgment, and sometimes audacity in the OR. It is no wonder that surgeons, and especially an orthopedist, sometimes reject the idea that anyone can tell them anything. You may notice that the more a surgical specialty is perceived as requiring a higher degree of skill, the greater the prevalence of arrogant and self-centered in it. That is why there are so many (NOT most!) plastic surgeons, cardiothoracic surgeons and neurosurgeons, even though at least the latter two are heavily dependent on science and EBM.
AO's rant on cook books and cookie cutters misses one important point: The reason we have them is they work. They tell us that eggs go with salt and pancakes with syrup, and prevent us to have to suffer through raspberry jam omelet and asparagus pancakes. So those aren't as bad as homeopathy and reiki, at least there's some remaining nutritional value.
A raspberry omelet sounds ... interesting. (makes shopping list for ride home)
Because somebody has to post it: http://xkcd.com/720/
Aside from the fact that doctors who believe double-blind studies will castrate them have a *very* inadequate understanding of human physiology, there's some truth that more scientific knowledge harms the aura of medicine, and makes it more cut-and-dried. But that's a good thing.
Think of how many lives have been saved by polio and smallpox vaccines that can be administered by technicians without even nursing degrees. The ideal would be, in theory, that any illness short of extreme old age could be treated with over-the-counter remedies. Of course, physicians would lose most of their "aura"-- but they, along with the rest of us, would lead healthier lives.
I would say that just about any professional with a hard-earned credential doesn't like being told what to do by uncredentialed persons. When the directions come from within the profession, it is more complex. A true professional thinks for himself (or herself), and should be able to understand the reasoning behind the conventional wisdom of the profession, and also, in some cases, when to move beyond the limits of the conventional wisdom. Otherwise, how could progress be made?
Progress ought to be made within accepted standards, however. When lives are at stake, which is the case for professions other than medicine, progress ought be made carefully.
The way I see it, examples of "cookbook medicine" have either been empirically established to improve patient outcomes (in which case doctors can't ethically NOT use it) or they haven't (in which case it's not EBM/SBM).
AO is therefore putting his own ego above the well-being of his patients. Nice.
Oliver Wendell Holmes, Sr. (the doctor and poet) published "The Contagiousness of Puerperal Fever" in The New England Quarterly Journal of Medicine and Surgery in 1843. He also described homeopathy as "the pretended science."
In other news, there are 26 cases of whooping cough in McHenry County, Illinois. I haven't been able to find any information on the vaccination rate.
I found this with a link from Ben Goldacre's Bad Science blog:
"Testing Treatments" is a book for both the lay public and medical practitioners on the science behind medical studies and how patients and practitioners can work together to provide the best care.
This is Goldacre's page:
Ben Goldacre wrote the forward and reproduces it in his blog.
Whose autonomy is involved here? Not, as far as I can tell, the patient's. If I am seeking medical care, I may well have information or opinions about what I want done. Those opinions might be derived from evidence-based medicine (for example, I want the surgeon to wash his or her hands). They might be from personal experience (I have a friend who knows she reacts idiosyncratically to anesthesia, for example).
But if we're talking about autonomy, I'll take the doctor who says "here are the possible treatments, and the advantages and disadvantages of them" (side effects, time to recover, chances of success...) over the one who is sure that he is The Doctor and doesn't need other doctors' opinions, or mine.
The places where the patient's input is irrelevant are likely to be the cookie-cutter ones, not things where large amounts of clinical judgment is called for. (I at least hope that a doctor is willing to take outside information on which antibiotics work well for which problems.)
Layperson here: Wouldn't it make it easier to have a "cookbook" type of treatment for the stuff that's common which would hopefully leave you time to work on the uncommon and provide individual care? Sorry if this is a stupid question. I thought there already kinda was a "book" of diseases, symptoms, etc that was used. I don't see how an evidence book would affect your autonomy. I see it as not having to reinvent the wheel each time. I don't get it. Evidence based medicine sounds more like...freedom. Thoughts and criticsm?
@MikeMa: Jam in omelets was an actual thing in the 1950s - I have some old magazines that provided the recipe for such things, more than once. I think it was a Smucker's conspiracy to make us reject Communism, or something, because the idea of strawberry jam in an omelet makes me want to egest with alacrity.
I am terrified of doctors who think they know best and don't need anyone "telling them what to do" - such doctors have told me to "not worry so much" when I've had bouts of illness, and refused pain meds while telling me "it's all in your head" without even consulting an x-ray. I'm not a fan of the "by the gut" style of doctoring.
How does that old joke go - "No, Dr. Smith does not have a God complex. God has a Dr. Smith complex."
If you look at the Angry Orthopod's blog, you will see him bashing a couple of treatments for plantar fascitis for lack of evidence.
The thing that our colleagues continue to miss about protocols and safety practices is that these constitute a floor not a ceiling for care.
I once saw a Powerpoint presentation called The Value of Engineering. It was a modification of a Powerpoint prestentation about the by the president of Petrobras (Brazilian National Oil COmpany) on the design and construction of the world's largest floating oil production platform. how they had saved money and created a win-win situation for the company and it's suppliers by not being bound by conventional engineering practice in the design and construction of the platform. The orignal backgrounds of the slides had been replaced by pictures of the plaform listng, capsizing, turning turtle and finally sinking.
There's an old adage: The race does not always go to the swiftest, nor the fight to the strongest, but that's the way to bet.
The point being that the way to maximize outcome in the long run is to always choose the mostly likely result, even knowing that you are going to fail some of the time. Selection strategies that try to outsmart the probability are ultimately less successful (the classic example is to try to predict whether a die will land on 1 - 4 or 5 -6; the maximum success rate you can have is to select 1-4 on every roll, because that is the most likely outcome (2/3 correct); too many people think that it's better to mix up your guesses making 2/3 of them 1-4 and 1/3 of them 5-6. However, that will end up being correct only 5/9 of the time; granted, it is the best approach to get larger short term gains (which is why it is a common gambling activity) but it also risks having larger short term losses, and, as noted, is a less effective long term approach)
So back to medicine: the challenge in medicine is to try to determine in which statistical sub-group any individual patient should be assigned. Once you can characterize them thusly, you can choose the best therapy/approach for that sub-group.
What AO is suggesting is that his super doctoring knowledge allows him to sub-categorize patients beyond that discrimination available in any EBM trial, and that he can tell which of the patients within the group are the exceptions to the most likely outcome. However, the problem with that approach is that if you do it, you damn well better be right, because as our example above shows, if you aren't, you will compromise your effectiveness. Is he right? I can't say, but I can ask a few questions:
1) do his exceptions agree with what other doctors would say? If 5 doctors looked at the same sample, would they come up with the same 5 exceptions? If not, it means that someone is wrong, and who is to say it isn't AO? Overall, 4 doctors guessing wrong and 1 doctor guessing right is not going to be a successful approach.
2) Keep an eye on quality control: does he view a disproportionate number of cases as "exceptions"? For example, if a RCT finds that 75% of cases are successfully treated with X, but the doctor is treating 50% of the cases with Y because of "special circumstances" then you know he is inventing them. The doctor needs to have a large number of "this is a classic case of G, and the first thing to do is to try X"; and I think that exceptions are best made sparingly, in the most obvious cases. In the example above, even if the doctor is only calling 25% of the cases exceptions, that is probably way too high, incorrectly diagnosing a large fraction.
But I think my point 1 above is the most poignant. Doctors all the time try to think that they have special knowledge due to the individual circumstances. However, I don't think that works near as often as they think. As such, the best approach is to bet the favorite. Sure, you know you are going to miss the exceptions, but if you start shooting from the hip, you miss the exceptions AND you miss the most common.
AO sounds like the epitome of the old orthopod joke.
What does it take to be an orthopedic surgeon?
You have to be strong as an ox, and half as smart.
Does the Angry Orthopod write for the Journal of American Physicians and Surgeons?
Now, if only I could get my patients to be more accepting of EBM, because they do NOT like the idea that they conform to norms and statistical patterns. I have more patients who insist that they simply do not respond to medications (or other therapies, like Physical Therapy, or like symptomatic treatment rather than antibiotics for viral infections) in any statistically-predicted way, than I have patients who understand that they probably do fall well within the typical distribution. Why do they want to think that their medical care is a Top Chef individualized masterpiece of asparagus and raspberry panna cotta?
Here are two more orthopedist jokes.
Q. What does the "MD" after an orthopod's name stand for?
A. Me Doctor!
The all-purpose orthopedic surgical note:
"Bone bad. Me get knife. Me get hammer. Me fix bone."
His disdain for cookbooks makes me think of how badly that can go in the culinary world. Yes, a good chef can make a smashing meal with no recipe -- but few chefs are foolish enough to do so for paying customers. You want a recipe that's tested. Even if you developed it yourself and now have it memorized so that you don't have to actually have the book open, you want to know what you're doing and not just wing it. A good chef *can* wing it, but the results will be unpredictable. And for certain things, likely disastrous. You can improvise a stir-fry (though the basics, like what temperature to cook the chicken at, are standardized for good reasons) but you can't improvise a cake. Oh, you can make slight mods, but the basic chemistry of most cakes is sensitive enough that if you get too creative, you'll have a disaster.
Nowhere have I seen this more ably demonstrated than on a legendary episode of one of the Food Network's challenge shows. There is a cake "artist" who disdains convention and builds things her way. She does still use recipes to make the cake itself (which may be unfortunate, as she might do less damage if the cake had no structural integrity) but then she wings it from there. This particular cake will blow your mind, and prove that being a maverick isn't always a good thing.
(I saw the full episode too, and it's staggering. She is utterly oblivious to her lack of ability.)
Your mentioning of the NCCN brings up an issue that's important to me as someone who works in health care but not in medicine.
Evidence-based guidelines (we've been using them here for years and they help a lot) also makes life MUCH easier for health researchers. If we know that an "average" _____-cancer patient receives either X, Y or Z (as opposed to the whole alphabet), we can more easily make predictions about how best to allocate resources and help the population at large. While the guidelines should never be crafted in order to make my job easier, the fact is that they do - a lot.
Insofar as me being good at my job helps the health care system as a whole, guidelines have a "knock-on" effect beyond individual patient interactions.
The other thing is to know where you are and aren't exceptional: if a patient knows that X, Y, and Z migraine treatments don't work, because she's tried them, she shouldn't have to go through that again because her insurance changes. But it doesn't follow that she won't respond to the most common treatment for an infection.
I know someone who tends to be very sensitive to the "side effects" of medication. That doesn't mean her doctor won't try her on new treatments. It means that if the recommendation is to start at between 10 and 40 mg of something, she'll start at 10. (I put "side effects" in quotes because they're all drug effects, it's just a matter of what you're trying to do and focusing on. Consider the people who take advantage of "side effects" of OTC antihistamines to help them sleep.)
@Old Rockin' Dave
Thanks, I hadn't heard those. Very funny.
I have spent a great deal of my career in the O.R. with Orthopedic surgeons. Many of have been smarter than the average bear and quite pleasant to work with. Unfortunately there are more than a few who seem to fit all the stereotypes.
Sceptinurse, the whole issue of the so-called "surgical personality" is interesting and complex. After working for a year with the sadistic, sarcastic and sometimes racist surgeons at a major New York hospital, I was amazed that at the next major NYC hospital I worked at that the surgeons one and all that I met were kind, thoughtful and decent. The asshole surgeons (I don't mean colorectal surgeons!) that I have met most often seem to cluster around neuro, cardiothoracic, vascular and plastics. Most in those areas are pleasant and decent, but a large minority behave like stereotypical spoiled rock stars both inside and outside the hospital, with the same outsize egos, the same petulance, the same tendency to smash things up, the same contempt for everyone not in their "band", the same casting of blame for their own failures. Often, their patient handling skills are slick to the point of being smarmy,but they can be supremely nasty to the "little people". They may be brilliant in the OR, but they excel not because they care much about the patient, but because their egos demand that they be acclaimed the best. Some of this is because department heads seem to select house staff that resemble them in character and some because these specialties are attractive to this kind of person. Oddly, ophthalmic surgeons have to have the same level of finesse, but the ones I have met tended to be quiet and even humble. I'm not sure why this should be.
Actually sweet omelets are an old tradition, and perfectly fine as long as you don't add too much salt or bacon or something stupid. Ever had a sweet souffle for dessert? Or a classic chocolate mousse? Same ingredients - egg, sugar, flavour. It's quite a good option for those who can't have french toast or pancakes because of gluten issues.
Example: raspberry souffle omelet.
@ Calli Arcale: I tried the link to your "cake episode" and this is what came up:
This video contains content from UMPG Publishing and EMI, one or more of whom have blocked it in your country on copyright grounds.
Sorry about that.
I was thinking about "cookbook" style meal preparation versus baking even before you posted...and everything you state is true.
Just one more orthopedic joke:
Q. What do you call two orthopedics reading an EKG tracing?
A. A Double blinded study.
Enough of these anti-anti-communist attacks on jam omelets.
From Strong Poison by Dorothy L Sayers (London: Victor Gollancz, 1930) [may or may not contain spoilers]:
[The hanging judge is summing up] âThe final course was a sweet omelette, which was made at the table in a chafing-dish by Philip Boyes himself. ... Four eggs were brought to the table in their shells, and Mr. Urquhart broke them one by one into a bowl, adding sugar from a sifter. ... Philip Boyes then beat the eggs and sugar together, cooked the omelette in the chafing-dish, filled it with hot jam... and then himself divided it into two portions, giving one to Mr. Urquhart and taking the remainder himself.
âI have been a little careful to remind you of all these things, to show that we have good proof that every dish served at dinner was partaken of by two people at least, and in most cases by four."
This jam omelet[te] for dessert raised no eyebrows, in 1930 England. It goes without saying that this omelet, made from eggs broken right at the table and immediately cooked with sugar and jam (both of which happened to be available for later analysis) and consumed by both people at the table had no connection whatever with the death of Mr. Phillip Boyes (who had cooked the thing) from arsenic poisoning, the first symptoms of which began to appear an hour or so after the end of dinner.
Sadly this fellow seems to represent the classic 20th century surgeon.
The use of patient safety techniques and culture change that includes the use of checklists, similar to aviation and nuclear industry safety programs, is a separate and significant error mitigation system that really doesn't belong lumped with Evidence Based Medicine.
Evidence Based Medicine, while both theoretically and sometimes practically is laudable, is perceived by some of us as dangerously close to being taken over by those with motivations to control physicians to either control costs or direct profit. This orthopod seems from another planet, but a skeptic like myself does not desire to be lumped in the same category because I am wary of the EBM movement.
Take the screening mammogram controversy between the ACS and the USPSTP (or whatever, its too late for me to remember all the letters in the right order). Both organizations feel they have significant evidence to back their recommendations. Is one right and the other wrong? How can we reconcile this kind of problem in EBM without watering it down?
Is there any evidence that EBM has better patient outcomes than alternative methods? Even if there was, you are going to start running into the second incompleteness theorem that states a system cannot demonstrate its own consistency. Perhaps doctors don't take enough math courses in school...
Is there any evidence that EBM has better patient outcomes than alternative methods?
Yes. If alternative methods had the evidence to support them they'd be EBM.
Even if there was, you are going to start running into the second incompleteness theorem that states a system cannot demonstrate its own consistency. Perhaps doctors don't take enough math courses in school...
I find it amazing how many people throw around that theorem in contexts where it's completely inappropriate. GÃ¶del's theorem is talking about formal systems, not everything that humans call systems. Even if the theorem was applicable in this context, it would be just as applicable to the jumbled grab-bag "system" of alternative medicine as to EBM, so anyone trying to use it as an argument for the superiority of alternative methods is frankly acting kind of stupid.
lilady -- aw, they took it down? That's a shame. It was hilarious. I'll summarize. There was a competition where four teams would bake birthday cakes for a mystery judge, who was at the last minute revealed to be the sternest, meanest judge they have on that show. (And who, it transpired, had never had anyone bake a birthday cake for her before. So it ended up being pretty special for her.) One team had its leader suffer a heart attack and be taken off to the hospital; his assistant stepped up admirably into the lead role while a bakery-experienced audience member stepped in to assist him -- they ended up taking second place with a stunning, hand-painted cake. The winner was bold enough to attempt sugar lace in a time-limited context like this (they had four hours) and it came out gorgeous. It was a very close call between the two leads. The third place effort was beautifully done, but misjudged the tastes of the judge; they were supposed to make something themed particularly for her and they all got a chance to interview her as if she were a normal client.
And then there was the last place one. She was on a completely different planet from the others. Her kitchen was a mess -- chocolate and syrup spilled all over, flour and sugar dusting everything -- and her "concept" was a sheet cake on its side. That's apparently her major artistic motif -- doing things sideways. (She's like the Ed Woods of edible art -- completely oblivious to her own ineptitude.) The cake was made by taking lots of loaf-shaped cakes and assembling them into the sideways layer cake structure as if they were bricks. But due to her sloppy methods, they didn't have enough cake to meet the height minimum for the challenge. So she improvised by putting strawberries on skewers and putting that on top. Then the cake was covered by wrapping it in edible paper glued on with frosting. And yes, that looked horrible, because her application methods are not remotely neat. Then she and her assistant drizzled melted chocolate and melted sugar over it to "decorate" it. The piece de resistance was her innovative substitute for birthday candles -- bowls of ethanol placed among the jagged edges of the cake and then lit on fire. Since she'd covered the cake not in frosting but in flammable edible paper, this shortly led to the cake itself catching on fire. A member of the studio crew swept in with a fire extinguisher and put it out. And that's how it looked when judging time came around -- saggy and covered in fire extinguisher foam. The judge was nearly speechless, which is rather remarkable for her, as she's sort of the Simon Cowell of the cake world. One of the other judges managed to find something nice to say, and the baker became convinced that he "gets it" when in fact it was obvious he was only trying to be polite. She has a picture of the cake, drenched in foam, on her website -- yes, she is still proud of it, despite how it ended up.
Gödel showed that if a formal system is sufficiently complicated, it cannot be used to prove itself complete and consistent.
This is interesting but not relevant, unless you know of someone who is claiming that medicine is a complete, consistent formal system. Completeness might be a goal, but no doctor is going to claim that it has been achieved. We're not even close. If medicine were a complete system, we would have cures for cancer, heart disease, diabetes, malaria, and many other things that now we can, at best, treat.
See also what Antaeus said about formal systems and what they aren't. Furthermore, that a system cannot be used to prove its own consistency and completeness doesn't mean that it isn't useful: ordinary arithmetic is complex enough that Gödel's theorem applies. That doesn't mean it isn't valid and useful. It means that somewhere, there is at least one true statement within arithmetic that cannot be proven using the rules of arithmetic. If this applied to medicine, which it doesn't, it might just mean that there was some unknown plant that could be medically useful, or some way that a surgical technique could be further refined, or that we don't know quite the optimal dosage of aspirin as a preventive for a 47-year-old woman who has never given birth and has a family history of heart disease.
"Yes. If alternative methods had the evidence to support them they'd be EBM."
Any links? I'd enjoy deconstructing the circular logic.
"I find it amazing how many people throw around that theorem in contexts where it's completely inappropriate.
GÃ¶del's theorem is talking about formal systems, not everything that humans call systems."
So mathematics doesn't apply to medicine but medicine is science? That has to be the most ignorant thing I've read today. Congratulations, we have a winner!
"Yes. If alternative methods had the evidence to support them they'd be EBM."Any links? I'd enjoy deconstructing the circular logic.
The logic's not circular, merely pragmatic. If methods you consider "alternative" had really good evidence behind it, they'd be evidence based - and thus evidence based medicine.
That might be slightly facile, ignoring the tendency of people to reject or accept things as "true" based on their preconceptions, but I think it's basically stating the definition of EBM.
And, yes, it's the responsibility of someone who suggests that a method works to provide evidence.
So mathematics doesn't apply to medicine but medicine is science?
This is clearly arguing against a position no one has taken. Mathematics certainly applies to medicine, and medicine is certainly (at least in part) science - but don't kid yourself that all of medicine is defined mathematically.
If you have data to show that GÃ¶del's incompleteness theorem applies here, or that anyone has stated that science based medicine is a complete formal system, please share.
This is interesting but not relevant, unless you know of someone who is claiming that medicine is a complete, consistent formal system.
IF MEDICINE ISN'T ABOUT SOLVING THE "HALTING PROBLEM," WHAT IS?
Yes. If alternative methods had the evidence to support them they'd be EBM.
Any links? I'd enjoy deconstructing the circular logic.
There's a difference between circular and tautological logic, and the only reason the tautology had to be spelled out is that you didn't get it. Would you like to explain how alternative medicine can "show evidence of better patient outcomes" (and safety, needless to say) without becoming evidence-based medicine?
I find it amazing how many people throw around that theorem in contexts where it's completely inappropriate.
GÃ¶del's theorem is talking about formal systems, not everything that humans call systems.
So mathematics doesn't apply to medicine but medicine is science? That has to be the most ignorant thing I've read today. Congratulations, we have a winner!
Pfahahaha! Wow, you really play in amateur leagues if you really expect that crap to fool anyone here. No one said "mathematics doesn't apply to medicine." What I might have said before, and will say now just so there's no room for doubt, is "The particular mathematical theorem you are trying to throw around to impress people only applies to particular specialized systems called 'formal systems,' almost all of which are purely mathematical; since medicine is not a formal system or a mathematical system, that mathematical theorem cannot be applied to it." Trying to do is really quite frankly idiotic, like those creationists who try to argue that the Second Law of Thermodynamics means Earth life cannot be increasing in complexity; they are trying to apply a law which is stated to apply to closed systems to the open system of the Earth, which is continually receiving energy from the Sun.
But hey! If you really think that you can bluff your way through this just by throwing around impressive phrases about "incompleteness", why, then, go ahead! As you know, having taken so many math courses and all, GÃ¶del's incompletness theorem is based upon the calculation of GÃ¶del numbers for every legitimate member of the formal system in question! So why don't you just tell us the GÃ¶del number of an appendectomy? Or of neomycin? Aww, hell, I'm feeling generous; you can pick any procedure, from mainstream medicine or even from your favorite alt-medicine woo! Just tell us what the GÃ¶del number of the procedure is, and explain in detail how you calculated it! That should be a snap for you, right? I for one can't wait to see you demonstrate your stunning grasp of advanced mathematics; if you can calculate the GÃ¶del number of a medical procedure, you can surely follow it up with the square root of the electromagnetic weak force! Dazzle us, Einstein!
"This is clearly arguing against a position no one has taken."
Oh no? If you are going to base a system of medicine on evidence, then you better be able to prove your evidence is correct, and that it results in better outcomes. How do you do that without math?
"Mathematics certainly applies to medicine, and medicine is certainly (at least in part) science - but don't kid yourself that all of medicine is defined mathematically."
I don't. But the blog author would likely argue that point. If its not defined mathematically, how is it defined?
"If you have data to show that GÃ¶del's incompleteness theorem applies here, or that anyone has stated that science based medicine is a complete formal system, please share."
That is letter to the editor. As far as we know it could be an article on Godel's fear of being poisoned. Do you have something a bit more substantial?
Oh no? If you are going to base a system of medicine on evidence, then you better be able to prove your evidence is correct, and that it results in better outcomes. How do you do that without math?
And where did anyone say anything against using math in medicine? It is done all the time. Often the statistics used to determine the efficacy of a procedure.
What are you trying to argue? What does it have to do with the Angry Orthopod's opinion?
As far as we know it could be an article on Godel's fear of being poisoned.
I don't have Lancet access, but the cross-references suggest that it's of this flavor.
Ah, I see. Something that could be mistaken for a Sokal Hoax. The following paragraph reveals the author does not know what he is writing about:
Then Albert Einstein (1879â1955) published his first theory of relativity in 1905, which aroused considerable controversy. In 1919, crucial experiments confirmed that Einstein was right and Newton was wrong.
Actually Newton was not wrong, the ideas had to be tweaked at certain extremes. When I analyzed structures for vibration and dynamic stability the velocies, masses, forces, etc. were all in the range where Newtonian physics still worked.
With all due respect (I love your blog), it appears that your were cherry-picking certain comments by the Angry Orthopod to make a point that some doctors are resistant to EBM. The problem is, I think you missed the thrust of what he was trying to say. He was not knocking EBM (he pointed this out several times), but was instead voicing concerns over the prospect of EBM dictating health care policy. This actually sounds like a good idea on the surface, but I believe he was voicing mistrust on *who* decides this (big government, cough cough). As anyone who routinely reviews the available literature in their field knows, there is an enormous amount of chaff, and very little wheat. When should "EBM" dictate policy? When is the evidence good enough (outside of a good meta-anaylsis)? Who decides this? These are good questions.
On a side note, orthopaedics jokes aside, for the ignorant, orthopaedic surgery is one of the toughest residencies to get into. You've got to be among the top of your class, or you don't have a chance. Contrary to what some think, orthopaedic surgeons are pretty freaking far from dumb. And the reason they don't know how to read an EKG is the same reason the internist doesn't know how to diagnose compartment syndrome, understand the indications for washing out a painful joint, read and interpret a musculoskeletal MRI, et cetera. While we all learn quite a bit in medical school, if you don't routinely use the information you learn, you lose it. You retain what you need to practice effectively in your specialty. Every specialist who is guilty of snickering about some other specialist not knowing something forgets this (orthopods included).
@ dan: If you took note of the comments (orthopedist jokes included), most were made by the "regulars" here. Many of us are doctors and nurses and we all practice medicine and nursing that is evidence-based.
I believe you were referring to me who joined in the fun by the telling of the EKG tracing...double blind study joke.
Dan, if you have been following this blog for any length of time, you would realize that I take particular delight in engaging chiropractors who think they are in any way qualified to treat orthopedic problems.
I believe you would find that I posted in the past, about my disabled child who was treated superbly by a pediatric orthopedist for a supracondylar femur fracture. He underwent a closed reduction in the operating room and was in a hip spica cast for 10 weeks. He recovered in a hospital bed set up in my dining room.
Here's a good nurse's joke (I am a recently retire R.N.)
Interns think of God. Residents pray to God. Doctors talk to God and nurses are God.
See if you can come up with a better one.
BTW, I don't believe Orac "cherry-picked" statements for this article.