In 2011, 10 years after this practice began and 6 years after the stent received FDA approval, the first robust study of its efficacy was published. This study is referred to as the SAMMPRIS study. (SAMMPRIS stands for “stenting and aggressive medical management for preventing recurrent stroke in intracranial stenosis”—not one of the more elegant acronyms.) In the study, a group of patients similar to Anita was randomized to treatment with all of the medicines we mentioned above or to treatment with these medications and insertion of a stent. Three years after the trial began, an independent review committee stopped it because one group of patients was doing much worse than the other. Unfortunately, the group that had received the stents was having more strokes.
After an average follow-up of one year, 20 percent of the patients in the stent group had strokes, compared with only 12 percent of those in the medicine-only group. That is an increased absolute risk of 8 percentage points, or, in news-report language, a 64 percent increase in the rate of stroke. For every 13 patients who got a stent, 1 of them had a stroke because of the stent alone. Stenting intracranial arteries is a failure and a perfect example of reversal. A procedure that was accepted and deployed was overturned by solid data.
Anita Kramer is disabled, and her stent may be to blame. For one patient, we can never tease apart the gossamer strands of causality (we get into this more in chapter 10), but on average, for patients like Anita, we know it is true. Many were harmed by the stenting.
THE FIRST HARM
Step back for a minute, and consider the case of intracranial stenting. A major, costly intervention for stroke prevention, which was done for a decade and paid for by many insurers, actually increased the risk of stroke.* Moreover, the cost of the therapy was not just the stents. There were the payments to the doctors who placed them. There was also the cost of the brain MRI scans, often done just to look for narrowed arteries in the brain. The only reason these scans were done was to find these stenoses, so some patients could get stents, which did not work. We cannot say how much all this cost, but it was not cheap.
The SAMMPRIS finding exemplifies the first harm of medical reversal. The patients who received the treatment during the years it was in favor were harmed. Anita may be worse off because of the treatment she received. In cases where the treatments did not do harm but simply did not work, patients wasted their valuable time and money. Remember vertebroplasty, the injection of medical cement into fractured bone? A few patients were hurt by this procedure, but most simply had their time and money wasted. And it was not just patients’ money that was wasted—through Medicare, Medicaid, and other insurances, we all had to pay. The first harm of medical reversal is harm to the patients who undergo the therapy when it is popular and to everyone who has to pay for it.
THE SECOND HARM
The second harm is even more troublesome. One would think that a major study like SAMMPRIS, one that is well done and published in the New England Journal of Medicine, would immediately put a halt to the practice. Medical practices, however, are like freight trains—they do not stop quickly. Let us take a look at some of the responses to the article that overturned intracranial stenting. First from Stryker, the maker of the stents, in an article from USA Today: “Stryker continues to support the Wingspan Stent System as an FDA-approved Humanitarian Use Device for improving cerebral artery lumen diameter in patients with intracranial atherosclerotic disease who have failed medical therapy.” Comments like this, after a well-done, negative study, are hard to understand. Making medical devices is not like being at the helm of a ship. If it sinks, the captain does not have to stay on board. (Maybe the metaphor makes more sense if the ship is full of gold, and the captain cannot bear to part with it.)
If the makers were unwilling to abandon ship because they believed that the treatment still held promise for another indication, their response could have been, “Well, it is back to the drawing board.” They could have commenced a randomized trial of the device for another use, perhaps treatment of a subset of strokes in a subset of patients. In this manner, the SAMMPRIS study might have marked the start of a successful therapy rather than the end of a flawed one.
One might question whether one negative study is enough to overturn a practice. If the data that exist in support of the therapy are not strong, and the trial that overturns it is, one study should be enough. (We delve much deeper into what makes data strong and weak in the coming chapters.)
It was not just the device’s maker who stood by the product. One of the study’s authors noted, “There may be a place for stenting in patients who don’t respond well to medical therapy alone.” And another prominent researcher noted, “The findings will likely greatly reduce the number of these procedures.” There may be a place? Greatly reduce?
This is the second harm of reversal. The contradicted practice does not stop immediately. It continues for years to come. Future patients continue to be harmed by the practice. Although it is terrible that the therapy we gave to Anita might have caused harm, it would be even worse if a similar patient is treated the same way and has the same outcome one year after SAMMPRIS.
We see this pattern, a refusal to abandon a practice that does not work, repeatedly. People with stable angina continue to receive coronary stents years after the practice was overturned by the COURAGE trial (discussed in chapter 1). It is like telling a child not to play with matches, turning around for two seconds, and hearing a match being struck.
Bias in the interpretation of the results of medical trials may play a role here. In a clever follow-up study to COURAGE, researchers reviewed all of the articles that cited the COURAGE trial. Some of those articles agreed with COURAGE, while others argued against the conclusion that stents are no better than traditional medical therapy. Not surprisingly, the articles that took issue with the conclusions were more likely to have been penned by interventional cardiologists (the doctors who place the stents) than were those that were supportive of the findings.
In the case of vertebroplasty, another procedure that we discussed in chapter 1, one doctor wrote, in an accompanying editorial, “When best evidence suggests a toss-up between treatment options and no benefit, informed patient choice is essential.” Toss-up? When a treatment is no better than a sham or placebo procedure, that is not a toss-up—that is a failure. We should not offer patients a failed therapy. That therapy should be abandoned.
The same group of researchers who studied the articles about COURAGE also analyzed the medical literature to discover just how long it takes the medical community to abandon a practice after a therapy is proven ineffective. They tracked citations to three major practices that were found not to work: beta-carotene to prevent cancer; estrogen to prevent Alzheimer’s disease; and vitamin E to lower cardiovascular risk. They found that 10 years passed before the research community stopped referencing the flawed practice. This estimate of a decade of inertia fits with our own experience. It takes that long for the train of medical practice to stop.
Recently, two physicians published a piece in which they suggested other biases that may play a role in medicine’s slow pace of “de-adopting” practices shown not to work. They discussed optimism bias, by which people are more willing to accept and act on information that is in their favor. This bias is certainly magnified when “in their favor” includes monetary benefits. Confirmation bias, the tendency for people to more readily accept information that confirms their beliefs, is almost certainly also at work. Loss aversion is another bias with which we are all familiar. This is the bias that makes casinos profitable. In casinos, people will continue to play because they feel strongly about money that they lost and want to win it back. They are less attached to new winnings (a fact magnified by the presence of chips rather than cash). In medicine, the loss of a trusted therapy is hard to accept. These biases, combined with the strong economic interests often at work, produce the second harm of reversal.
THE THIRD HARM
The third harm of reversal may be the mos
t damaging: the loss of trust in the medical system. There are rare people who think that everything doctors do is either worthless or harmful. These folks are quick to cite any evidence that says some medical practice does not work as evidence for their worldview. We cannot condone this view, but we must acknowledge that medical reversal gives such people something to complain about—even if the conclusions they draw are mistaken.
Although in this book we argue that some medical practices are without value, we know that many others are, without a doubt, beneficial. Our professional lives are spent recommending these therapies. Most people, being both reasonable and intelligent, can be confused by all the flip-flopping in medicine. How can you predict which practices will end up like statins for heart attacks and which will end up like statins for heart failure? In this case, there are two different reasons to take the same pill. The former is beneficial, the latter an example of medical reversal. People should not need to complete medical school in order to make informed decisions regarding their own care. (As we discuss later, the average doctor, who did complete medical school, is still woefully undertrained to think about evidence.)
After the reversal of long-standing mammography screening guidelines, we saw anger directed at the medical community. Fox News published a letter from the head of a breast cancer organization with the headline “Lives Will Be Lost with Proposed Changes to Mammography Guidelines.” Journalists frequently tried to connect this reversal to the Affordable Care Act and interpreted the change as a cost-saving initiative.
Even Jerome Groopman, a thought leader in medical decision making, was irritated by changes in mammogram (and prostate screening) guidelines. In the New England Journal of Medicine, he and Pamela Hartzband wrote, “It is neither ignorant nor irrational to question the wisdom of expert recommendations that are sweeping and generic.” These authors went on to question whether the task force that changed the guidelines considered the balance between the risk of death and issues important to the quality of life.
We include these arguments here not to question their validity—they are important concerns—but to point out how reversal undermines trust in medicine and makes people just plain angry. Patients are not the only ones who are frustrated; so are their doctors and leaders in the field.
There is another type of harm associated with reversal that we have not mentioned because it is more debatable. This harm occurs when therapies known to offer benefit are withheld because of unfounded concerns about their potential to cause injury. When research shows that such an intervention was safe all along, we learn that we could have been treating patients more effectively all along. Calling this a reversal is a bit of a stretch. Here we are not dealing with an ineffective or dangerous therapy being offered without proof of its benefit, but a beneficial therapy being mistakenly withheld because of concern for its safety. Future physicians learn on their first day of medical school that their first responsibility is to do no harm. Withholding a therapy, when one has doubts about its safety, is generally good medicine. One cannot blame a physician who waits until he is sure not only that the therapy is beneficial but also that it is safe.
This issue is even more complicated because there is a spectrum of how likely the beneficial therapy is to cause harm and of how severe the harm may be. As doctors, we are usually at our most cautious with pregnant women. The thought of doing harm to an unborn child is anathema. It is also not without precedent. Whether from the use of lithium, thalidomide, or diethylstilbestrol during pregnancy, medicine has a long, sad history of using medications, for the supposed good of the mother, that turn out to cause lifelong injury to the child. Withholding treatments, even those known to offer clear benefit to the mother, until they are proved indisputably to be safe is hard to question. On the other hand, withholding treatments based on tenuous physiological arguments, from patients who would likely benefit from them, is more debatable.
In the past 10 years, studies have been done that have overturned long-standing concerns that vaccinations could precipitate flares of multiple sclerosis and that oral contraceptives could precipitate flares of lupus. Another study reversed the practice of delaying spinal anesthesia until a pregnant woman’s cervix was dilated beyond 4 centimeters because of the mistaken belief that this would increase rates of cesarean section. Are these examples of good, cautious medicine, where we waited for all the data to be in before offering a therapy? Or are these examples of a unique type of reversal, where proven therapies are withheld for unfounded reasons leading to patients being harmed when they did not receive the best available treatments? This is a tough question, one we do not answer. Instead, we focus on the much more common scenario in which doctors push a therapy for years, only to discover it did not work.
The harms of reversal are threefold. When a therapy is widely deployed before it is proved to work, patients are put at risk. If the therapy turns out to be useless, or even harmful, the patients who received the therapy suffer. The harm may be physical, financial, or both. After the therapy is proved ineffective, further harm continues to be done as the therapy remains in use, supported either unwittingly or by those with vested interests—whether intellectual or financial. Lastly, reversal undermines the trust the public and doctors place in the medical system, and rightly so. In cases of medical reversal, we have put our faith in medical science even though there was no good science to put our faith in. Some extremists use medical reversal to throw the baby out with the bathwater—give up on all of traditional medicine. This is surely a mistake, but, at the same time, it is a consequence of our record of getting some major things wrong.
PART 2 AN INTERLUDE ON EVIDENCE
9 A PRIMER ON EVIDENCE-BASED MEDICINE :: WHAT IS EVIDENCE IN MEDICINE?
FOR MOST OF US, THE WORD evidence evokes scenes of our favorite cop show. Being old-school, we will go with Law and Order. “What evidence is there that the defendant committed the crime?” Sam Waterston would ask in a tone implying . . . plenty. Sometimes evidence is definitive: an eyewitness, the defendant’s DNA, a motive, a history of violence, and a confession. Usually, however, there are only bits of evidence, but a decision has to be made: Is the defendant guilty? It is seldom easy.
In medicine there are similarly difficult issues. Hypotheses are put forward, a study is done, and then a claim is made. But what is the evidence that this claim is true? Living at high latitudes increases the risk of multiple sclerosis (MS). Women who have twins have more postpartum depression than women who have single pregnancies. These are interesting findings. When these stories come on NPR, we put down our cups of coffee, shush the kids, and listen. Many scientists conduct research on these sorts of hypotheses. The truth is, though, that they interest us, as doctors, only peripherally.
You see, we are clinicians, the kind of doctor who cares for people, the kind you go to when you are sick (or when you are well and want to avoid getting sick). And being a clinician is about one thing, doing something (or knowing when to not do something): making a diagnosis (gallstones); prescribing a pill (atorvastatin); providing a prognosis (your knee injury will get better); performing a procedure (an appendectomy). Doctors have to make decisions about medical practices, the things we do for people. For the claims about MS and postpartum depression, a clinician would pose different, but related, questions. The clinician is not terribly interested in the claim that living at high latitudes causes MS but is very interested in whether taking trips to lower latitudes or adding vitamin D to the diet could decrease rates of MS. The clinician may find the question about the relationship between twins and depression intriguing, but what she really wants to know is, Should she screen for depression in mothers of twins?
Many people think the answers to the scientists’ questions and the clinicians’ questions go hand in hand. People assume that doing something about the risk for a disease will change the outcome. The truth is, the answers to the scientists’ questions and the answers to the clinicians’ questions have very little to do with each
other. Sure, risk has to exist for clinicians to even pose their questions. No one would think about moving northerners south if there is no risk associated with living in the north, but typically the risk exists no matter what. All of us have some risk of developing MS or depression. Are Swedes at such a high risk for MS that we should treat them differently? The important question here, and this is really key, is whether doing something different changes the outcome. Does screening for postpartum depression really help people feel better?
In medicine, the evidence we need to support a claim depends entirely on the type of claim being made. In the early chapters of this book we present many examples of medical practices that do not produce the desired and expected outcomes. High blood pressure is bad. It increases the risk of death. Atenolol lowers blood pressure, but it does not lower the risk of death. The evidence supporting the claim that hypertension increases your risk of death is very different from the evidence proving that using atenolol to decrease your blood pressure lowers your risk of death. Most studies you read about in the newspaper or hear about on TV and radio are about the former type of evidence. The studies in which you are most interested are about the latter type of evidence.
Perhaps now you are coming to see things from the clinician’s perspective. In science and medicine there is nearly an infinite number of interesting questions. In clinical medicine, the only relevant question is: Does this medical practice improve my patient’s survival, health, or quality of life? If the answer is no, there is no reason to prescribe this therapy. If the answer is yes, follow-up is warranted: What are the side effects or harms of the treatment? Is the cost of the practice worth the gain?* With all this in mind, let’s turn to real claims and see what types of medical evidence are used to support them. This information is critical to understanding what makes evidence strong and what makes evidence weak and to understanding the causes of many of the reversals we experience in medicine.
Ending Medical Reversal Page 11
