Ending Medical Reversal

by Vinayak K Prasad

  Receiving money and working so closely with industry is a recipe for corruption. Of course, most thought leaders are not only honest but wholly committed to the best science. The problem is that the influence of industry is subtle. A doctor who spends significant amounts of time with a pharmaceutical company may tend to give the company the benefit of the doubt with its new drug or study. The thought leader may come to believe that the industry is right—maybe that a one-month improvement in survival for a terminal disease is a big deal. A thought leader may hear so many times that “depression is a chemical imbalance” that he begins to favor pharmacotherapy over counseling for all mood disorders. It is hard to believe that thought leaders are not influenced or at all swayed by this work.

  The only way to reduce this influence is to keep experts from engaging in these relationships. There have been steps in the right direction. Some universities have set limits on the amount of outside funds that their faculty can receive in this manner. The Physician Payments Sunshine Act took effect in 2013. This act requires manufacturers of drugs and medical devices to track and report certain payments and items of value given to physicians and teaching hospitals. Researchers in the U.S. federal government, such as those at the National Institutes of Health, are forbidden to engage in these practices. This seemingly draconian rule should probably become the norm.

  WORKING WITH INDUSTRY

  Even if industry does not pay thought leaders for their time, many academic physicians still have to work with for-profit companies. New drugs are often tested across several universities, with money being transferred from companies to the hospitals to help defray the sizable costs of running a trial. To some degree this is inevitable—how can industry conduct trials without academic partnerships? Drug companies do not have direct access to patients. At the same time, such relationships can lead to perverse incentives. Universities become dependent on the substantial income they receive from running trials and thus try to increase the number of trials they conduct. These deeper ties to industry come at a cost as well as a profit. It may be challenging to question the design of trials conducted by a particular pharmaceutical company if your institution’s financial health relies on a relationship with that company.

  A first step to correct this problem would be to introduce greater transparency into the process. How much money do medical centers make from conducting industry trials? We should also ask whether there should be limits to the proportion of a department’s income that can come from industry. Should an oncology department have 30 percent of its revenue come from running trials? 50 percent? How much is too much?

  Limits such as these could have a positive effect, but the ultimate solution to the problem of industry influence would be to remove industry from designing clinical trials altogether. We have already discussed all the ways that industry involvement in trial design undermines the reliability of data. How could we get industry out of the business of running trials? Imagine an independent body charged with prioritizing trials to answer the most important clinical questions, rather than the questions whose answers might generate the greatest profits. Once a question is chosen, centers could then compete to design and conduct the trials. This process could be, in part, supported by industry fees, as the FDA currently is. Industry could also be required to provide devices and drugs free of charge as a prerequisite to testing. The members of the committee that would design trials would need to be devoid of conflicts of interest, both past and present. A patient representative could also be included. Such a rational system would remove the grip industry currently has on the academic community.

  PATTERNS OF PROMOTION AND TENURE

  Let us move from the sordid influence of industry on the academic medical center to the issue of the promotion and tenure of academic faculty. Academic physicians are promoted based on their productivity, expertise, and accomplishments. What have you added to the field? Did you discover a new protein? Did you discover a new drug? Did you run a successful (likely positive) clinical trial? These are all appropriate standards. These standards reflect a central role of the academic medical center: discovery.

  Discovery means generating a new and promising result. Discovery, however, is only part of what is important in science. There are two parts to research: discovery and replication. Replication is arguably the more important of the two. Replication means selecting a research study and verifying the results with a new sample or in a new setting. Replication is critical because a discovery can be a “true” discovery or a “false” discovery— random noise that only looked promising. The only way to differentiate a true discovery from a false one is to redo the experiment—ideally a few times. The need for replication is well understood. The FDA has historically required two randomized trials to show efficacy before approving a new drug. In 2014, the National Institutes of Health announced its commitment to replication by funding specific programs to make sure it happens.

  Much of the debate about replication and discovery concerns preclinical research—laboratory results that test hypotheses about physiology and the mechanisms of treatments. Does this new drug attach to the opioid receptor on nerve cells? Can we improve outcomes in mice that have been engineered with a mutation in the transporter of thyroid hormone? Replication at this stage is crucial, but when it comes to reversal, replication is most important when we make the jump from theory to practice. Does this new drug decrease pain? Does the new joint last longer than the old one? Does this new implantable defibrillator save lives?

  When we are trying to confirm that a treatment really works in people, we need at least two randomized trials confirming effectiveness. These trials should show benefit when you include people who look like the people who will someday use the drug. This sort of standard would prevent reversal by ensuring that a practice really works before it is used. But for this to happen, replication must be valued at least as much as discovery. Faculty need to be promoted not for discoveries that might turn out to be meaningless but for discoveries that stand the test of time—or for doing the studies that demonstrate that discoveries are valuable.

  This actually runs counter to what is often the culture in the centers of discovery. No one likes a critic. There is a quiet bias that it is better to be an innovator than a critic. If you are an assistant professor trying to develop a new web-app to monitor heart rate, you are likely to get more attention than is the assistant professor who performs studies showing that this app does not improve mortality, decrease hospitalizations, or save money. In America, we like innovators. If a computer programmer makes a new game, we think it should be brought to market as soon as possible; then the public can decide. Is this game as good as Angry Birds or as bad as Robotikill Fight Battles (you never heard of it?—for good reason!)? However, as we discussed in chapter 13, medical markets are different. No one can decide what new medical technology to adopt simply based on how it looks. If you cannot judge a book by its cover, you certainly cannot judge a medical innovation by its promise.

  This is why the culture of academic medicine needs to change. A recent article by Allan Brett about preoperative clearance gives a nice example of what this change should look like. Prior to elective surgery, many patients are referred to a special clinic for “clearance.” This process is akin to tuning up your bike before a race. The doctors who work in these clinics try to optimize a person’s health before surgery, ideally reducing complication rates. Over the years, these clinics have become dominated by guidelines, which often recommend drugs and procedures, such as cardiac stress tests, based on little or no evidence.

  The preoperative clinic is now bloated with dubious practices. Dr. Brett’s article took a stance against our current practice. It provided a detailed and thoughtful summary of what we do and what the evidence supports, and ultimately argued that most of the recommendations were flawed. It was a bold and inspired piece of work and required an impressive command of current evidence in the field. Yet, in the hierarchy of academic medicine,
it would be classified as an “opinion piece,” a status slightly higher than a case report, below observational studies, and not nearly as well respected as even a small, poorly run, before-and-after study.

  Although the best work in medicine is not always a discovery, discoveries of any quality are what are lauded and valued most in the process of promotions. Replication of results, or careful, thoughtful, unbiased thinking about the results we do have, is terribly undervalued. We need to recognize and support faculty who work slowly on “opinion pieces.” The process of thinking, debating, and arguing is usually more valuable, scientifically, than generating funds by running industry-sponsored trials. We need to empower the faculty at academic centers to be critics—in the best sense of the word. We need academic medicine to reflect the vibrant and contentious world that universities are supposed to be. The output should be less about what is new and potentially profitable and more about whether what we are doing really works.

  We have talked about how discovery is highly valued within academia and how replication and the development of reasoned opinion is under-valued. What gets even less acclaim in academics is the actual doctoring and teaching. Although most institutions proclaim a tripartite mission of patient care, education, and discovery (usually referred to as scholarship), when it comes to promotion, discovery counts more heavily than patient care and education. If you are an excellent clinician, there is no clear path to a tenured professorship. No one takes a look at your diagnostic and therapeutic acumen and gives you credit for being the doctor to whom doctors send their families. This is not entirely inappropriate. The academic medical center exists to do more than care for patients. However, the centers would not exist if they did not provide exceptional patient care. Measuring clinical excellence is difficult. There is no canonical standard for what makes a good doctor. But, at the same time, practicing evidence-based medicine— keeping up with the latest studies and erring on the side of doing only what works—could be better assessed if we were motivated to practice this way.

  USELESS RESEARCH

  Here is a statement that we are uneasy about putting on paper: There is a lot of research that gets done in medicine that is useless. This statement sounds offensive and runs contrary to everything we have been taught: any search for knowledge is good and may yield great results decades hence. Each month, however, we read dozens of articles in major journals that tell us nothing new and nothing surprising.

  For instance, hundreds of hours are spent studying the effects of exercise in observational data sets. Let us say: please stop. Should you exercise? Yes. Of course! Get out there and go for a walk, a swim, a bike ride. Cross-country ski in the winter and play basketball with friends in the summer. Stay hydrated, and don’t kill yourself. Exercise feels good and probably improves health. And, yet, each year, hundreds of researchers perform more observational studies showing that exercise is good for you. Exercise is associated with lower rates of heart disease, and diabetes, and pancreatic cancer, and whatever. One more study on the benefits of exercise among the healthy is not needed. We have reached consensus. The people who are not exercising are not waiting for one more piece of evidence to start.

  It is easy to list some of the topics that do not require more observational research. Should you eat moderately or gorge yourself? Should you eat plenty of fruits or vegetables? Is fast food bad for you? Is sleep deprivation bad? Is smoking bad? Each year, dozens of new studies prove that smoking is bad. Taxpayer money funds research to demonstrate that eating fruits and vegetables is good for you. Sadly, publishing a study saying that smoking increases your risk of pancreatic cancer is better for an academic doctor’s career than working with a patient to get her to stop smoking.

  There is one commonality among all this research that we put under the heading “useless.” In each case, a researcher tries to link something all of us already believe is good and valuable (exercise, eating more fruits and vegetables) to specific good outcomes—or something we all think is completely unredeemable (smoking), to specific outcomes we wish to avoid (pancreatic cancer or stroke). But the link between any one factor and any one outcome found in an observational study is likely very weak. Future work may very well reverse that particular claim, but the general point will likely remain correct.

  At the same time, this work diverts effort from research that could be done on the same topics that would actually provide meaningful information. For instance, exercise is great, but there are very few randomized trials showing that a specific exercise recommendation can help a specific group of patients. One of these rare gems is a trial showing that weight-lifting actually improves lymphedema symptoms for women who have had a lymph node dissection as part of the treatment for their breast cancer. What type of exercise program is best for patients with arthritis? Emphysema? These are questions best tested in randomized trials, and the valuable answers could be confidently deployed in medical practice.

  A general recommendation about how to transform useless research into useful research is to perform trials of specific interventions on specific populations. Instead of an observational study showing that smoking slows healing after wrist injuries, how about a study showing whether a particular smoking cessation strategy improves survival among patients with diabetes? Simply demonstrating—over and over again in observational studies—that smoking is bad for you is like the movie Groundhog Day without Bill Murray and Andie MacDowell ever falling in love.

  Yes, replication is important, but flogging a dead horse is not. These topics are chosen over and over because it is easy to publish this work (and get press coverage of it). These studies, however, do not push us forward as a profession.

  How does this relate to reversal? Useless research often produces poorly founded conclusions that are then acted upon. Only later do we learn that our interventions were ineffective. Performing the types of studies we suggest might reveal therapies that run counter to what we would have guessed. We told you that a gem of a study found that weight-lifting improves symptoms of lymphedema. This was an example of reversal (see study 126 in the appendix). For years, doctors told women not to lift weights after their lymph nodes had been removed, for fear it would worsen the condition. When you actually begin to test ways to implement what you think you know, you start to learn interesting things.

  AN EXCELLENT PHYSICIAN

  For all the reasons presented in this chapter, we have to do better to define what makes an excellent physician. An excellent doctor is not the one who studies whether eating strawberries in moderation is good for you. He should not be on the nightly news or necessarily promoted at his university, especially while the doctor who will hold your hand while she tells you the test results that no one wished for is penalized for not being academically productive.

  The ideal academic doctor is someone who spends a modest portion of his time caring for patients and the balance teaching, reading, thinking, and conducting research—research aimed at discovery or replication of potentially important findings. The ideal doctor is not someone who spends four months at conferences, collects honoraria, and works as a tireless advocate for industry. The best publications, research or thought pieces, are those that engage the medical community and prompt it to think about how to best care for people. They are not the next industry-sponsored, uncontrolled device trial. The best professor is someone who inspires her students to become better doctors. She is not the one who gives students a gift authorship on a paper showing that exercise improves blood flow. A productive faculty member is not the one who brings in $2 million from GlaxoSmithKline, but the one who brings wisdom and compassion to the bedside, to his students, and to his trainees and develops thoughtful research questions during his clinical work. Medical reversal is, in part, the price we pay because what we value in the academic medical center has become skewed.

  A little reform might go a long way toward making sure the goals of academic medical centers are what we believe them to be: to promote human health. In
terventions to realign these goals must be studied to determine whether the modified goals really do produce better patient care and more robust discoveries. If they do, they should be widely adopted. If they do not, we should go back to the drawing board so that we do not reform our medical centers only to have to reverse ourselves in the future.

  16 REFORMING THE SYSTEM :: THE BURDEN OF PROOF AND NUDGING OUR WAY PAST REVERSAL

  MEDICAL REVERSALS ARE EVERYWHERE. They are rooted in our tendency to accept new practices without really knowing that they work. So far in this part of the book, we have suggested that reforming medical education and academic medicine could lessen the prevalence of reversal. We have also listened to our own sermon: we have tried to caution that most of our reforms are not ready for prime time and should be subject to pilot programs and testing in randomized trials. In this chapter, we introduce two more proposals. Both are ambitious and both take aim at the foundation of medical reversal. One requires a new ethic among drug and device developers, and the other calls patients and doctors to commit themselves to clinical trials.

  THE BURDEN OF PROOF

  Semper necessitas probandi incumbit ei qui agit. The necessity of proof always lies with him who lays the charges. This legal principle, if applied to medical innovation, would go a long way toward curbing reversal.

  The principle of burden of proof means that for a given claim, one side has the burden of proof—the responsibility to show that its statement is true—while the opposing side has the benefit of assumption. If the first side is unable to prove its claim, the other side is assumed to be correct. An example makes the principle clear. Imagine that a PhD student in biology is doing a dissertation on the reptiles of the Nicaraguan rain forest. He returns from a field expedition and claims that he discovered a new species of snake. The snake he found is longer than a boa constrictor, spends most of its life under water, has the ability to alter its colors (like a chameleon), and has a sharp barb at the end of its tail. The question is, Does this hitherto undiscovered snake exist? The obligation to produce evidence falls upon the graduate student. He could show a picture of the snake. He could produce a molted skin. He could bring a specimen, dead or (terrifyingly) alive. These pieces of evidence would rank on a scale of lesser to nearly certain proof.