by Thomas H Lee
How does patient experience relate to other outcomes? Michael Porter’s classic New England Journal of Medicine paper “What Is Value in Health Care?”2 describes a three-tier hierarchy for measuring and thinking about patient outcomes. Porter points out that there is no single outcome that tells the full story for any group of patients; multiple outcomes collectively define success. The complexity of medicine means that multiple outcomes must be pursued, and sometimes competing outcomes must be weighed against each other (e.g., near-term safety versus long-term functionality).
In Porter’s hierarchy, the most important outcomes are at the top, and lower-tier outcomes become a focus only if there has been success at the higher levels (Figure 4.2). Tier 1 outcomes are “hard” clinical outcomes, with survival being the one of greatest importance for many conditions. Right below survival is the degree of health or recovery achieved, which can often be measured only by asking patients how they are doing. For example, virtually all hospitals have similar five-year survival rates for prostate cancer surgery (about 95 percent), but there is tremendous variation in outcomes such as incontinence and sexual dysfunction.
Figure 4.2 Michael Porter’s three-tier outcomes measurement hierarchy
Reprinted from Michael E. Porter, “What Is Value in Health Care?” New England Journal of Medicine 363, December 23, 2010: 2477–2481.
If a hospital is statistically worse than expected on Tier 1 outcomes, especially mortality, it faces a crisis that must be addressed as quickly as possible. However, it is difficult to clearly and consistently outperform other good providers on mortality and other Tier 1 outcomes. If a provider group is doing well on Tier 1 outcomes, it must then turn its attention to Tier 2 outcomes, which are related to the process of care.
The first level of Tier 2 outcomes reflects the time required to achieve recovery. These outcomes include delays and the disutility of care; such outcomes may not influence mortality but matter greatly to patients. Patient experience data dominate this tier of Porter’s framework. Tier 3 outcomes reflect the durability of the benefits of a healthcare intervention. For example, a hip replacement that lasts 2 years is not the same as one that lasts 15 years.
The implied message of the switch from patient satisfaction to patient experience is that efforts aimed at improvement that focus solely on patients’ perceptions of their service are not strategic investments of resources. Those patient satisfaction efforts may bring a temporary bump in volume, but they do not actually improve value, which is measured as outcomes versus the cost of providing those outcomes. Improvement of value for patients has to be the core of any viable long-term strategy for healthcare organizations.
Improvement of patient experience means getting better at meeting patients’ needs, clinical as well as psychosocial. If clinicians feel skeptical and lack a visceral connection to patient satisfaction, they tend to resonate with improvement of patient experience and the concept of value once they grasp their meaning: meeting patients’ needs as efficiently as possible.
Data show that patients have understood what was really important in healthcare all along. In analyses of the factors most strongly correlated with top ratings for hospitals, the following were at the top:
• How well the staff worked together
• Responsiveness to concerns and complaints
• Whether patients were included in decisions regarding their treatment
• Responsiveness to emotional needs
• Whether nurses kept patients informed
In these analyses, the quality and temperature of the food ranked near the very bottom, along with the courtesy of the person who served the food. Good “service behaviors” can be helpful in reducing patients’ anxiety and stress, but patients seem to know that when they seek healthcare, they are not coming to a resort for recreation or a restaurant for a great meal.
Concerns About Patient Experience Data
If patient experience and other outcomes become the focus of healthcare delivery, it makes perfect sense that measuring patient experience and giving providers feedback on the results are essential. However, there is a problem: no one likes feedback that indicates that there is room for improvement.
This dynamic is part of the human condition and is summarized well in the book Thanks for the Feedback: The Science and Art of Receiving Feedback Well.3 The authors, Douglas Stone and Sheila Heen, point out that we all receive feedback all the time but dread it and would rather dismiss than accept it. There is a basic conflict between our desire to improve and our desire to be appreciated and accepted as we are. Guess which one kicks in first most often.
Stone and Heen argue that we probably need to spend a bit less of our energy on perfecting the data and a bit more on preparing people to receive the feedback (actually, a lot more on the latter). In healthcare, that means recognizing that the first response to critical feedback is to look for reasons why the findings may be wrong. The second is to try to discredit the sources. To have these reflex reactions is to be human. The goal should be to recognize them in ourselves and not let them pose a barrier to using feedback to improve.
As you might imagine, my colleagues at Press Ganey and I hear criticisms, concerns, and skepticism about patient experience measurement all the time. Providers worry that the measures reflect hotel functions rather than the issues that determine patients’ health outcomes. Also, they are concerned that the conclusions drawn from the data are not statistically valid because the sample sizes are too small and that the ability to adjust for variables that might skew the results is limited.
Some of these concerns are myths or are simply untrue. Others are real and need to be put in perspective while the problems are addressed. Fortunately, measurement is evolving in ways that are taking on these challenges and making the feedback on patient experience and other types of data more useful for driving an epidemic of empathy.
Adopting a framework from my colleague Deirdre Mylod, I divide these concerns into two groups. Conceptual issues include topics such as concern that greater patient satisfaction might be linked to worse outcomes and that measurement of patient experience might actually worsen care. Data issues are concerns about the validity of the information. Are enough data being collected? Are the results stable and statistically valid? Do they have meaning for individual physicians?
In effect, these common concerns represent feedback for organizations that collect patient experience data and for the provider leaders who use them. Clinicians should recognize that their reservations about patient experience data might be a reflection of the natural tendency to try to deflect feedback. However, my colleagues and I and provider leaders should similarly avoid the tendency to hastily dismiss these reservations and thus miss opportunities to improve.
Conceptual Issues
Does measuring patient experience have potentially perverse effects? This question is frequently raised by physicians who are concerned that their care is being evaluated on the basis of data collected from patients. Here are some examples of the issues they raise:
1. Medicine is complex, and patients do not have the training to know what is in their best interest. Therefore, patients’ opinions of what constitutes good care are of uncertain value at best.
2. There are many patients who have strong but misguided ideas about what they need, for example, an MRI of their brains for a headache or antibiotics for a cold. Society’s interests are best served by avoiding overuse or inappropriate use of such care, but withholding it may cause patients to rate their physicians poorly.
3. Some patients seek narcotic pain medications and will rate physicians poorly if those medications are withheld.
Critics argue that perverse consequences can result if too much attention is devoted to patient experience data, as if it were the only dimension of quality that mattered. They worry about unintended effects such as acceding to patients’ requests for costly, inappropriate, and potentially harmful therapies if financial incentives are based on
patient experience measures. Furthermore, critics worry that physicians are leaving their practices because of the intensity and potential perversity of pressures related to patient experience.
Is there evidence that physicians are being subjected to personal financial incentives that are focused on narrow targets, such as pain management? I have not found hospitals or physician groups tying compensation to changes in individual measures such as the percentage of patients who are satisfied with their pain management. If there were any, I would be critical of such an incentive.
Still, perception can be more powerful than reality. I recently asked an angry but thoughtful emergency medicine physician if he knew of any doctors who had a financial incentive that was based on patients’ pain scores, if he knew doctors who gave narcotics or antibiotics to every patient who asked for them, or if he knew any physicians who had left the practice of medicine because they believed those measures were causing them to do the wrong thing. He agreed that he could not document that any of these assertions were true. “But,” he said, “it feels like they are true.”
In a sense, he was reflecting the duress that clinicians are under in the modern era of healthcare. That duress is real and causes physicians and others to latch on to any data that might undermine the validity of the feedback they are receiving.
One paper is brought up repeatedly as evidence that higher patient satisfaction may lead to worse care. A 2012 study published in the Archives of Internal Medicine was based on data on 51,946 adult respondents to the national Medical Expenditure Panel Survey from 2000 to 2007.4 Researchers led by Joshua Fenton, MD, of the University of California, Davis, looked at data for each patient over a two-year period, including mortality through the end of 2006. Patient satisfaction was assessed with five items from the CAHPS survey from the first year, and correlations were sought with healthcare utilization in the second year.
The researchers found that patients in the highest quartile of patient satisfaction had lower rates of emergency department use, higher rates of hospitalization and medication use, greater total spending, and higher mortality compared with patients in the lowest quartile. The paper is often quoted as suggesting that higher patient satisfaction leads to higher death rates and higher costs.
The authors did not draw any such conclusion, of course. They noted the limitations in the data available to them. They could only work with five questions regarding patients’ assessments of their primary care physicians’ performance on specific activities (e.g., listening carefully, explaining things in a way that was easy to understand) and a global assessment of all their care from all physicians and providers on a scale of 0 to 10. These data were rolled together, with each item weighted equally, into a measure that was supposed to represent year 1 satisfaction. They could not look, for example, at whether patients’ satisfaction with their hospital care correlated with hospital mortality.
If one reads the paper carefully and looks at the actual data in the tables, there is not a “dose-response” effect: a trend in which the higher the patient satisfaction, the higher the mortality. Yes, the patients in the highest quartile have the highest mortality, but the patients in the third quartile actually have slightly lower mortality than do the patients in the second quartile. If higher satisfaction led to worse outcomes, one would expect the opposite. The same is true for hospital admissions.
The authors understood the limitations of their data and their analysis and pointed out that “patient satisfaction may be a marker for illness, identifying patients who rely more on support from their physicians and thus report higher satisfaction.” They expressed concern that excessive attention to patient satisfaction incentives may distract physicians, but in their concluding paragraph they wrote, “These associations warrant cautious interpretation and further evaluation, but they suggest that we may not fully understand the factors associated with patient satisfaction.”
The caution they urge has not always been exercised by critics of patient experience measurement who quote their paper. Nevertheless, when I ask, “Do you really think that better communication with patients leads to higher hospital admission rates or increases mortality?” not one person has said yes.
The Archives paper is quoted so often in large part because it stands alone. The overwhelming findings from other research studies are in the opposite direction: better patient experience correlates with better quality and better patient outcomes. For example, the New England Journal of Medicine paper published in 2008 by Ashish Jha and colleagues from the Harvard School of Public Health analyzed the relationship between patient ratings of hospitals in the HCAHPS survey and other quality metrics related to hospitals. They found that hospitals in the top quartile of HCAHPS ratings had better performance on quality metrics for all four clinical conditions examined: acute myocardial infarction, congestive heart failure, pneumonia, and prevention of surgical complications.5
A study of more than 100,000 patients undergoing surgery by the American College of Surgeons National Surgical Quality Improvement Program analyzed the relationship between HCAHPS performance and well-documented clinical outcomes, including 30-day postoperative mortality, major and minor complications, and hospital readmission.6 After adjusting for the extensive clinical data in their data set, the researchers found that hospitals in the top quartile in HCAHPS performance had a 15 percent reduction in risk of patient death compared with hospitals in the lowest quartile. Their conclusion: “Using a national sample of hospitals, we demonstrated a significant association between patient satisfaction scores and several objective measures of surgical quality. Our findings suggest that payment policies that incentivize better patient experience do not require hospitals to sacrifice performance on other quality measures.”
When this general topic—is patient experience related positively or negatively to actual patient outcomes?—was examined by researchers from Duke, they concluded that “patient experience measures provide robust measures of quality.”7 They shared others’ skepticism that amenities improve patient outcomes but noted how the HCAHPS survey gets at activities that are at the core of good medicine. They cited as examples three specific questions from the survey:
• Question 3, from the section “Your Care from Nurses”: During this hospital stay, how often did nurses explain things in a way you could understand? (Answer options: Never, Sometimes, Usually, Always).
• Question 17, from the section “Your Experiences in This Hospital”: Before giving you any new medicine, how often did hospital staff describe possible side effects in a way you could understand? (Answer options: Never, Sometimes, Usually, Always)
• Question 20, from the section “When You Left the Hospital”: During this hospital stay, did you get information in writing about what symptoms or health problems to look out for after you left the hospital?” (Answer options: Yes, No)
They thought such measures were powerful and intuitively valid measures of quality but noted that we have much to learn about how to collect and analyze the data.
On the specific issue of prescribing pain medication, there are no questions that patients with drug-seeking behavior exist, particularly in the emergency department setting. Not many of those patients actually complete patient experience surveys, but nevertheless, the fact that patient experience is being measured definitely crosses physicians’ minds when they find themselves in conflict with patients. Regardless of how physicians may respond to this concern in individual cases, it is worth noting that a low threshold for prescribing narcotics would be a poor strategy for trying to raise one’s patient experience scores.
First, a study of 4,749 emergency department patients showed no relationship between use of opioid analgesics and Press Ganey measures of patient experience.8 If anything, there was a trend in the opposite direction: more narcotic use was associated with lower scores. Second, pain management did not figure at all in the analysis of the drivers of likelihood to recommend described in Chapter 2.
An
analysis of data from emergency department patients shows that pain management is important to them, but after adjusting for data on communication, compassion, and coordination, the impact of pain management on the likelihood to recommend an emergency department is no longer statistically significant. If physicians are prescribing narcotics more freely in an attempt to improve their ratings or win their patients’ loyalty, it is a poorly conceived strategy. What their patients actually want is compassionate, connected care, at the core of which is sensitivity to their physical and other types of suffering.
There are other studies showing that better patient experience scores correlate with lower hospital mortality,9 better guideline adherence,10 and better resource use.11 Nevertheless, there is no denying that medicine is a complex activity, and conflicts between values within it abound. The best example may be the paper by my late friend and colleague Jane C. Weeks, MD, who described how many patients receiving chemotherapy for incurable cancers do not understand that the chemotherapy is unlikely to be curative. Her study showed that physicians who seemed to be more effective in communicating the true prognosis to their patients might be rated more poorly for their communication skills. In other words, physicians who were more truthful might have lower ratings of patient satisfaction.12
The findings in this paper should never be used as an argument for misleading patients about their prognosis in the pursuit of higher patient experience scores. Instead, the data demonstrate the complexity of medicine and the importance of adopting a nuanced approach to the use of the data. First, however, one must address the concerns about the data themselves.
