“There is no such thing as bad data. It's in the analysis and interpretation where we go awry.” That was the overriding theme of noted health care quality and information technology expert Ashish Jha, MD, MPH, in his keynote speech at the 28th annual Symposium on Health Care Services in New York: Research and Practice. His message was echoed throughout the meeting as presenters discussed innovative programs that use data to advance patient engagement strategies.
The November 14 symposium, attended by some 200 health services researchers and practitioners, was jointly sponsored by United Hospital Fund and GNYHA Foundation, in collaboration with leading health research organizations.
Ashish Jha
Challenging common beliefs in “An Ounce of Evidence: The Possibilities and Limits of Data,” Dr. Jha said he was determined to dispel five research myths accepted by many in health care: that there is bad data; that correlation is never causation; that a single study can answer a question; that p-values, which measure the probability that a hypothesis is proven true, should be ignored; and that measuring what you can is better than no measurement at all. The K.T. Li Professor of Global Health at Harvard University, Dr. Jha is also senior associate dean, research translation and global strategy, at Harvard's T.H. Chan School of Public Health; director of the Harvard Global Health Institute; and professor of medicine at Harvard Medical School.
Dr. Jha discussed several studies, including some of his own, to illustrate some of the pitfalls of interpreting data. Most controversial was a study he led, published in February 2017 in JAMA Internal Medicine, exploring whether women are better doctors than men. The data revealed slightly lower 30-day mortality rates among the Medicare patients of female physicians than among those of male doctors, when controlling for all other variables.
The difference, Dr. Jha noted, was only a modest 0.4 percent, resulting in an almost negligible p-value. However, that small gap becomes far more significant when considered from a public health perspective: nationwide, if the patients of male doctors had the same outcomes as those of female physicians there would be 32,000 fewer deaths per year in the Medicare population. “While the effect size is not huge, it's certainly not trivial,” he said.
When attempting to determine whether a study proves causation or merely correlation (a question that swirled around the physician gender study), Dr. Jha said researchers must think about bias and chance, explore alternative explanations, and look for “quasi-experimental designs,” that is, non-randomized studies that look at pre-intervention and post-intervention results (his presentation and others from the meeting appear below).
The symposium moved from theory to the real world with the “Measuring Patient Engagement Strategies” panel, moderated by Tim Johnson, executive director of GNYHA Foundation. All three panelists agreed that patients are generally eager to have a greater role in determining their care, and that clinicians appreciate the opportunity to help patients become more engaged. As one panelist, Allison McIntyre said, “If you want to know how your patients are doing, just ask them.”
(From left) Tim Johnson, Jessica Steinhart, Allison McIntyre, Leigh Simmons
Ms. McIntyre, clinical program manager for patient-reported outcomes at the University of Rochester Medical Center's Medical Faculty Group, discussed UR VOICE (Validated Outcomes in Clinical Experience), a patient-reported outcomes platform used by 550 physicians in 30 clinical programs at the hospital. The widely used PROMIS (Patient-Reported Outcomes Measurement System) works with UR VOICE to capture key health indicators, such as physical function, mood, and pain. Sharing data with different groups throughout the hospital identifies problems and helps develop innovative solutions and improvements for both patient and community health.
Leigh Simmons, MD, medical director of MGH Health Decision Sciences Center, reviewed the 10-year history of the shared decision-making process at Partners HealthCare in Massachusetts, in which patients and their doctors together decide on a course of care. Partners, which encompasses Massachusetts General and Brigham and Women's Hospitals, started a pilot program in 2005, and in 2006 expanded to 18 Mass General primary care practices. Today the process has been adopted by all primary care practices and has been extended to specialty practices in orthopedics, cardiology, and obstetrics/gynecology. Using electronic health records, a variety of decision aids, and clinician training, the system has been well-received by doctors, including specialists, and patients.
Jessica Steinhart, director of Ambulatory Care Initiatives, Staten Island Performing Provider System (PPS)—a network of more than 50 agencies—described Staten Island Hospital's Patient Activation Measurement (PAM) project, designed to identify patients' ability to manage their health, reduce emergency department use, and increase the use of primary care services. In the first two years of PAM's activation, the PPS has seen a steady improvement in patient engagement scores and a decrease in ED visits.
The symposium also featured 15 poster presentations examining the use of data in a broad range of health care applications, including substance abuse treatment, community and population health issues, electronic health records and other health care technologies, antibiotic stewardship, and service delivery transformation.