Who Knows Best?

Jerome Groopman’s essay, “Health Care: Who Knows “Best”?” in the February 11th New York Review of Books isn’t going to help pass the moribund legislation before Congress. Groopman argues that there are serious problems in relying on studies of comparative effectiveness to improve the quality of care--one of the central components of both the House and Senate bills.

Obama often says, Let’s study and figure out what works and what doesn’t. And let’s encourage doctors and patients to get what works. Let’s discourage what doesn’t.

According to Groopman, it’s not quite that simple. The findings of comparative effectiveness research are always open to further testing and ultimate refutation or qualification. What is true today is often false tomorrow.

Groopman cites several examples: “Best practices” research once demonstrated that blood sugar levels should be tightly controlled in critically patients in intensive care. Later research showed that this practice was not only shown to be wrong but resulted in a high likelihood of death when compared with measures allowing a more flexible treatment and higher blood sugar.”

He points to another failure in treating hip and knee replacement by orthopedic surgeons. In this case, conforming to or deviating from the “best practice” procedures based on comparative research had no effect on the rate of complications from the operation or the outcome of the treated individuals.

Groopman claims physicians are growing increasingly dubious of efforts to standardize clinical practice based on comparative effectiveness studies.

“…clinical trials yield averages that often do not reflect the real world of individual patients, particularly those with multiple medical conditions. Nor do current findings on best practices take into account changes in an illness as it evolves over time. Tight control of blood sugar made help some diabetics, but not others.”

As our knowledge of the disease process grows, the care and treatment of patients has become overwhelming complex. So it is not surprising that the clinical application of the best practices model is fraught with difficulties. As Groopman points out, the findings fail to distinguish between those patients where it works and those where it doesn’t.

To add to this complexity, Groopman develops further the role of cognitive biases in medical decision making and the pitfalls of human reasoning in situations where an easily made error can end a person’s life. He takes special note of the following potential sources of error

• Overconfidence Bias—over estimating the importance of his own work and analytical skills

• Confirmation Bias—the tendency to ignore and discount contradictory evidence

• Focusing Illusion—basing a decision on a single patient change that is mistakenly employed in predicting the effects on the overall condition.

Groopman’s essay calls into question one of the basic tenants of Obama’s health care proposal. He concludes: “The care of patients is complex and choices about treatments involve difficult tradeoffs. That the uncertainties can be erased by mandates from experts is a misconceived panacea, a focusing illusion.”

This does not imply comparative effectiveness research should be disregard. To the contrary, it means there is much more that has to be taken into account in medical decision making-- the history and state of the individual patient, the boundary conditions of the research findings, the experience of the physician in dealing with the patient’s problem and the numerous value judgments that both the physician and the patient will want to consider.