Evidence based medicine

The New York Times magazine’s headline story this Sunday is on evidence-based medicine.  It talks about how a physician, Brent James, has been developing objective empirical means to measure outcomes and use the data to design medical protocols.  This is a perfect example of the migration from a highly skilled and paid profession (what I called an NP job in a recent post) to a more algorithmic and mechanical one (a P job).  Here are some excerpts from the story:

For the past decade or so, a loose group of reformers has been trying to do precisely this. They have been trying to figure out how to improve health care while also holding down the growth in costs. The group includes Dr. John Wenn­berg and his protégés at Dartmouth, whose research about geographic variation in care has received a lot of attention lately, as well as Dr. Mark McClellan, who ran Medicare in the Bush administration, and Dr. Donald Berwick, a Boston pediatrician who has become a leading advocate for patient safety. These reformers tend to be an optimistic bunch. It’s probably a necessary trait for anyone trying to overturn an entrenched status quo. When I have asked them whether they have any hope that medicine will change, they have tended to say yes. When I have asked them whether anybody has already begun to succeed, they have tended to mention the same name: Brent James.

…In the late 1980s, a pulmonologist at Intermountain named Alan Morris received a research grant to study whether a new approach to ventilator care could help treat a condition called acute respiratory distress syndrome. The condition, which is known as ARDS, kills thousands of Americans each year, many of them young men. (It can be a complication of swine flu.) As Morris thought about the research, he became concerned that the trial might be undermined by the fact that doctors would set ventilators at different levels for similar patients. He knew that he himself sometimes did so. Given all the things that the pulmonologists were trying to manage, it seemed they just could not set the ventilator consistently.

Working with James, Morris began to write a protocol for treating ARDS. Some of the recommendations were based on solid evidence. Many were educated guesses. The final document ran to 50 pages and was left at the patients’ bedsides in loose-leaf binders. Morris’s colleagues were naturally wary of it. “I thought there wasn’t anybody better in the world at twiddling the knobs than I was,” Jim Orme, a critical-care doctor, told me later, “so I was skeptical that any protocol generated by a group of people could do better.” Morris helped overcome this skepticism in part by inviting his colleagues to depart from the protocol whenever they wanted. He was merely giving them a set of defaults, which, he emphasized, were for the sake of a research trial.

… While the pulmonologists were working off of the protocol, Intermountain’s computerized records system was tracking patient outcomes. A pulmonology team met each week to talk about the outcomes and to rewrite the protocol when it seemed to be wrong. In the first few months, the team made dozens of changes. Just as the pulmonologists predicted, the initial protocol was deeply flawed. But it seemed to be successful anyway. One widely circulated national study overseen by doctors at Massachusetts General Hospital had found an ARDS survival rate of about 10 percent. For those in Intermountain’s study, the rate was 40 percent.

The criticism of evidence based medicine, as voiced by Jerome Groopman, is that:

… a fundamental problem with “systems analysis,” as he calls it, is that it discourages doctors from considering a wide-enough array of possible treatments. He also worries that if doctors are judged based on how well they follow a protocol, they may follow it even when they are correctly skeptical of it. Groopman says that the proper solution to misdiagnosis instead lies with individual doctors. If they are taught the ways in which their instincts can lead them astray, and if they reflect on their previous mistakes, they can avoid some of the pitfalls of intuition. They can become more self-aware.

I think that there is some element of truth to this line of thinking.  Let’s take the example of radiology.  In the future we will have automated pattern recognition software that will scan X-rays.  These tools will likely involve some sort of machine learning process that is trained, with inputs from real radiologists, on classifying large data sets.  However, these systems can never be perfect.  There will be parameters that must be selected and choices must be made on how to balance specificity vs. sensitivity.  There will always be situations where the machine will miss a potential life threatening disease that a well trained radiologist might catch.  But my guess is that as time progresses the number of radiologists that do better than the machine will steadily diminish.  The machine will also be more consistent and won’t be affected by emotional state or fatigue.

Does this mean that eventually all physicians will be replaced by machines?  Again, I don’t think so.  There will probably always remain a small handful that will work on non-routine cases and calibrate the machines.  Some people will always insist on being seen by a human doctor. However, for most of us,  we’ll just interact with a nurse who will feed data to and act on the outputs of a machine.