I’ve been following a great discussion on Susannah Fox’s blog today. The Lohr article that sparked her post and subsequent comments have taken up the position that Evidence Based Medicine tends to dismiss the role of intuition. I think that is true and reflects a cultural phenomenon that seeks a deus ex machina. Fads emerge and advocate for the latest savior (or rainmaker), eschewing the difficulties and hard work involved in engaging with truly complex, multi-factorial systems.
I’m reminded of a music professor who told my composition class that, “you can’t break the rules unless you first understand the rules.” I’ve thought about this for a long time, why is it you can’t be creative in ignorance of the rules? Ignorance of impossibility, after all, is often the fuel for innovation. Yet it seems to hold true that the best musicians don’t run away from convention, they master and move beyond it. Does a powerful command of the grammatical rules and literary traditions of English limit your creativity in writing? No, it hones and unleashes it!
In it’s ideal form, EBM is a foundation of evidence and rules that should empower and propel individual physician and patient creativity. In the quality improvement context the standard of care is there to help us avoid repeating other’s mistakes, not to tell us what to do under all circumstances. The prosaic landscape of diagnostics and therapeutics are governed by probability distributions over endpoint outcomes, but beyond this there remains endless room for innovative thinking and intuition because any time we come into connection with real people and real bodies, we exceed the ability of our formalisms to compute. We can only meet the challenge of other’s humanity through the lens of our own.
Underlying much of the Big Data hype is an implicit, and dangerous belief that “feeding big data to algorithms will yield superior and actionable insight.” It ignores the subtle issues of context that dictates the utility of data and knowledge; the problem is that context is often uncomputable. Google has largely institutionalized the religion of the algorithm, leading to many technically impressive but ultimately disappointing products; I believe this is because their culture devalues the design thinking needed to tame the uncomputable.
This is not unique to technology, public policy pundits indulge profoundly in magical thinking with regard to false gods like market mechanisms on the right and legislative foresight on the left – let people compete and the invisible hand will fix all (e.g. vouchers for education); legislate or mandate against bad outcomes and all will be improve (e.g. dietary guidelines for heart disease). The former ignores the contextual realities of geography, wealth disparity, and the psychology of incentives, the latter the inherent multi-causality of human health.
There is no god in the machine, only pieces of a puzzle complex beyond our mathematics and the mind of any one of us. Only the dynamic, adaptive fabric of our culture and an expanding sphere of knowledge and tools to harness it can encompass the scope of challenge we face. It is culture more than anything that provides the fuel for our individual intuition — if we are lucky we get to occasionally produce something that alters that culture for the benefit of others. Yet, we must be careful, for many cultural contributions are negative and new memes can easily blunt and wither our individual intuition. Memes can spread like wildfire in our connected, tweet-infused world and we must beware that we are not the agents of false gods.
This was the essence of Bacon’s revolution, to escape intellectual idolatry through a commitment to a rigorous, collective discipline that places evidence at the center and relies on debate to wring concensus regarding the extrapolation from evidence to truth, and so advance knowledge. Bacon also described the four intellectual idols of his day which his methodology was intended to combat, and which remain surprisingly relevant today:
– Idols of the tribe, or false notions due to the human nature and common to all men. Inability of most people to reason naturally about statistical phenomenon and multi-causality and to rely instead on naive extrapolation from example to rule.
– Idols of the cave, or personal interpretations due to individual makeup or disposition. Do you believe that human intuition is fundamentally flawed? Is it fundamentally distinct and powerful? Do you care?
– Idols of the market-place, or the problem of language and the confusion of words and terms. What the do we all mean when we say Big Data? What do I mean when I say I want to fix healthcare?
– Idols of the theatre: The dogmas of philosophies that are received from wrong “laws of demonstration.” Reasoning from examples of commercial or research success in one area directly to proscription or policy. This describes far too much of what passes for intellectual discourse today, including most punditry (Nate Silver excepted).
This is the larger context that should be applied to Evidence Based Medicine. We can only contribute to enhancing shared culture if our intuition operates with a deep understanding of our rules/data/algorithms/predictions including their limitations, the circumstances under which they apply, and the extrinsic factors that confound our observations of their effects. Our unique intuition can then pose better questions, and then use the right tools to answer them. We need a commitment to creating systems to share both evidence and outcomes so we can better debate our inferences, learn from one another and in so doing, propel our culture forward to more constructive ends.
Frumious Abstractions 
So who will put together the FAQ on "a deep understanding of our rules/data/algorithms/predictions including their limitations, the circumstances under which they apply, and the extrinsic factors that confound our observations of their effects."What would the sources be for that (I’m a librarian so it’s my duty to ask this)?
There really is no such FAQ that can replace education in critical thinking. At a high level I am advocating for doubling down on the scientific method, but with an engineer’s mindset, not a scientist’s. One of my favorite quotes was by Tom Knight who said: "The biologist seeks to understand one way of producing a reaction, the engineer wants to understand every way of producing a reaction and the parameters of each method." That is to say, it isn’t enough to describe a phenomenon, we need to model and characterize it so we can understand when we can make predictions with some accuracy and when we cannot. We regularly make massive public policy blunders, like the low-fat diet prescription, which has had massive unintended consequences because the potential benefit (replacing saturated with unsaturated fat in our diet) instead led to the rise of cheap, sugar-based foods labeled as ‘healthy low-fat foods’ and precipitated a diabetes and obesity crisis, ultimately worsening heart disease (if we factor out the impact of statins, etc). Sometimes doing nothing is better than trying to fix a problem without really understanding it, as hard as that can be to do. Neither doctors nor Congress like to say "doing nothing is the best medicine." The better approach was likely making small changes in diets and observing the effect. Similarly, the FDA is highly conservative about letting drugs through their approval process because there is only penalty for mistakes, no reward for lives saved. What if we were less stringent on approval and instead improved our post-approval surveillance using modern EMRs? What if we allowed doctors more autonomy in making risk judgements with post-Phase I drugs? Would more people in aggregate be helped by shortening and reducing barriers to production, even if some were harmed by approved drugs? http://www.tedmed.com/talks/show?id=7267 (Of course in general I think that drugs on average are a poor patch for our limited understanding of human health, but that argument is longer than this comment will support!)
I participated in Susannah Fox’s blog and expanded more on my original idea there in my most recent post at Wired: http://bit.ly/ZQsLMw. Please feel free to check it out and let me know your thoughts.