The Fog of Innovation

The Fog of War is invoked to describe the uncertainty that permeates combat operations. This term is generally attributed to a quote by Clausewitz: “The great uncertainty of all data in war is a peculiar difficulty, because all action must, to a certain extent, be planned in a mere twilight, which in addition not infrequently — like the effect of a fog or moonshine — gives to things exaggerated dimensions and unnatural appearance.” (via wikipedia).  

The evolution of an innovation or innovative company faces a similar fog.  At any point in time, we extrapolate from current conditions a set of possible future outcomes and take action to try to bring the best outcome into being.  In one view, we should make economically rational decisions, those that maximize our expected return over the outcomes weighted by their likelihood.  

However, Nassim Nicholas Taleb’s “The Black Swan” argues convincingly us that the seeds of disruption and destruction lie in events that exist outside all reasonable extrapolations of a current state of affairs.  The occurrence of a highly improbable event is, in fact, inevitable.

Since we can’t know anything for sure, innovators have to operate at a completely different level than analysts.  It’s a chess game where the rules can change mid-play forcing a complete re-evaluation of strategy.  For example, the iPhone was a Black Swan to the cellular industry, as this analyst’s perspective shows. This dynamnic is why you should ignore Forrester and other analysts – they tell you what the world will be like if nothing surprising happens, but surprising things always happen.

How do we make investments of our time and energy amidst all this uncertainty? Taleb’s advice summarizes as becoming robust in the face of bad surprises and developing a capability to exploit the positive ones.  Infrastructure investments, hiring, business processes, etc can all be viewed in this light; everything should have and be capable of more than one specific use.  

Startups are often more innovative than large companies not because they are smarter, but because they are constantly forced to take action and challenge assumptions to survive.  Moving forward, imperfectly, is a virtue because taking action is a powerful form of learning that dispels many unknowns.  Knowing when you know just enough to make a course change is the heart of what differentiates a great innovator from the average one.

A classic failure mode of startup CEOs is changing course too easily and too often.  Rapid course changes have a switching cost, negative impact on morale, long-term costs in product infrastructure (abandoned code, designs, etc), investor and customer confidence, and market positioning / sales. The opposite extreme, sticking to your plans regardless of evidence, is often used to characterize the factors that drove success stories — the genius founder focused on their goal and despite all naysayers achieved the promised land.  Those reports I think reflect our reporting bias, not a sound strategy for success.  Navigating the fog of innovation requires a set of personal and organizational tools that I believe can be learned and developed.

Case Study

My first company, Silicon Spice, was funded to productize a reconfigurable computing array we developed at the old MIT AI Laboratory.  Our array was general purpose, but the company was targeting the software dial- modems market — promising a 10x improvement in computational density in that era.  We discovered a year later that the intellectual property issues dominating the modem market made any technology innovation, not just our own, difficult to monetize.  We started looking at adjacent emerging markets, specifically high-density digital voice coding, where the intellectual property issues were less challenging.  We continued on our original general product course, but it quickly became clear that voice coders had very different computational properties. Our prototype device vastly undershot the performance targets for the new product due to unanticipated computational bottlenecks. We also learned that at the densities we were targeting, power density and cooling hotspots were larger issues than the total cost or absolute size of our solution.  This meant we couldn’t afford a solution that used off-chip memory.  Put all this together and we were forced to rethink the entire solution.  

However, by this time, we knew more about the details of the market that was emerging than anyone else – the marketing story for our modem product bought us an audience with all the product architects in the marketplace and because we had no choice but to innovate for a new market, were more receptive to their needs than our competitors.  Moreover, we had developed a set of technical ideas, tools, and team capabilities that made it possible to design from scratch a brand new microprocessor architecture, a 20 core device (in 1998!), compiler, OS, development IDE, application software, and systems test lab in 15 months. 

I would love to claim, as many founders and CEOs have, that this learning by doing and winning by adapting was a conscious strategy, but it wasn’t.  We were lucky with how the timing played out. My principle point of pride in the team at Silicon Spice is that we executed on that redesign with nearly flawless precision; I’ve never since worked with or seen a team that functioned as well as we did in those critical transformative years. It was our ability to make a decision to overhaul our approach, and then execute like crazy against the new goal, that made all the difference.

On the engineering side, we had a saying: “slip once, slip big”.  If your plans don’t unfold as you expect and you have to make big changes, wait until you know what the change is and can reduce the unknowns to predict the implications of the change, then pull the trigger.  In the meantime, adopt our other important cultural value: “the plan of record is the plan of record until it is no longer the plan of record.”  The implication being that it is confusing to everyone to have a constantly moving target out there – stick to your plans while you plan for the change.  Keep it clear in everyone’s mind that the current plan remains job #1 until we have committed to a concrete change.  This creates a culture that can execute without distraction, but is able to adopt changes when the time is right. Good product and engineering managers will be able to blur the lines between these two worlds and prioritize present plans with an eye to future changes.  (both of these phrases are credited to our VP Silicon Jim Miller).  


Blogs and books love to highlight successes and extract from them heuristics of behavior that, if followed, will ostensibly lead us to similar outcomes. Our collective tendency towards revisionist history in storytelling means it’s hard to get a bead on when someone was lucky, and when they were skillful.  When did they predict the outcome, or when were they skillful enough to roll with what the world threw at them?  The answer to that question may be unknowable unless, like Steve Jobs, you repeat your success over and over and I’m not sure any of us should take lessons from an extreme outlier like Jobs.

On the whole, I think the ‘mimic the greats’ approach to learning how to innovate is wrong.  I think we find success in an uncertain world by remaining flexible, a maniacal and constant focus on learning, and
learning as many lessons as we can from failures – we win by avoiding failure and adapting to our environment.  Like Arthur Dent in the Hitchhikers Guide to the Galaxy, the way to learn how to fly is to throw yourself at the ground…and miss.

So the big question for the investor or entrepreneur is when diving into the Fog of Innovation is a well-considered strategy, and when it is a shot in the dark.  On one view, choosing a path and executing down it unveils parts of the landscape we can’t see during planning.  However which path we choose will have a huge impact on what we uncover, but by definition we can’t know which path is best.

The factors the govern the selection of a path are varied, but I do think we can choose a paths that all things being equal, help us build a general capacity in marketing, engineering, sales, product design, administration, etc that can be recouped when changes do come (see Rob Ryan’s “Sunflower Model”).  Paths that involve large, highly specific expenses, should be shunned when the fog is thick and only taken when the evidence is overwhelming.


To come full circle on my war metaphor, I turn to Robert McNamara’s 11 lessons of war, many of which apply well to innovating in the face of uncertainty.  These come more from his early war experience (WWII) than his later (Vietnam).  My editorialized list is:

  • “Get the data” — Trust your intuition, but have the discipline to verify.
  • “Belief and seeing are often both wrong” – What you think is true, and what the customer tells you is true, are often both wrong.
  • “Be prepared to re-examine your reasoning” — The only thing that is certain is that you will make mistakes.  Focus on learning to recover from them.
  • “Maximize efficiency” — the more efficient you are, the better positioned you are to exploit unfolding developments and those pesky Black Swans.
  • “Never say never” — You might see a hundred ways to lose, but you only need to find one pathway to win.
  • “Rationality will not save us” — It is also important to drive our actions from a moral center, not pure utilitarianism.
  • “In order to do good, you may have to engage in evil” 

I’m unsure I like or agree with that last one, either theoretically and practically.  It brings to mind Google’s early China strategy – that a lesser evil of being complicit in state censorship leads to the greater good of easier access to information for all.  I think in the innovator context, the better lesson is that the world can get very messy and the line between practicality and morality can become fuzzy.

For example, startups are usually selling a vision of the future they haven’t yet realized and where realization is not 100% certain.  Are we lying to our customers and investors when hide the inherent uncertainty from them?  I don’t believe so; what we are really selling in early-stage projects and companies is our team’s ability to successfully navigate the fog of innovation and deliver on our promises.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s