The rise of dynamic software development methodologies such as Extreme Programming or Agile Programming, reflect the inherent dynamism of modern software design. The malleability of software, the rapid evolution of consumer and technology driven requirements, the difficulty of writing accurate specifications given all the unknowns, and the sheer complexity of the software ecosystem itself makes the ancient development waterfall from specification through execution and QA to release a hazardous and mostly futile affair.
Most software developments fail. While the situation has improved over the last decade, this remains mostly true today. Less than a third of all software projects meet their objectives in approximately the time expected. Over 10% of all projects fail without deliver anything, and most of the rest under-deliver, are terribly late, or way over budget.
This blog post is a thinking-out-loud exploration of how modern Agile methods address these problems and how my thinking is evolving with regards to how success is defined and the probability of success maximized.
The Fog of War is invoked to describe the uncertainty that permeates combat operations. This term is generally attributed to a quote by Clausewitz: “The great uncertainty of all data in war is a peculiar difficulty, because all action must, to a certain extent, be planned in a mere twilight, which in addition not infrequently — like the effect of a fog or moonshine — gives to things exaggerated dimensions and unnatural appearance.” (via wikipedia).
The evolution of an innovation or innovative company faces a similar fog. At any point in time, we extrapolate from current conditions a set of possible future outcomes and take action to try to bring the best outcome into being. In one view, we should make economically rational decisions, those that maximize our expected return over the outcomes weighted by their likelihood.
However, Nassim Nicholas Taleb’s “The Black Swan” argues convincingly us that the seeds of disruption and destruction lie in events that exist outside all reasonable extrapolations of a current state of affairs. The occurrence of a highly improbable event is, in fact, inevitable.
Since we can’t know anything for sure, innovators have to operate at a completely different level than analysts. It’s a chess game where the rules can change mid-play forcing a complete re-evaluation of strategy. For example, the iPhone was a Black Swan to the cellular industry, as this analyst’s perspective shows. This dynamnic is why you should ignore Forrester and other analysts – they tell you what the world will be like if nothing surprising happens, but surprising things always happen.
At my first startup company, Silicon Spice, we were extremely disciplined in how we structured internal communications, identified and followed technology standards and incorporated automatic testing into all our technology development efforts. The company-wide discipline we established allowed for an unprecedented pace of product innovation in the rapidly evolving communications market of the late 90’s. Peaking at only 120 employees, we constructed a complex set of technology targeting telco equipment vendors including reference voice switching boxes, a 2.5 Watt 21-core DSP processor, companion ASIC devices, a vector compiler, a real-time OS, a fully-featured set of voice processing software and a complete suite of development tools for our platform. A few people in key roles were able to spot opportunities that spanned numerous product sub-teams and we were able to quickly implement coordinated design changes across our home-grown technology stack.
A lead engineer from Texas Instruments, our primary competitor, once commented to me that they couldn’t believe that we had build our product in only 3-4 years with 100 people; I didn’t have the heart to tell him the product they were evaluating for acquisition was built by an average of 50 people over 16 months! We seized 70% of the carrier-class market from them in the 18 months following our acquisition.
During the work day, interrupts from e-mail or social media can lead me to compose e-mails or get sidetracked, 20 web searches away, hot on the hunt for answers to interesting questions raised by the latest forwarded content. Today I discovered a useful, free application called SelfControl (http://visitsteve.com/made/selfcontrol/) that blocks your computer from accessing certain domains such as Facebook.com, Twitter.com, or incoming mail servers. It blocks these connections for a fixed period of time and it is nearly impossible to disable once you’ve started the clock. It is just as if I were going into a meeting where I wouldn’t normally be checking my e-mail, but this meeting is with my own mental creativity team.
RescueTime has a similar feature called “Get Focused” which uses their database of distraction ratings to turn off all sites you have labeled as Very Distracting. Unfortunately, there isn’t yet a way to add your e-mail to this service so SelfControl provides a useful extra bit of filtering.
(It’s really too bad this doesn’t apply to my spare computer, my iphone, ipad or my wife’s computer…but technology only takes us so far in supporting behavior change)
Over the past year, I have had the pleasure of advising the non-profit organization Lybba. Lybba’s vision is closely aligned with the work we’ve been doing at New Media Medicine. This month I accepted a one year Research Fellowship with Lybba; the objective is to complete and apply my PhD research in the context of Lybba’s ongoing projects. Chief among these projects is the Collaborative Clinical Care Network (C3N) which I wrote about a few weeks ago.
I’ve been extremely impressed with the scope of their ambition, the quality and breadth of their team and partners, and the concrete projects they’ve chosen to invest time in. Continue reading