A colleague recently emailed an argument that higher level computing providers like Heroku are doomed due to the “Fundamental Law of Another Mouth to Feed” where the margin requirements of the services these providers are reselling means they will eventually lose to someone without that margin in their supply chain. As with any asymptotic complexity argument, this is sound theoretical thinking, except it ignores the significance of constants in the equation. Following is a slightly cleaned up version of my response.
I used to hear a very similar argument in the early-90’s with regards to owning your own semiconductor fab – how can one compete as a semiconductor company if the cost of goods includes a contract fab’s margin? Few chip makers own their own fabs anymore, unless they’re Intel or a speciality manufacturer for things like memory or analog componentry.
The same argument was used in the late-90’s for the use of application specific chips (ASICs) vs. field-programmable gate arrays (FPGA) in networking equipment. ASIC per-unit prices were significantly cheaper, but the fixed cost of designing an ASIC kept going up while FPGA fixed and variable costs rapidly declined. More importantly, the flexibility of FPGAs became a necessary feature due to increasing system complexity making the variable cost mostly irrelevant. (To be fair, Amhdal’s law was at play too).
I suspect that virtualized computing platform providers will evolve along a similar trajectory; the fixed costs of reproducing increasingly standardized industrial software capability keep going up, while simultaneously the financial benefit of faster time to market dwarfs any margin benefit over the life of the product.
The proper analogy for Heroku is not an ISP, but a value-added reseller. Clients for whom a commercial cloud like Amazon’s Web Sevices (AWS) makes economic sense still face a large number of additional overhead costs to handle system administration, application deployment, replication, backup, database administration, network security, host security, etc. These are not core competencies, and also not obviously incremental features that AWS will add in the near future. The Herokus of the world charge for providing and managing a higher level abstraction with performance and reliability guarantees. They then simply pass through the underlying AWS costs. For Heroku’s customers, the net costs of the needed computing infrastructure are far less, particularly during the project startup phase.
As a heavy AWS user at Compass Labs, I can attest that exploiting the spot instance bidding market they provide requires a great deal of application infrastructure to be built — the effective MTTF of a node goes up by several orders of magnitude and machine shut-down can become a daily/weekly rather than exceptional event. Higher level platform abstractions can amortize the investment necessary to exploit these price reduction opportunities. This implies that Heroku in the long run is likely to pass through less AWS cost than if I bought and managed the resources directly. I expect that most application projects and nearly all startups that aren’t directly selling hardware-as-a-service will have little economic basis for building software on top of captive physical hardware or even to spend too much time at the Unix terminal prompt.
The big exception I see to this today are high-performance and network-intensive compute clusters; AWS and others still suck at the economics of these applications, probably by an order of magnitude. (Aside: breaking abstractions is the key to performance. Cloud Computing by definition puts many abstractions between you and the silicon) As we collectively standardize on higher level compute abstractions like Hadoop, this will change too. The other big discontinuities where captive resources may still make sense are applications that emphasize system-level properties like data security and data provenance where tight control over the entire pipeline is paramount.
I think what has changed, and why “New Thing Hype” may not be the whole story of Cloud Computing are the physics and economics of complexity. The asymptotic cost consequences of the margin in the middle remain, but the constant costs to properly exploit that hardware are increasing rapidly due to complexity, just like they did in semiconductor manufacturing or ASIC design. The project scale at which you can justify the fixed costs of building and managing a captive data center is increasingly rapidly.
Hmmm…can I come up with a relevant tie-in to healthcare? Here’s a related article which may spark another post in the near future.
Frumious Abstractions 
<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><span class="Apple-style-span">Hi Ian, didn’t understand a word but retweeted – attention grabber for son </span><span class="Apple-style-span" style="color: rgb(51, 51, 51); font-family: HelveticaNeue, ‘Helvetica Neue’, Helvetica, Arial, sans-serif; font-size: 14px; line-height: 18px; "><a class="account-group js-account-group js-action-profile js-user-profile-link" href="https://twitter.com/#!/Aloisius" data-user-id="14885927" style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 14px; font: inherit; vertical-align: baseline; color: rgb(153, 153, 153); text-decoration: none; outline-width: 0px; outline-style: initial; outline-color: initial; "><strong class="fullname js-action-profile-name show-popup-with-id" style="border-style: initial; border-color: initial; text-decoration: none; outline-width: 0px; outline-style: initial; outline-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 14px; font: inherit; vertical-align: baseline; font-style: inherit; font-weight: bold; color: rgb(51, 51, 51); ">Jordan Mendelson</strong><font class="Apple-style-span" color="#999999"><span class="Apple-style-span" style="border-style: initial; border-color: initial; font: inherit; outline-style: initial; outline-color: initial;"> </span></font><span style="border-style: initial; border-color: initial; color: rgb(153, 153, 153); text-decoration: none; outline-width: 0px; outline-style: initial; outline-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 14px; font: inherit; vertical-align: baseline; ">???</span><font class="Apple-style-span" color="#999999"><span class="Apple-style-span" style="border-style: initial; border-color: initial; font: inherit; outline-style: initial; outline-color: initial;"> </span></font><span class="username js-action-profile-name" style="border-style: initial; border-color: initial; text-decoration: none; outline-width: 0px; outline-style: initial; outline-color: initial; padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 12px; font: inherit; vertical-align: baseline; color: rgb(153, 153, 153); direction: ltr; unicode-bidi: embed; "><s style="text-decoration: none; color: rgb(204, 204, 204); ">@</s><b style="padding-top: 0px; padding-right: 0px; padding-bottom: 0px; padding-left: 0px; margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px; border-top-width: 0px; border-right-width: 0px; border-bottom-width: 0px; border-left-width: 0px; border-style: initial; border-color: initial; font-size: 12px; font: inherit; vertical-align: baseline; font-weight: normal; ">Aloisius</b></span></a>. </span><span class="Apple-style-span"><div>have a cool idea to explore with you. VA exec told me perfect for VA funding – want you to run.<br><div><br><div><br><div><br></div><div><br><div><img id="28a7a443-87c3-4096-9195-a056aeabe007" height="227" width="487" apple-width="yes" apple-height="yes" src="cid:5A41D795-8589-43D9-A097-4883A0089186"><br><div><div><br><div><div></div></div></div></div></div></div></div></div></div></span></body></html>