The cloud must be decentralized

Amazon S3 has been down for several hours now and as Jonathan Boutelle (who, by the way is speaking at the Singularity Web Conference) writes, when S3 goes down, the Internet goes down. Along with images on Twitter, SlideShare, SmugMug, and a host of other sites, images on ads are also down. We've also had intermittent issues with Amazon's SimpleDB. I feel that these issues only serve to highlight the age-old danger of having all your eggs in one basket. Especially a proprietary basket. This is why I applaud Google in releasing the Google App Engine SDK as open source.

The launch of services like Amazon's EC2, S3, SimpleDB and Google's Google App Engine herald the birth of the Commodity Web, wherein web infrastructure is infinitely available and metered just like electricity, water, and gas. For the most part, we don't think about the limits or availability of commodities (as the impending ecological nightmare we've woven for ourselves would attest to, if nothing else.) But when we do try those limits, or when the systems of delivery break down (as in blackouts, for example), the extent of our reliance on these utilities becomes painfully clear.

Planning for contingencies is an important part of running a successful business. Hence, for example, we have UPS units and generators to provide (at least short term) redundancy for the general power grid. Unfortunately, the same cannot be said currently for many applications that rely on the variously-available "cloud" computing solutions for their hosting.

The problem is that we lack standards and interoperability.

If tied in to a proprietary system, I cannot change my hosting provider as easily as I can, say, switch from the power grid to using my own generator. They're not the same voltage. Heck, they're not even the same frequency.

This is why having the Google App Engine SDK be open source is of utmost importance and, I believe, a spark of genius on Google's part. They've created a de facto standard for the Commodity Web. It is only a matter of time before other vendors will offer compatible and scalable application hosting based on Google's standard. And that's how it should be.

The cloud cannot and should not be a centralized behemoth that creates a single point of failure for the Internet. The cloud must be decentralized and needs standards, even if only de facto, and I feel that Google App Engine is a step in the right direction.

If you're interested in the Commodity Web, I'd highly urge you to check out the writings of another Singularity speaker, Simon Wardley, who writes and speaks extensively on the topic.