Cloud Computing: The Truth About What Runs on Amazon

You hear lots of talk that Amazon's on-demand public cloud offering is used mostly by small companies, but that's not the reality, says CIO.com's Bernard Golden. And as happened with open source, developers are leading the charge.

I continue to encounter an interesting phenomenon regarding cloud computing as I speak at conferences, present to IT groups, and talk to businesspeople interested in the subject. Most people recognize the importance of cloud computing, acknowledge the relevance to their environments, and describe their initiatives.

However, one thing I often hear is that while Amazon is the clear leader in cloud computing, "it's mostly used by SMB organizations." In other words, public cloud computing is not being used by big enterprises. And, people add, even if large companies are using public clouds, it's for "unimportant" applications. The canonical example offered by people making this observation is that "no one is moving SAP to the cloud."

What I find interesting about these discussions is how poorly they match my experience and observations. From our experience, Amazon and its fellow on-demand cloud providers like Rackspace are being used by large organizations quite a bit, for important applications, and certainly the use is increasing.

So why the disconnect between actual facts on the ground and perception?

Who Led Open Source Charge? Developers

I've long thought that the rise of on-demand cloud computing reminds me a lot of the adoption of open source, and that perspective was reinforced by a couple of blog posts I read this week by Stephen O'Grady of the analyst firm RedMonk.

The first is his discussion (titled "Meet the New Kingmakers, Same as the Old Kingmakers") about a Forrester report analyzing open source adoption within enterprises. The report found significant use, buttressed by survey results and other facts. In his post, O'Grady mildly criticized Forrester for coming to a conclusion that RedMonk did four years ago.

RedMonk, he explains, believes that developers are the true decision makers in organizations and from its interactions with those type of folks, RedMonk knew years ago that open source was being used in a big way. Forrester, he notes, surveyed CIOs and senior IT decision makers -- that is, management. He quotes a post by former Red Hat sales exec Billy Marshall that "CIOs are the last to know," meaning, IT organizational decisions are actually made bottom up (i.e., by developers), and that senior management perception lags reality by a significant margin.

O'Grady states: "We are founded upon the idea that developers are the single most important constituency in technology. Open source dramatically lowers the barriers to adoption, such that developers may build upon what they want rather than what they're given." Both O'Grady and Marshall emphasize that developers use open source because it makes doing their job easier.

The implication for organizations, as Marshall's anecdote illustrates, is that decisions made by developers create commitments for the organizations they are part of -- commitments that the organization does not recognize at the time they are made by the developer and may, in fact, be decisions that, had the organization understood them at the time they were made by the developer, it would have eschewed them. The result is that two or three years down the road, these organizations "discover" technology decisions and applications that are based on choices made by developers without organizational review.

This may account for the curious lack of respect given Amazon on the part of IT organizations and vendors. O'Grady addresses this in a second post titled "Hiding in Plain Sight: The Rise of Amazon Web Services." In it, he primarily addresses the fact that most technology vendors evince little fear of Amazon, preferring to focus on private cloud computing environments. He attributes this, in part, to the vendors' desire to keep traditional margins rather than descending into a pricing battle with Amazon.

I might attribute it to a different factor: Vendors primarily seek to talk to senior management, those who control budgets that pay for the vendors' products—and, as we've just noted, those managers often miss the reality of what developers are actually doing. Consequently, they won't be telling vendors how much public cloud is being used, and the vendors will respond with the time-honored "we don't really see them much in competitive situations" (I used to hear this a lot from proprietary software vendors about the open source alternatives to their products).

Make no mistake about it, developers are using public cloud computing—a lot. We certainly see it in our daily business interactions. The high level of usage (and its somewhat clandestine nature) was illustrated at a conference earlier this year when one CIO noted that he had gone through the expense reports turned into him for reimbursement and found 50 different cloud computing accounts being used by developers in his organization. The reason? It's a lot easier for a developer to obtain computing resources from a public cloud provider than to undergo the extended waits typical of the existing compute environments.

The upshot of all this is that today decisions are being made by developers about use of cloud computing that upper levels of the organization are unaware of (the CIO just cited is undoubtedly an exception to the general rule). A corollary to this is that IT organizations will discover new public cloud-based applications in a year or two that they had no idea were being placed in those cloud environments.

The ramifications to this are significant:

1. Organizations will be taking on operational responsibility for applications located in cloud environments for which they are unprepared.

They will need to quickly get up to speed on the cloud environment and learn how to operate an application within it. This will include the need to retrofit monitoring, security, and management into applications, as developers often fail to address these areas because they are not required for application functionality. We refer to this responsibility migration as the "cloud boomerang." You can see a video we created that describes this phenomena in greater detail here.

Action: You should have your operations people evaluate how to take over operational responsibility for external cloud applications.

2. Compliance requirements will need to be analyzed for these "discovered" applications.

Changes to the applications, or even migration from their initial cloud hosting locations, may be required to bring the applications into conformance with the requirements of governmental and industry laws and regulations. Given the significant semantic differences among IaaS public cloud provider functionality, this will be more challenging than item number one, above.

Action: You should set up a remediation team with skills in these different areas so that you are prepared to respond when an unknown application running on the public cloud surfaces.

3. Investment patterns of IT spend will be modified in a stealthy fashion

This change will happen not deliberately, but unconsciously. While public cloud computing is relatively inexpensive (the complaint of vendors as described in the second of O'Grady's posts), the costs mount up as applications scale and as applications multiply like rabbits (as they inevitably do—that was certainly the pattern with open source; as one project achieved success with open source components, other projects embraced the solution as well). At some point, someone will notice that a bunch of money is being spent on public cloud computing and figure out that it represents some significant fraction of total IT spend; at that point some will call for the cloud spend to be curtailed, while others will assess the benefits being achieved by those cloud applications and conclude that maybe the cost cuts should fall on the internal IT budget.

Action: You should have your finance people looking through expense reports and have others sniffing around to see what use of public cloud computing is being done in other parts of the organization. Don't be the last to know.

And by the way, that shibboleth about "no one's going to run SAP up in the cloud"? In a somewhat ironic milestone his week, SAP announced that a portion of its on-demand offering has been migrated to AWS. Perhaps a new example of how public cloud computing isn't sufficient to meet the real needs of enterprises needs to be found.

Bernard Golden is CEO of consulting firm HyperStratus, which specializes in virtualization, cloud computing and related issues. He is also the author of "Virtualization for Dummies," the best-selling book on virtualization to date.

Follow Bernard Golden on Twitter @bernardgolden. Follow everything from CIO.com on Twitter @CIOonline

Named by Wired.com as one of the 10 most influential people in cloud computing, Bernard Golden serves as vice president of strategy for ActiveState Software, an independent provider of CloudFoundry.

Copyright © 2010 IDG Communications, Inc.