December 2009 Archives

As we enter 2010, I am going to focus on a series of articles to define the cloud storage opportunity and the business issues for the enterprise.  First, there are some "universal truths" that we need to better understand and define. 

The growth in unstructured data will continue, unabated.  We all know and understand that.  The issue is how to manage this phenomenon, while operating with the assumption that the growth will likely accelerate.  Since the growth is driving increased costs, the enterprise is on a continuous search to improve the way they can cost-effectively manage this growing data.  

Data may exist on removable media, on PCs and PDAs, on various servers within the organization, at data centers, at remote facilities, and potentially at various outsourced service providers.  The data may range from employee personal information (and even personal information from the employees associates) that is not associated with the needs of the business to non-confidential and confidential business information, some of which may be highly critical.  Disparate policies will need to be applied to the data ranging from no control to extreme control.   Of course, there will be the existence of  multiple versions of files adding to the total storage and further exacerbating the challenges of management.

There are many potential solutions to the problem as stated above, and most of them involve some sort of additional controls, policies and restrictions that control the proliferation of data and make it more orderly and secure.  These solutions are then combined with additional focus on reducing storage costs by staying aligned with new storage technology (which continues to reduce costs of storage), and the cycle repeats, endlessly.  In each cycle, trade-offs associated with costs, availability, security, access, restrictions occur, and rarely is there a "perfect" solution.

Is cloud storage a possible solution to the issues as surfaced above?  Is it a discontinuity, a departure, from the "business as usual" cycles associated with ongoing, incremental and continuous storage improvements when new technologies are introduced as they can be accommodated?  

Let's start with discussing cloud storage and its various capabilities.  Note that we are talking about a storage cloud that is housed at the enterprise data center, not a storage service provider.

(1) First, centralize the storage problem:

Cloud Storage addresses the necessary size and scale of unstructured data growth in the enterprise.  Generally, highly scalable file systems, including newer object based systems, provide the ability to manage incredibly large numbers of objects (objects of all sizes) in an efficient fashion.  This is combined with low cost commodity storage devices and servers.  Then a centralized storage pool is ready for use.  It is generally easy to add additional storage to this pool, and both backup and disaster recovery schemes are in place.  So, the first well known method of problem solving that cloud storage utilizes is "centralization."  Let's get a solution in place that we know can scale to the size of the data needs of the enterprise.
 
(2) Second, make it easy to use:

You can't use it if you can't get it, and this is where the topic of "thin provisioning" emerges.  Thin provisioning just means that it is easy to get a storage account (whether I am an individual user or an application / server) and I can get it quickly, no matter how much I need (in theory).  Further, as my storage needs increase, it is easy to get more - quickly.  There are issues like accounting for storage; managing growth and billing for it that also surround the notion of thin provisioning. 

Access is another big topic that surrounds ease of use. The enterprise has multiple needs here.  Legacy applications, utilizing file access methods like CIFS or NFS, will want to utilize the storage cloud.  New applications, written to REST Web services APIs, will also want to coexist.   Finally, individual users will want access from all their device types, including PCs (Windows and Mac, Linux), the Web, and PDAs.  All of this access manifests itself in interesting ways, including identity management of the credentials associated with using the service, bandwidth requirements for accessing the service from many diverse locations, and geo location of data (i.e., if you have several locations where the cloud data is kept, how do you decide which location to use?).

(3) Third, sync your files to the cloud:

Now that you have cloud storage, you ought to think about backup and sync to the cloud.  These two applications are different but somewhat linked.  Sync to the cloud can be used for both cloud loading (getting the data from the device to the cloud, in a background way so that the latency will not be a problem) as well as keeping a current copy in the cloud, but using the local copy on your device (the best of both worlds).  Since your most current copy is in the cloud, it is your backup copy.  Sync is also a solution for keeping files "sychronized" between devices and the cloud, so you always have an authoritative source of your file stored in the cloud.  Of course, all this is based on having cloud access from any device, anywhere (see number two, above).

(4) Fourth, create new, higher impact applications with programmable storage:

Programmable (using http, SOAP or REST APIs) access to storage is the next big revolution in storage.  Tagging, sharing, collaboration, easy search, easy and secure access and multiple views make creating new, high impact applications easier than before.  Take advantage of new functionality that is easily delivered.  Create applications that rely on your data and data that is external to the enterprise.  Develop these applications quickly and at lower cost.  If all you want is cheaper storage, you may be able to get by without a cloud, but without this capability you are missing the revolution that is upon us.

(5) Fifth, secure your cloud:

In my own survey of the industry, security is the major issue on the minds of the IT department evaluating cloud storage for the enterprise.  Several different aspects of security come into play.  Many of these issues are most often associated with using a multi-tenant storage cloud from a storage service provider. Nevertheless, four major security issues prevail before we even begin to consider the issues of going to the cloud at a service provider.

The four issues are:  physical security, unauthorized access, data loss (disaster or device failure related) and bit rot (a subset of data loss, granted).   All of these issues are no different than what you face with your traditional shared storage solutions and most of the solutions are similar.  Your current IT physical security solutions apply to an enterprise hosted cloud.   The identity management policies and practices associated with creating and maintaining account credentials address unauthorized access, just as they do with your current data management practices. Encryption can provide additional protection from unauthorized access. As a matter of fact, the security issues are already in play with your current storage methodology, so nothing new here, unless you move to a service provider hosted cloud (more on this later).

(6) Sixth, lower the cost of storage:

Cloud storage delivers the benefits as discussed in items one through four above, while requiring similar security to current storage activities.  How does it address costs?  First, cloud storage solutions generally allow for using commodity hardware, very scalable file systems, and highly automated provisioning and management solutions.  So, the hardware price equation of differentiation and premium pricing is disrupted.  True, the software doesn't come cheap, but remember that the public cloud storage services are "making the market" and the combination of commodity hardware, environmentals, and enabling software (file system, management and middleware from one or more suppliers) is meeting the external marketplace pricing.  Here is a simple model you should use (all figures expressed in cents/GB/Mo):

Commodity Hardware depreciation                                      $  .02
Environmentals  (data center, power and cooling)                     .02
Management (primarily people resources)                                .02
Enabling Software                                                                  .03 
Other                                                                                    .01                           

Total costs:                                                                      $  .10 (10 cents/GB/Month)

This represents a significant saving for a solution that provides all the capabilities that cloud storage delivers.  What's the catch?  Well, not every type of application and use case for unstructured data is ideally served by cloud storage.  However, many are, and the exceptions should be dealt with as one offs.  The real catch is not taking advantage of this new technology, and all the opportunities it offers, for lowering cost while delivering improved capabilities to end users and applications around the enterprise.

My next post will discuss hybrid, private and public cloud storage offerings, and where savings and security can drive significant benefits for enterprises who take advantage of the cloud storage offerings of service providers.

CloudStorageStrategy.com welcomes OpSource CEO Treb Ryan for an in-depth interview on cloud computing, from the perspective of the service provider.

NOTE: OpSource is a customer of Mezeo Software, the underwriter of this blog.

What are the opportunities you see in the cloud computing space, both for OpSource and your customers, and what impact has the downturn had on this?

It's interesting, but when people talk about cloud computing, they immediately go to the downturn and pricing - and cost being the big driver.  There's no question that cloud computing is cost effective, and it's accelerating adoption many times over, but what we're really seeing is something much more fundamental - a generation of users who are entering the workforce who've been using cloud computing all along; they've grown up on the Internet, and their interface to technology has always been through the Internet. 

As a result, this "Cloud Generation" has clear expectations of how technology should work:

1) it should be immediately available,
2) you do a search and get going,
3) it should be very flexible,
4) you should have ubiquitous access - anytime, anywhere,
5) sharing and collaboration - the expectation to collaborate and share anything they are working on.

This is not a generation which distinguishes between work data and home data - like my generation did. They've grown up with the concept of APIs and communities that grow around them; for instance, we see programmers who have grown up with Google and Facebook APIs, and now they expect that kind of thing in their work applications as well. So they're coming into the workforce and driving change in the workplace. They see technologies like client-server applications or hard-coded storage arrays pretty much the same way my generation saw green screens, mainframes, and mini-computers - as dated, inflexible, technology - hard to use, without nearly the power of cloud-based systems. So they have the day-to-day experience of the "consumer cloud" which they're now driving into business applications as well. 

To the Cloud Generation of programmers this means anything they can interact with on the Cloud they can program to through APIs. The idea of infrastructure being an item that can be addressed as part of the application, instead of something the application lays on top of, is a radical concept.  It has allowed not only for innovative applications, but also for true elastic computing making the Cloud environment even more flexible.



Great Cloud offerings have great communities around them. This is the aspect of Cloud computing that is so often missed - and even scoffed at - by the IT folks who think it's all about virtualization. One of the biggest gripes about Cloud computing is that support is done by the Community and not the vendor. While most will agree that far more proactive vendor support is necessary for Cloud computing, Community support is just as critical. For questions of configuration and usage tricks, the Community is a far better source of information than some call center employee with limited access. Often the Community devises more innovative solutions than the vendor ever could. And in addition to support, the Community can create third-party add-ins that make the Cloud even more useful.

The downturn has accelerated adoption from the top down as well.

We're seeing executives who have become enamored with this idea of the cloud - because of the ability to turn capital expenditures into operational expenses - and are pushing cloud computing into their organizations.  The CEO of one of our customers went so far as to tell his technical people - "now can you finally start using the cloud so I can get the board off my back?"

So, for different reasons, we have both top-down and grass-roots support for cloud-based applications, which makes this very interesting to say the least.

Which customer segments do you see leading the way in adoption?

Obviously, our traditional focus has been on ISVs and start-ups coming into Software-as-a-Service, business applications in the cloud, and we're seeing continued adoption of cloud infrastructure by those segments, but what has been interesting is that now that we offer the ability for any company to buy and use cloud infrastructure for any type of application, we're seeing a much broader spread of usage and adoption. Beyond the enterprise we also see widespread adoption by systems integrators, consultants, and VARs - upto 40% of our customer base - all without us targeting that segment at all.

How does OpSource differentiate its cloud offerings from other service providers?

We offer the best of the public cloud, combined with enterpise security and compliance, performance guarantees, and enterprise controls.

For instance, we offer:

• easy online sign-up & purchase with infrastructure provisioning in minutes
• pay by the hour and only for what you use, with no commitment (or purchase a monthly plan for a discount)
• a rich online community to share and collaborate with peers; get third party add-ins, images and configurations
• a web interface plus complete set of APIs

On the straight cloud, we provide a lot of the more robust, enterprise tools than you see from more consumer-based providers like Amazon, for example.

We focus on three different areas:

1) Security and Compliance: we provide a much more secure environment, because Opsource provides every customer with a Virtual Private Cloud within the public Cloud, allowing them to determine their own degree of public Internet connectivity. We also provide:

• Unique customizable security for firewalls
• VPN administration of all servers
• Unique username/password for each administrator
• Audit logs of all environmental changes
• SAS 70 audited
• 100% uptime SLA

2) Performance: we offer a multi-tier architecture with guaranteed latency in-between systems, sub-millisecond access time, industry standard technology, like VMware, instead of open-source, because that's where enterprise is comfortable.  Our 24/7 suppot also makes a diffence.

3) Control: today's cloud environment are single user environments, one user name and password, which is fine for individuals, but not so useful for the enterprise. We offer the ability to provision multiple users, do things like cross departmental billing, execute policy based control - which user can do what - and finally link all that back though an API to your existing management systems. So you can control how your users use the cloud same as you do your corporate datacenter.

So do you see any links into these large companies where they need to use ITIL for systems management?

Absolutely. OpSource has always focused on compliance as a major issue for our SaaS customers, eveything from SAS 70, PCI to European Safe Harbor, and even industry-specific ones like HIPAA, or government-specific certification, but in the cloud, we think about sophisticated  management techniques like federated authority and single sign-ons, and things like ITIL - while it's still in its infancy, it's shocking that most providers don't even have the ability to give their customers the critical capability to have more than one person manage the cloud for them - because they have a single user accounts. So while you can institute more sophisticated IT governance regimes like ITIL with the OpSource cloud, we give IT the capability to manage who does what, and track who did what, even if they aren't ready for something like ITIL.

So IT gets to do their own provisioning?   
  
Yes. So you want to know who provisioned what, how much it costs, and we give them that visibility instantly across their entire user community.  That way there are no surprises or charges they aren't aware of. It sort of reminds me of the controls I had to put in to alert me to my daughter's texting costs - so I'm aware of the charges before they get out of hand! I just blogged about this issue.

That's why you say that OpSource is what Amazon wants to be when it grows up... 

Absolutely.

And that's how you respond to cloud critics - the ones that say that the Cloud is not yet ready for the enterprise.

There are large parts of the cloud that are not yet ready for the enterprise. The cloud is still young, and it would be like asking that first 286 PC to run all of your corporate financials. However, a lot of these issues around enterprise adoption like security and compliance have been addressed, and are being taken care of, so as the cloud becomes more robust, we'll see increased adoption. We're seeing enterprise-level capabilities come to market that did not even exist six months ago.

We have just signed a partnership agreement under which OpSource will resell Gomez's Web performance management solution to our enterprise customers as well as use it to validate and monitor our own cloud performance service level agreements (SLAs). Through this partnership, we'll bring powerful performance monitoring to cloud computing, making it easier and more compelling than ever for enterprises to justify bringing their applications to the cloud.

Do you see infrastructure elements like storage growing now?

For true, full use of the cloud, we have to have the ability to access storage, go though the APIs to get to it, and give our customers a range of storage solutions, including cloud storage based on the specific application or need. We're giving our customers the widest range of choices.

What about agile programming? I heard you use agile methods to improve the customer experience.

Agile programming methods have helped us with not only development, but compliance and security as well. We talk to our customers to see how they are using our cloud offerings though our community, and we learn what's important to them.

We also test our offerings by having two programmers work on the same keyboard - literally  - one with the user story - so they can make sure that the customer is getting the exact functionality they need.

It's agile customer service.

Can you tell us a bit about your enthusiasm for composite applications (corporate mashups) and how they help your platform?

Of all the phenomenon in the cloud, we see the need for anytime-anywhere access and the idea that anything I can interact with I should also be able to program to.  So when Facebook enthusiasts start working in the enteprise, they bring their enthusiasm for integration as well.

So we see things in the cloud like direct access to the infrastructure as part of the application, which allows for all sorts of flexibility and robust usage.

We see real-time reporting applications of every kind you can imagine.  I myself am addicted to checking on everything that's coming out of our billing and customer systems tied into our Salesforce tabs.  So I'm always checking on the business in real-time via my iPhone.

I say this a lot, but integrating SaaS is a huge issue for today's enterprise. OpSource Connect can help SaaS companies -- of any size -- overcome integration hurdles and break out of the SaaS-only box. This speeds up adoption of SaaS in larger enterprise environments, opening the door for on-demand companies to cultivate business with large systems integrators. Plus, I'd say we're the only company providing Web operations from the ground up, addressing operational infrastructure, application management, and business operations. Today, integrations are expensive and one-to-one. For instance, while you can currently integrate your application with Google Maps as a composite application, OpSource Connect lets you integrate your app with many others, using just one platform. You can integrate your application with, for example, SAP, salesforce.com, Intuit QuickBooks, NetSuite, and a host of other SaaS and legacy applications. 

Everything is much more dynamic today, and programmers expect that. 

Here's an interesting read on some of the issues that traditional file systems face which can now be overcome with an object-based system. 

According to the author, Beth Pariseau:

Unstructured data is expected to far outpace the growth of structured data over the next three years. According to the "IDC Enterprise Disk Storage Consumption Model" report released last fall, while transactional data is projected to grow at a compound annual growth rate (CAGR) of 21.8%, it's far outpaced by a 61.7% CAGR predicted for unstructured data.

This is a direct result of the digital content explosion. 

Robin Harris, senior analyst at StorageMojo observes:

"There are going to be extreme amounts of data as things like digital video and mobile networks grow; in five years, pretty much every phone will be 'smart,'...All of us storage geeks agree on that, and different people are beginning to visualize what that kind of growth needs in terms of storage infrastructure."

The article makes the case to "Think APIs, not files."  In essence, the point is as follows (as explained by Harris):

"File systems make less sense over time as the amount of data grows. Architecturally, it makes more sense for each file to have a unique 128-bit ID and use an Internet-like system for locating that file; a URL points to an address and there are files at that address, and object-based storage interfaces are essentially operating on the same principle."

The result, writes Pariseau, is that "with an object ID replacing a file name, more extensive data can accompany an object than the simple 'created,' 'modified' or 'saved on' fields available in traditional file systems. Thus, detailed policies can be applied to objects for more efficient and automated management. Without NFS or CIFS to serve up files to applications, object-based storage systems need to replace that layer of abstraction between raw blocks of data on disk and files that applications can recognize. Today's object-based systems use standard APIs such as Representational State Transfer (REST) and Simple Object Access Protocol (SOAP), or proprietary APIs to tell applications how to store and retrieve object IDs.

One of our key decisions when we designed Mezeo was the adoption of object-based architecture for cloud storage.  Mezeo can use traditional file systems as object based systems to deliver cloud storage, and can also expose cloud storage as a traditional file system (even though it has objects underneath the covers, or as an object system).  This reflects our view that there will be a prolonged period of co-existence followed by a migration to object based systems.

If you'd like to learn more about how Mezeo offers an agnostic storage services platform for storage service providers (SSP), take a look at this paper (registration required) by the same Robin Harris: Building a scalable shared file infrastructure. The paper gives service providers an introduction to:

• Cloud storage applications and customer drivers
• Mezeo's storage architecture and options
• Basic shared file storage reference designs
  

In the paper, Harris says that there are multiple ways to build highly scalable storage for cloud storage applications. He tells us how SSPs can differentiate their offerings:

The Mezeo platform allows the special features of the storage to be delivered to customers, while giving SSPs a powerful platform on which to build a business. Understanding what storage choices will better meet target market needs is a critical success factor. SSPs can differentiate their cloud services by careful selection of back end storage systems. The Mezeo platform gives SSPs great flexibility. Understanding how to use that flexibility will be key to growing a successful cloud storage service business.

Harris also presents five reference configurations (see diagrams below) in the paper, which vary in performance, availability, scalability, self-management and, of course, cost.

CONFIGURATION # 1: NEXENTA





CONFIGURATION # 2: PERMABIT




CONFIGURATION # 3: PARASCALE




CONFIGURATION # 4: Red Hat Enterprise Linux




CONFIGURATION # 5: NetApp




DOWNLOAD:
Robin Harris' Building a scalable shared file infrastructure >>

A lot has been written about the reluctance of many to use the cloud for their mission critical applications, and in particular, the enterprise.  While this may be a popular topic from the perspective of many, the Cloud is most certainly seeing a significant increase in adoption as more and more companies build their SaaS offerings on platforms from Amazon, Google, Force.com and Microsoft.

Platform as a service (PaaS) is defined in Wikipedia as "the delivery of a computing platform and solution stack as a service. It facilitates deployment of applications without the cost and complexity of buying and managing the underlying hardware and software layers, providing all of the facilities required to support the complete life cycle of building and delivering web applications and services entirely available from the Internet--with no software downloads or installation for developers, IT managers or end-users. It's also known as cloudware."

In general, PaaS offerings include workflow facilities for application design, application development, testing, deployment and hosting as well as application services such as team collaboration, web service integration and marshalling, database integration, security, scalability, storage, persistence, state management, application versioning, application instrumentation and developer community facilitation. These services are provisioned as an integrated solution over the web.

We just saw another Cloud validation as three established ISVs announced offerings on platfoms from PaaS providers.  Both BMC Software and CA announced their intent to offer apps built on Force.com next year. Quest Software also announced the launch of its first set of Software as a Service (SaaS) Windows management solutions on Microsoft Azure.

Note also the following examples of SaaS services built on AWS, Google AppEngine and Force.com.  This "explosion of entrepreneurship"  further the case that platform-as-a-service is rapidly gaining acceptance in the market.



What we are witnessing is a boom in platform-based businesses, made possible by the cloud model: pay-per-use, instant scalability, and the elimination of up-front capex costs.