Storage In The Private Cloud

By now, more than a few people have been exposed to -- or commented -- on the growing discussion around private clouds in large enterprises.

I can't claim there's widespread agreement -- yet -- but I see people starting to orbit the core ideas in a very celestial manner.

With a private cloud as a reference model, now can have an extended discussion around what might need to change to support this evolving model: servers, networks, virtualization, management, security, etc.

But this time, I'd like to share some thoughts about storage in this new world.

Before We Get Started

Undoubtedly, there are going to be many cloud models to choose from: consumer clouds, SaaS clouds, content clouds, uber-clouds from Amazon and Microsoft, and more.

But many of us believe that none of these directly address enterprise IT's need to exploit the attractiveness of clouds yet retain control, smooth migration, flexibility and choices, etc.

Hence the proposal that private clouds end up being the dominant model for enterprise IT.  Indeed, I've met a few brave customers who are doing some of this internally, and are passionate about moving forward.

So, What About Storage?

First, you'll probably observe that the style of IT being proposed is more of a dynamic, just-in-time variety rather than static and preprovisioned.

This implies that storage needs to behave the same way -- certainly for capacity, but -- more importantly -- for delivering service levels.

When we throw a containerized application into a VMware environment, it can automatically and dynamically flex server performance up and down depending on what's needed. 

Storage will need to do the same thing -- dynamically flex service levels up and down as needed -- whether that's for an entire application's data store, or -- more ideally -- for the different parts of it independently.

If I did a product dive into the EMC portfolio, not only would you see extensive choices for various tiers (enterprise flash, SATA, spin-down, etc.) but more and more powerful capabilities to dynamically alter service levels while in-flight.  More to do, but you can see the foundations.

Second, the orchestration model changes. 

It's pretty clear to me that it's not useful to think of "storage management" the way we do today.

Certain attributes (performance, availability, recoverability, security, etc.) will be ideally defined and orchestrated external to the storage domain.  Ideally, you'd click a few additional boxes when you filled out the form for the new virtual machine, right?

Sure, we'll need tools that look at the aggregate picture of the storage resource -- but more and more functions will make sense to think of in terms of closely coordinating with server and network functions.

Back again to the EMC portfolio, you've seen already the extensive integration with things like SRM, ControlCenter, Smarts, etc. -- more to do, but the theme is very observable.

Third, geography will become a very interesting conversation.

If you look at what VMware does today with DRS (dynamic resource scheduling), it basically pools and load-balances clustered server resources, and does it well.

It's not hard to imagine the strong desire to do this at progressively larger and larger scale, and incorporate longer and longer distances as part of the cluster complex.

Which brings up a real physical constraint -- it's one thing to dynamically move compute images around; it's entirely another to dynamically move around the associated information around with it.  And, in many computing environments, they need to move together to provide the application service levels that the enterprise demands.

If you think for a moment of what EMC Atmos does for content at a geographic scale, you can see some of this thinking already, but in an entirely different context.

Certainly an interesing problem, no?

What Will We See In The Future?

It's interesting to note -- at least to me -- that each of these capabilities will be useful regardless of whether or not you believe that the private cloud will be the emergent model for enterprise IT.

Dynamic tiering of storage will be useful, period.  As will integrated provisioning of virtual machines alongside their required storage services.  And, of course, anything that moves data around dynamically is going to be popular, as we've seen with Atmos so far.

So, in one sense, we're going to have many of the required ingredients for storage in the private cloud coming into the marketplace one way or another.  

And I think that's an important trend we all should watch closely.

Comments

[Sudhir Brahma]

Hello Chuck,
I will appreciate if you share your views on how you expect demand for Storage appliances to change, especially with regard to the aspect of drive sizes getting larger. I believe most drive manufacturers have already announced >2TB drives at normal market rates.
To me, it looks like this aspect is already putting pressure to plateau out the demand for more "drive slots" in storage appliances.
That is, with the same number of drive bays one can get larger storage space, by just migrating to larger drives.
Now a humble 24 slot JBOD can boast of >48TB storage space.
That, together with de-dupe, storage virtualization and better compression technologies only make storage usage more efficient and add to the plateauing pressure.

Do you think there could be a time of equilibrium, wherein demands of additional storage could just be met by enhancements drive manufacturers make to drive capacities?

The cloud storage requirements you refer to, could cause a temporary spurt in demand for storage in the short run, as the service providers ready up their infrastructure. Once they are in business and ready, they could further put pressure on storage demands, due to the efficiencies they bring in.
May be the future of storage appliances is more inclined to having better management features and more importantly, the ability to operate across products seamlessly. The Large sized storage boxes may just not be in so much demand, since the drive manufacturers would address volume requirements directly.
I will appreciate if you can let us know your views.
Thank you
regards
sudhir.brahma@gmail.com