Better Tools ... Or Better Models?

I read with empathy recent posts on Wikibon regarding the general perception that storage management tools could be a whole lot better.

One excellent post spoke of managing storage performance.  Another speaks to large warehousing workloads meandering through the fabric, and the havoc that caused.  Good reading.

It'd be easy enough to say, "yes, darn it, we need better tools!".  And plenty of storage admins would agree with me wholeheartedly.

My argument, however, is that tools in isolation can only get you so far.  At some point, the model needs to change.  And that's a more difficult proposition.

Managing Storage In Isolation Is Becoming Unproductive

In my simple way of thinking, managing IT infrastructure is less about managing the thing itself, and more about managing the outcome.

And to effectively manage the outcome, you need context. You can't solve performance problems without context.  You can't solve availability problems without context.  And you can't solve capacity problems without context.

But where should that context be established?

Round One -- Grow The Storage Management Model To Establish Context

This was the thinking behind EMC's ControlCenter during the last decade.  Put agents on everything in your environment.  Use those agents to help create context that answers the key questions for storage administrators.

Add that agent-heavy mission to the legacy mission of "granular element management" for multiple flavors of storage array, access models, storage fabrics, etc. -- and you can see the potential for ugly mission creep.

And, in the middle of the last decade, that's exactly what happened.  The agents got heavier and heavier, and there were more of them, resulting in a strong preference for minimizing or eliminating storage-specific agents in host and application environments.

As storage devices got more complex and capable (and relevant standards such as SMI-S fell farther and farther behind), the mission of being a granular storage element manager became more difficult as well.

That being said, EMC's ControlCenter is still perhaps the most widely used storage management framework in the industry today.  Despite its historical challenges, it still does the job for thousands of enterprises around the globe.  It is far from perfect, though.

Going forward, the goal is to make the ControlCenter architecture less dependent on agents, far more pluggable, work closely with underlying storage element managers, and receiving useful context from other enterprise management environments, such as EMC's Ionix.

We believe that many customers will still need a powerful storage-centric view of the world (especially in larger enterprises), but we need to start thinking of it more as a layer, and less as a specific product.

Round Two -- Focus On The Virtual Machine To Establish Context

The widespread popularity of VMware has created a new focal point for establishing context with regards to storage management: the virtual machine itself.  Witness the headlong rush by EMC and other vendors to create all manner of powerful and elegant plug-ins for vCenter.

And, to a certain extent, this can be a better -- albeit not perfect -- model.

In this model, everything centers around the virtual machine and what it needs to get the job done. Storage performance, availability, capacity, etc. -- all done from a VM-centric point of view.

In these models, the storage admin is more of a generalist -- and a specialist.  The storage admin is responsible for providing aggregate capacity, performance and availability to the VMware farm, but usually doesn't get bogged down in the details.

Unless there's a problem, and then the storage admin is suddenly a specialist :-)

Make no mistake, this VM-centric view will be very popular in small to mid-sized environments.  But that still leaves us with larger environments, and -- not to mention -- non VMware environments.

Round Three -- Focus On The Application Service To Establish Context

In enterprise IT, it's all about service delivery.  Infrastructure exists to deliver services to users and applications, period.  And, if you're looking for storage context: performance, availability, capacity -- that's the most useful (and most difficult) model to establish.

Start with the delivered service.  Map back to supporting application, middleware and database components. 

Correlate the supporting infrastructure, virtualized or otherwise, including storage.  Now, go ahead and ask your storage-specific questions -- and you'll get useful answers that are hard to match other ways.

This line of thinking is what lies behind EMC's Ionix suite: service delivery management through correlated application and infrastructure discovery.

But the balkanized state of affairs in most enterprise IT shops makes achieving this more of a political and organizational challenge, rather than a technological one.

Unless you're a service provider, that is :-)

Storage Models Can Help As Well

Up to know, I've been sharing how the management model itself can potentially help to alleviate some of the storage management challenges.

But the storage model itself is showing strong promise in being able to minimize this challenge.

At a granular technology level, there is compelling evidence that object-based storage models (think Centera and Atmos in the EMC portfolio) are as about as close to "zero touch" in terms of management as is humanly possible.  Of course, not all forms of information can be conveniently stored as objects today, but it's hard to ignore the evidence here.

For more traditional forms of storage, one powerful model that I've been exploring here (and that EMC is working on) is "virtual storage" -- a complete separation of logical from physical, in much the same way that virtual servers seperate logical from physical.

Much in the way that a modern vSphere cluster can pool resources and automatically adjust to changing needs -- even over distances -- virtual storage should do the same.  Here's a pile of resource.  Here are the outcomes I want.  Go do -- and do so with a unusual degree of efficiency, automation and resiliancy.

But we believe that there's more that can be done in terms of redefining the boundaries between storage and the people who use it.

Recently, I sketched out another direction EMC was working towards -- the "information utilty" which takes the idea even further by (a) transparently incorporating storage-related services that many treat as a standalone function today -- backup, replication, archiving, compliance and other forms of information governance -- as well as (b) putting more responsibility for managing storage into the hands of end users.

The first concept isn't that controversial.  The second one will be undoubtedly controversial, much in the same way that self-service computing is controversial for many enterprise IT shops today.

Putting It All Together

Coming full circle, there's no disagreement about the growing challenges associated with managing ever-increasing amounts of storage -- and the information it contains.

While it's easy to say that tools need to be better (and they do), maybe we need to consider the management model as well.

And, trust me, the people at EMC spend a *lot* of time thinking about this topic.

Comments

[David Vellante]

You've been busy today Chuck... :-) Definitely a better model is in order I agree 100%. We were talking about that after the Wikibon call. Who wants to manage a zillion point tools...or even two really good ones?

Give the world Ionix integration w/vm that tells me what resources the virtual environment is consuming, how much, where and how is that trending. And provide an infrastructure manager that is unified and can be delivered as a service and you will make people very happy - I agree. And I also agree that is a better model. And I also believe this is where you are headed...great vision and probably lots of activity around this; today.

Having said that when a SAN manager has been struggling with storage growth of 15X over three years and he finds a practical solution to the problem-- I'm all ears...I'm sure you are too.

I also believe he'd think what you're doing is great and he'd say "give me a call when you're there."

[Stuart Savill]

Hi Chuck,

Great post around management - and i think it is an area that requires some real focus over the next few years.

The real interest to myself here is the huge paradigm shift that is occurring as a result of technologies such as: de-dupe, compression, thin snap, fat-to-thin, thin provision tools that are emerging (the list continues...) ways of reporting need to change drastically.

The storage admin's role has now got hugely more complex as he/she now has to (as an example) understand how well a given dataset will de-duplicate (or compress or thin provision etc...) we no longer think just about how much physical space is left, and our $/TB cost (physical tin) - we need tools that can report on this effectively..

Gone are the days where we can plan purely on physical TB's per application and from a capex perspective we need to stop thinking about $/TB physical and shift to a model where we can monitor and charge on $/TB of data stored.

I would offer that any tool set is far from able to offer this level of insight - and whilst the technology is there to do some of this "good stuff" that ability to be able to report to our senior management around how great a job it is doing - well, just does not exist...

Finally - these tools (and their ability to understand how data has been treated in terms of thin prov / de-dupe etc) is essential to the storage vendor to be able to prove to their customer they they really are getting the value add (and give the real motivation to keep their custom).

so back to your title - i still believe that you need the better tools before you can model appropriately (or integrate the functionality).

If you are interested - wrote a blog entry on this subject that can be found at: http://www.stuiesav.com/2010/01/measuring-cost-of-storage.html

Thanks for reading,

@stuiesav

[Martin G]

Firstly, we need to move from Storage Administration to Storage Management; the act of allocating storage etc has actually got easier. This is no longer a black art. Most of the administration tools are there or there about.

The SRM tools are basically still no better than spreadsheets in many cases and whilst for years, spreadsheets could do the job, they no longer can (well certainly without building incredibly complex ones). New features such as DeDupe, Thin Provisioning bring another dimension to Storage Management. Physical storage is meaning less and less when it comes to measuring how data is stored. So whilst for years we have tolerated poor SRM tools because most of our time was taken up keeping the storage estate running; we can no longer tolerate the fairly poor toolsets which are currently considered to be state of the art.

And without these tools, it is impossible to show the value that these new features are bringing to us. And without these tools, many storage managers are left in fear of the new features because they believe that overcommitment of storage, be it by thin provisioning or dedupe leaves them extremely exposed and they don't know what their ratios are etc.

Who'd be a Storage Manager?

[Chuck Hollis]

Hi Martin

On your first point on moving from storage administration to storage management, I'd agree. Provisioning is getting easy. It's the rest of the equation that's hard.

On your second point, I'd also agree. Underlying storage capabilities (insert long list here) have outstripped the management tools, which were none to robust to begin with. So we've made a bad problem worse, or at least temporarily.

And your final point is most telling: it's all about delivering outcomes. And, if you're a storage manager (not administrator), you need the toolsets that help you deliver the outcomes you need to the people who depend on you.

That being said, there's plenty of fuzz around what might be the ideal separation of roles and responsibilities going forward. How much of "storage management" needs to be taken on by the virtual machine administrator? The service delivery administrator?

Thanks as always ...

-- Chuck