One of these topics that I'm getting frequently dragged into is storage management -- what will it look like going forward?
What makes this harder is that the decisions you make on how you will manage storage will be a direct reflection of how you decide to manage IT overall.
And it's not as simple a discussion as many would like, as we'll see in a moment ...
A Bit Of Context
In most decent-sized enterprises, storage management has usually been thought of as a stand-alone discipline: like network management or database management or related topics.
If you were doing things well, you had a stand-alone storage team who delivered storage services on behalf of other IT consumers: server teams, application teams and the like.
These storage teams typically have their own tools and processes for provisioning new requests, monitoring service levels delivered, reporting on consumption and troubleshooting problems when they arose.
This "storage as a separate discipline" model is under pressure from various quarters, and it's not entirely clear if there's going to be a "right" answer in the traditional sense.
What really differs in each of the four models is the preferred control point for the majority of storage activities -- each one has a different view around who's in charge.
Let's look at the four emerging models, and talk about their relative strengths and weaknesses.
The Internal Storage Service Provider
This is basically the classical stand-alone storage management function on steroids. Better tools, better processes and more flexible options.
The exposed storage service level catalog is robust, and can cover the majority of the organization's storage needs. Underlying technologies aren't visible to requesters, just the capabilities they deliver.
The tools enable IT organizational requests to be processed in hours, not days or weeks. Storage service provisioning, while entirely controlled by the storage group, is almost completely automated: not only primary capacity, but auto-tiering, protection, archiving and other policy attributes.
The supporting processes are robust enough to satisfy 90%+ of the organizational requirements without the need for people to get directly involved -- provisioning, service delivery, reporting on services delivered (capacity, performance, protection, archiving, compliance, etc.), as well as troubleshooting.
Flexibility options means a lot in this context: not only a wide range of service catalog options, but the ability to quickly change between options (whether manually or automatically) in a non-disruptive and responsive manner.
Going a bit further, the ability to decide which storage services should be provided internally (usually something like primary storage) and whichones should be provided externally under the control of IT (think compliant archiving, or geographic content repositories, or perhaps DR) -- well, that's an attractive scenario as well.
Sounds good, right? Well, there are pros and cons to every proposed model, and this one is no exception.First, there's the notion of implied scale here -- not only in capacity, but in investment in the critical skills, tools and processes to make this all work. No magic hunk of glittering technology is going to solve this one for you.
Speaking strictly from an EMC perspective, we have customers and partners who are in this league, but it wasn't an overnight phenomenon. Not everyone will have the supporting business rationale to invest in rising to this level of proficiency.
Second, regardless of how well it works, it's still a stand-alone function. That means that potential opportunities and efficiencies are inevitably lost by not converging and integrating storage management with other IT disciplines. On a more pragmatic note, the storage team will always need operational context to get their job done -- think agents, passive listeners, etc. scattered throughout the infrastructure -- all reporting back to the storage team.Storage Management Integrated With Operating System / Hypervisor
Even going back several years, there was always a strong case to be made to exposing storage management capabilities to the people running the server environment. Historically, this meant pretty good integration with Microsoft products, followed by a long list of UNIXes that never really got there, integration-wise.
The mainframe, of course, is a different story entirely-- most storage management disciplines are decently integrated with the zOS environment.
So you had this strange dichotomy where smaller shops (e.g. Microsoft-based) and very large shops (e.g. mainframe-based) could consider exposing more of the storage management tasks to the folks who were delivering compute services. It made a certain sense -- simply because storage management was being done in the context of a related activity.The proliferation of hypervisors -- most notable VMware -- has made this model extremely attractive once again. Indeed, there seems to be an arms race going on between two vendors now as to who can expose the most storage management capabilities to the virtualization administrator.
In smaller environments, this increases the probability that you won't need a dedicated storage person or team -- the storage devices are becoming pretty straightforward, and there's not usually much call to go beyond what's exposed to the person working with VMs. That's a clear win.
In larger environments, though, you'll undoubtedly need a dedicated storage skill set -- especially if you're pushing the boundaries of scale, performance, availability, recoverability, etc. The storage team will just have more time to focus on the important stuff vs. day-to-day requests.
But there are clear challenges with this model that have to be acknowledged. In smaller integrated environments, although you don't have to know a lot about how storage arrays work, you do have to understand the basic storage concepts involved, and how they impact performance, availability and resource usage. Someone still has to understand backup, for or perhaps archiving.
And in larger environments, it's very likely that the virtualized environment will be a subset of a much larger domain that includes non-virtualized users of storage. That raises the spectre of two (or more!) operational procedures -- one for the server admins who are virtualized, and a separate process for other areas of the storage landscape.
Storage Management Integrated With Converged Infrastructure
The introduction of Vblocks and EMC Ionix UIM (unified infrastructure manager) raises a third attractive option -- a single converged dashboard where most of the heavy lifting gets done for server, storage, network, hypervisor, etc. -- raising the abstraction to "end user service delivered" rather than "underlying resources provisioned".
As an example, the version of UIM coming out this summer places a significant amount of storage management responsibilities squarely with the person running that quintessential "single pane of glass".
The dazzling appeal of this sort of model is simple: converged resources are provisioned, monitored, reported on, changed, etc. the way that users see them -- as an end-to-end service, and not a collection of piece parts. In a nutshell, it's a cloud-like operational model.
The challenges are also clear: these sorts of converged infrastructure platforms and management models are relatively new. They represent a clear departure to more traditional workflows and segmentation that we're all so familiar with. And, as a result, they're going to be very hard to adopt in an incremental fashion -- in one sense, you're largely blowing up one way of doing things, and replacing it with another. There's just no good way to ease into it incrementally.
At scale, you'll still need dedicated storage expertise -- just like you would with other technology disciplines -- but the role is very different: delivering a pool of storage-related capabilities that are largely managed and controlled by someone else.
Storage Management Integrated With The Application User
The vast majority of storage is consumed in the context of an application -- be it an established production application, or perhaps application development, or even a business user who's doing something interesting. This gives rise to a fourth model where a significant portion of storage management responsibilities move from the infrastructure side to the application side.
Should a proficient Microsoft Exchange, Sharepoint or SQLserver be able to express their storage requirements, monitor their usage, and modify as requirements change? Not only capacity, but performance, availability, recoverability, archiving, compliance, et. al.? What about SAP administrators? Oracle DBAs? Power analytics users?
How about anyone who's got access to a self-service private cloud environment?
This model is perhaps the most disruptive of any presented here: it gives consumers of storage services largely unfettered access -- and associated responsibilities -- to meet their own requirements. But, at the same time, there's no arguing about its process efficiency. People needing the resource get what they need with a minimum of process and overhead.
Putting It All Together
These four models are not entirely incompatible, if you think about it. Rather, they reflect a maturation in the ability of an IT organization to deliver a service, and put it in the hands of the next person farther up the stack without causing undue chaos.
The storage team gets really good at delivering a storage service. They decide to expose a subset of these capabilities to the VMware team. That matures for a while, and both storage and compute services are exposed up to a converged infrastructure manager. That matures for a while, and those capabilities are then exposed to proficient consumers of IT resources: application owners, developers, and the like.
Control points progressively move from the storage team, to the server team, to the converged infrastructure team, and -- finally -- directly to the person or entity who is consuming the service.
Kind of like how Amazon's AWS works today :-)
That's one attractive sequential path -- but there are other adoption models that I can point to as well. Indeed, I can point to organizations that are doing two or more of these storage delivery models today.And, although this is a storage-specific discussion, it's actually one aspect of IT's overall transformation from the legacy model (owner of technology) to the emergent model (deliverer of IT services). Everyone is on this journey -- or should be.
Implications
So much time and effort in this industry is spent debating what's right, and what's wrong.
For storage vendors, the implications are pretty clear. First, we have to construct storage management stacks in a way that can support all four models -- at the same time, using the same set of shared resources. Second, we have to invest in four distinct classes of user interfaces: one set for dedicated storage managers, one set for VM administrators, another set for a converged infrastructure users, and -- finally -- a variety of integrations into popular application environments as well as self-service models.
Finally, we need to have the consulting capabilities to put the various options in front of our customers and partners, help them decide which one makes sense for them, and progressively enhance the skills, processes and tools that help people get where they're going.
From a vendor perspective, there's a lot that needs doing :-)
As I turn my focus to customers of storage technology, I'm struck by two fundamental challenges.First, there's no simple right answer, just a sequence of models that different organizations are embracing as they evolve. That can be hard for people to get their head wrapped around. They're looking for "the answer", and not a sequence of answers.
Second, it's getting increasingly hard to have an isolated "storage management" discussion in the abstract -- it's getting very intertwined with just about everything else that's going on in IT these days. More people have to come to the table. Bigger issues have to be discussed first before getting to storage management. That's another source of inherent complexity and friction.
For those of you who are chartered with creating a storage strategy for your organization, it's interesting times indeed :-)
Good, thoughtful post, Chuck. One of the most interesting aspects of enabling storage management functions in various roles (I think the stack concept is a bit forced here) is the long-term maintenance of the infrastructure. Managing deleted and obsolete resources will become increasingly important when the storage management roles are distributed.
Customers will want their "infrastructure commons" to be clean with a minimal amount of virtual storage junkyards within. Management overlap is fine as long as there is a defined hierarchy of control and the ability to clean up messes and resolve problems. It's not just a matter of developing high level views of everything, but also building technology into products that do the sweeping up for all the people that are too busy to think about the mess they are leaving behind. We shouldn't expect the application level people to care much about that.
As an industry, we have a very long way to go where sweeping up is concerned.
Posted by: marc farley | July 12, 2010 at 04:10 PM