As a part of this series, we’ve already taken a look at the rationale and motivations behind software-defined storage, offered up a working definition, and taken a tour of the key elements found in our software-defined storage model.
If you’d like to catch up, I’ve created a page for your convenience.
With these fundamentals in hand, it’s time to turn our focus towards a key question: what are the important changes when considering SDS?
I’m not talking about technological differences; I’m talking about real things that people will see and experience.
And, if you think you know storage, it’s going to be a very different world that will shortly emerge.
I did my best to summarize the key points of our previous discussion below. Warning: this is a highly compressed and overly-dense summary:
Information is becoming strategic, and data volumes continue to explode with no end in sight. While storage media costs inevitably drop, that’s only a small portion of the overall cost/value equation.
Enterprise IT architects are thinking — how can we do better?
The path is software-defined: using automation to replace labor-intensive tasks. Software is eating the world, and transforming every discipline in the data center, including storage. You might say that software is dissolving storage: making it more liquid and malleable.
The single defining attribute of software-defined storage is composability: the ability to dynamically compose an application-centric set of storage services, uniformly and without regard to the underlying hardware. But how is this done?
In our SDS model, it all starts with applications, their containers — and requirements.
Requirements are used to create policies, which are bound to application containers as they are pushed down onto the infrastructure, resulting in specific services being requested.
The control plane mediates between applications, their policies — and the capabilities of the supporting infrastructure. In this model, the hypervisor is in an advantageous position to clearly see both sides of the equation: application requirements, and arbitrating its access to services and resources.
While it might be convenient to think of the control plane in terms of a single pane of glass, the reality is that storage (and storage services) manifest in a variety of roles across the enterprise. The control plane must provide for each role-specific view, with storage shown in context of other relevant information.
The control plane also provisions (and monitors) data services: things like data protection, performance management, encryption, de-duplication, or perhaps a specific personality like HDFS. Ideally, these data services are themselves composable, align precisely to application boundaries, and are largely independent of the underlying hardware being used to persist data.
Data services, in turn, rests upon a data plane which actually stores and persists data. Data planes can be any combination of traditional storage arrays, external cloud providers, or a software layer running on familiar industry-standard servers. Data planes expose their capabilities (capacity, protection, performance, etc.) which are then consumed by applications mediated by the control plane interpreting policies.
Got all that? Good …
So, given all of this, what visibly changes when we move to a software-defined storage model?
Storage Provisioning Is Now More Dynamic — And More Granular
In today’s server world, provisioning is extremely dynamic and granular. Thanks to server virtualization, new requests can be satisfied quickly and very precisely. Exact number of vCPUs, memory, etc. — just what is needed — all in seconds and minutes.
Seductive stuff, and no one's going back to the old way of doing things :)
By comparison, most of today’s storage environments seem that they're from a distant era ...
Depending on circumstances, responding to the request might take between hours or weeks — or maybe longer. Compromises are inevitably made. If space is available on a particular array, but it’s over spec’d, well — that is what will be used.
There’s another source of compromise: some shops have switched over to using pre-provisioned storage pools (gold, silver, bronze, etc.) in order to speed fulfillment time. In these scenarios, you use what’s been pre-defined -- unless there’s a really good reason not to.
And since storage allocations can be difficult to change once made (and coming up short will result in complaints), the natural human tendency is to overprovision capacity, performance and protection.
Here, a new storage provisioning request is matched against the potential capabilities of existing storage services: capacity, performance and protection is requested.
If an exact match can be dynamically constructed from the existing resources (data plane plus data services), we’re good. If there isn’t a good match, other options are presented.
One important difference here that’s hard for many people to grasp: the physical storage isn’t actually configured until the request comes in.
Let’s use VSAN as an example? Virtual SAN starts by seeing a bunch of unallocated disks and flash devices. When a request comes in for “my policy says that I need two copies for protection, use more cache for performance, and please stripe behind it” — well, that’s when the hardware is actually allocated and configured, and not before.
That’s a significant difference as compared to what’s usually done today.
If it helps, imagine you're ordering a bunch of stuff from Amazon.com. As your itemized request flows through their automated warehouse, each item you've ordered is precisely packed in a shipping container -- exactly sized for what you've ordered.
The result? Storage provisioning that approaches today’s virtual server provisioning model in terms of responsiveness and granularity.
Service Delivery Is Also More Dynamic And Granular
Provisioning speaks to Day 1 requirements; ongoing adjustments in service delivery speak to days 2, 3, 4 and beyond. Things change. What was needed today will inevitably change tomorrow, and today’s storage models don’t react well to change.
Thanks to server virtualization, our compute environments are dynamic and responsive — add more, allocate less, add protection, move workloads to get a better balance, and so on -- and do so non-disruptively.
Today, when you land a given application on a familiar storage array, you’ve largely constrained the range of services you can easily deliver to it.
Yes, it’s possible to adjust certain aspects up/down, e.g. move it to a faster pool, invoke additional data protection, etc., but it’s usually a lot of work — and it’s usually neither integrated nor policy-driven.
For starters, it’s usually the case that multiple applications share a single storage pool. Making an adjustment to just a subset of the applications within that bucket isn’t feasible, so that leaves you with having to move a workload and its data.
Moving an application container’s data to a new location involves its own challenges -- Where do I have room? What can be moved? Not to mention the actual moving of application data …
Adjust a policy, and the underlying service delivery dynamically adjusts: data plane and/or data services.
If your new request can’t be accommodated for some reason (e.g. lack of resources, not authorized, etc.), you get a clear indication of the reasons why. If movement is involved, it's done transparently.
The mechanics of how service delivery is actually accomplished has been abstracted away from the request itself.
Storage Becomes Just Part Of What People Do
I have met some remarkably insular storage teams in my travels. Want something new, or something changed? Make a request, we’ll get back to you. Want specific information around a specific application, or class of usage? Make a request, we’ll get back to you.
The reality is quite different — storage touches many people, both inside and outside of IT. If I deliver a service, own an application, analyze cost information, etc. — storage is part of my picture.
The implication of the SDS control plane is that it becomes relatively easy to expose relevant storage information (or potentially controls) in the context required for most any specific role. Better yet — there is no notion of a hard mapping of roles and responsibilities, which is important as organizations evolve and learn new ways to work together.
Yes, you will likely still need a core team that understands storage-specific architecture and operations. But no longer will they be a choke point for everyone else …
Much More Flexibility
You can think of a typical storage array as a convenient package for the elements we’ve already described here: a data plane, a set of data services, and a control plane — all neatly packaged into a single enclosure.
Don’t like any specific part of the package? Your options have historically been somewhat limited.
With our SDS model, we learn to think differently.
The core elements — data plane, data services, control plane — are separable if desired. This creates an order-of-magnitude more flexibility than we’ve seen in the past.
Yes, you could move to another array — but the effort required is considerable.
In the SDS model, all components are separable: for example, the data plane could potentially be swapped out without disturbing either data services or control plane.
While I’m sure we’ll never get to 100% mix-and-match — this architectural layering makes introducing new choices more easily considered.
Unit Costs Go Down — A Lot
I’m encouraged by more IT organizations taking a harder look at their fully-loaded costs to deliver a unit of a storage service as consumed — all-in: software, hardware, environmentals, operations, consumption efficiency, etc.
Just like a manufacturer or retailer has to fully understand all their unit costs — direct and indirect — enterprise IT organization are no different.
My argument for cost reduction with SDS is not heavily dependent on the raw ingredients (hardware and software) being less expensive — although that may be the case.
Instead, I would argue that “conversion efficiency” has the potential to vastly improve. There’s less effort, and less waste.
One example: composed services are now more precise, and aligned on application boundaries. An application container’s storage services are also much better aligned in the time domain: exactly what’s needed, when it’s needed.
Another example: the “pool” of storage resources is now vastly expanded, and can be used more generically. A given data plane (e.g. storage array) can be pressed into service more widely, either because it has the required native capabilities, or those capabilities can be enhanced through layered data services (e.g. caching, replication, etc.).
This pooling extends even further when considering software-based data planes and data services. The same pool of compute, memory and network that’s used to deliver compute services is now also used to run storage software.
A third example: when comparing frequent workflows and the labor involved (e.g. new service request, problem resolution, etc.), you’ll notice that there are a lot fewer steps and a lot fewer names involved.
Value Delivered Goes Up — A Lot
The core SDS notion of dynamically composed storage services creates additional IT value when fully considered.
Second, there’s the ease of adjustment. If I’m a consumer of storage services, knowing that I can quickly adjust my requirements with little-or-no organizational impact is worth something to me. I’m less likely to over-request, for example.
Third, there’s speed. I get what I want quickly, and accurately vs. waiting and waiting for something that may not end up being right. That’s worth something as well.
In today’s storage world, most of the focus is on costs. In the SDS world, we can look forward to value creation as well.
Monitoring And Compliance Becomes More Natural
I’ve joined enterprise IT problem resolution meetings when it feels like blind men in a room with an elephant. Where’s the problem? What was promised? Do we know the cause? What are our potential options? Etc. etc. And storage is usually an integral part of that discussion, as that’s where the information lives :)
While introducing SDDC concepts (and SDS specifically) won’t eliminate problems, it certainly makes the resolution framework more approachable. Every application has a set of requirements, specified in the policy.
Question #1 then becomes “are we delivering on the policy’s requirements?”. Easy enough to verify. If not, then the question becomes “why not?”. Since every storage service is composed, it then becomes a process of recursively examining the supporting components.
If instead the service is being delivered as requested per policy — and there’s still the perception of a problem — should the policy be changed?
In effect, the application container’s policy becomes a precise contract for services delivered, which makes things much more cut-and-dried. More clarity — and more accountability — than what is often the norm today.
Responsibilities Shift, With More Empowerment
Years ago, I would call a travel agent anytime I had business travel. These days, I use a travel portal. As a result, I’m now more knowledgeable — and accountable — regarding my travel decisions. And if things suck, it was my choice that led to it.
While there are still gatekeepers (and experts!), the expectation is that more people will need to be more knowledgeable — and accountable — for their storage decisions.
An application developer, for example, will have to have some idea of what their application requires for capacity, performance, protection, etc. Better knowledge = better decisions.
While any shift in roles and responsibilities will be controversial — and perhaps unpopular — the secular trend is hard to argue with.
The Software-Defined Data Center Becomes More Achievable
Attempting to do so without directly evolving storage to a SDS model means that — in many ways — the goal of SDDC itself becomes compromised.
If the storage layer can’t dynamically interpret and respond to the same policies that are being used to drive compute, networking, security, etc. — that’s going to be a problem.
Next up: obstacles and challenges in getting to SDS.
Like this post? Why not subscribe via email?