Mix two powerful themes in IT today -- "cloud" and "big data" -- and all sorts of interesting new conversations pop up.
For example, any meaningful cloud discussion will eventually hit on the difficult tradeoffs involved in moving information to the right place at the right time.
A while back, I wrote a well-received post on the subject ("That's Information Logistics" ) which sparked some interesting discussion.
Certainly, the provisional topic of "information logistics" can be interesting enough when we're talking modest amounts of relatively important information.
But change the view point to enormous amounts of very important information, and it's a very compelling discussion indeed -- one that will likely require new forms of enabling technology.
To Begin With
In a nutshell, clouds are IT environments that are built differently (pools of resources vs. static allocations), operated differently (delivered as an end-to-end service vs. individual IT disciplines) and consumed differently (convenient to consume for both the user and the IT organization).
If enterprise IT controls it, we call it a private cloud. If your service provider controls it, we tend to call it a public cloud. And if it's a mix of both, we often use the term hybrid cloud.
But very rarely is it presumed that all IT function is delivered out of a single location.
Even in today's traditional IT world, distance is involved. Most larger IT organizations have multiple data centers; and, if you do business globally, you have a global data center presence to match. Add in a growing number of external service providers to help shoulder the IT load, and you can see why conquering distance becomes progressively more and more important.
And that's even before you start realizing that the amount of relevant data to be moved is growing exponentially.
The speed of light is a constant, that's not going to change anytime soon, unless someone can figure out how to commercialize quantum entaglement. One can hope, but I'm not optimistic.
Most networks have defined latency and bandwidth characteristics. Indeed, a useful view is to consider "network" as your scarce resource, and then think in terms of how compute, memory, storage and algorithms can make you need less of it, or -- conversely -- how to do more with what's available.
Indeed, from content distribution networks to WAN acceleration to custom-engineered applications -- there's a lot of "overcoming distance with other stuff" thinking in the IT world today.
But -- from an architectural perspective -- there's a very strong incentive to put the information logistics as close to the storage as possible; "bake it in" at a fundamental level if you prefer. The lower-level the implementation, the more useful it can become -- more broadly applicable, easier to manage, more cost-effective, etc.
Indeed, if you're familiar with EMC's Atmos platform, you'll quickly see that its policy-driven "information logistics" capabilities are an integral part of the product -- not just some bolt-on or add-on capability. But that's just for objects, or -- in some cases -- files that can be managed using an object-based storage paradigm.
If your use case fits into that paradigm, it's not hard to make a compelling case for the technology. But what about something *really* generic, like block storage?
And that's where VPLEX comes in ...
VPLEX: Not Your Father's Storage Virtualization
EMC's VPLEX platform was announced last year at EMC World amidst the traditional fanfare, and then we sort of went quiet. The reason was simple: we wanted to work with customers around some of the newer and more intriguing use cases vs. simply fighting it out for the traditional storage virtualization market.
Now, a year later, that approach appears to be paying its expected dividends.
Not only do we have many customers who appreciate VPLEX as a better traditional storage virtualization layer, but an interesting breed who have built some truly fascinating use cases, essentially doing things that really haven't been done before, or – at least – not done well.
And that's cool.
In a nutshell, a VPLEX fabric uses advanced cache synchronization techniques to create a "stretched LUN" at moderate distances today, and longer distances in the near future. And that seemingly small implementation detail is opening a new world of possibilities.
Envisioning The "Stretched LUN"
Since there are so many storage-related technologies that move information around, it always takes a bit of effort to help people understand how VPLEX is fundamentally different than ordinary replication, or WAN caching, etc.
A good starting point is this uber-simple picture.
You've got multiple servers (and applications) essentially talking to the same storage pool. They could be sharing a file system, or a database, or something else.
It's presumed that they have the required logic to avoid corrupting each other's data (e.g. file locking, database locking, or something similar).
IT architects use this sort of approach for one of three reasons.
(1) They're looking to get better availability. One of the server nodes stands ready to step in if one of the primary ones fail for some reason. You'll find this sort of failover clustering in all sorts of flavors in the IT industry.
(2) They're looking to get better efficiency. If the workloads can be dynamically balanced (VMware's vMotion comes to mind), you'll end up with better use of existing resources, or a better ability to respond to demand spikes, or some combination of the two.
(3) They're looking to get better performance. Scale-out NAS is the classic example, but more common are producer/consumer relationships, e.g. one task is capturing information, another one is using it for a related purpose. One classic example of the latter is SAP -- transactions in the front, reports out the back.
Dramatically oversimplified, I know. I'm just trying to get to the essence here -- we'll save the gory details, exceptions and caveats for later.
Now, let's take that imaginary "shared LUN" and stretch it over progressively larger distances.
VPLEX does this with cache and algorithms. Broadly speaking, anything you can do with short cables you can do with progressively longer cables. Having the right data in a large local cache has proven to more than make up for the occasional latency hit when the needed data is at the wrong end of the wire.
Interestingly enough, the same three use cases are still valid; only, this time, they involve resources and participants that aren't in the same place at the same time.
And that is turning out to be *very* cool indeed, based on what our customers have told us.
Re-Envisioning High Availability
Normal best practices for remote data centers is very familiar: replication technology ships changed data out to remote storage using a variety of mechanisms, e.g. EMC's SRDF or RecoverPoint. In the event of a primary data center outage, all the needed information is there, all you need to do is fire up your servers, check your database, invoke your applications, etc.
Well, all of that "other stuff" can take considerable time, especially in larger or more complex environments.
With VPLEX, though, it's a different model entirely. Servers are already booted, applications are already running (perhaps in a read-only state), and everything is essentially ready-to-go at a moment’s notice.
This enables a whole new class of near-zero RPO and RTO that was very difficult to achieve before. (Acronym alert: RPO = recovery point objective, or how much data you're willing to lose, RTO = recovery time objective, or how long you're willing to wait before the lights come back on)
Not everyone needs or wants near-zero RPO and/or RTO, but some people do. As in "it's a really big deal for their businesses". VPLEX is finding a good home in those situations. And this should only increase as longer distances become supported.
Re-Envisioning Workload Balancing
Any time you have resources in more than one place, there's an implied optimization problem to be addressed. Using traditional approaches, moving a large and hairy workload from *here* to *there* (regardless of whether it's a physical or virtual workload) involves a significant amount of effort and planning.
Anything that's hard to do doesn't get done very often, and workload balancing is no exception. Our customers are now telling us that -- with VPLEX -- the amount of time, effort and associated risk associated with a workload rebalance has now been dramatically reduced.
All the servers and applications are already attached to the stretched VPLEX volumes. It's a far more tractable task of simply re-assigning roles and associated network addresses to do an "application move". Since it's easier, it gets done more often.
And, once again, this should only become more compelling as supported distances increase.
The real surprise to all of us was the vast number of producer/consumer application relationships that are amenable to this enabling technology: enormous amounts of information being captured in one place, and wanting to be used in one or more other places.
The classic example (though not generally applicable) was movie production -- or any large-scale digital capture, for that matter. Terabytes (or petabytes) are captured in one part of the world, and need to be used in another part of the world. And quickly, please.
The old-school approach is familiar: wait until a big pile of data is captured, then package it and send it to where it needs to go using a discrete transfer process like FTP or storage replication, or -- more frequently -- shipping a bunch of high-density media.
But valuable time is lost between information capture and information use, and -- in many models -- time is money.
The ability to "pipeline" vast information flows (writers at one end of the pipe, readers at the other end) is turning out to be much more applicable than we first thought. And using a foundational block-oriented caching mechanism creates a wide range of net-new solutions that couldn't be approached using other existing technologies.
Where are we finding interesting use cases?
Anything that involves digital capture at scale, for example. Traditional use cases like single-instance databases that needs to support multiple, diverse geographies. Data warehousing and business analytics where the people massaging the data don't live in the same time zone as the data warehouse. And much more, I'm sure -- it's early days.
The before-and-after compares (usually measured in time-to-value) are compelling, to say the least. And these people have already scoured the market looking for alternatives. I'm proud to say we've been able to bring them something new.
The Cloud Angle?
Yup, there's one there, and it's not hard to get to.
If you believe like we do that -- over time -- most organizations will use a combination of internal and external IT services to get the job done, it's not hard to see the need to have large amounts of "hot" data be essentially two places at the same time.
Larger service providers will also have strong economic incentive to use their multi-data-center footprints to improve availability, increase efficiency and slash application performance time along the lines outlined above.
Indeed, as we enter the era of big data, usable cloud architectures will undoubtedly need the ability to transparently share large data sets between multiple data centers separated by significant distances.
Sure, there are smart people who will argue that this needed functionality can be done at the application level, or -- perhaps -- the network level. But it's hard to argue against the attractiveness of baking in this sort of functionality as a lower-level architectural primitive -- right next to the storage itself.
A Potential Opportunity For IT Differentiation?
When we get into these serious discussions with our customers and partners, there's often a gleam in their eyes. Maybe it's not as fully baked as some might like, but they see the potential to do something very meaningful and very differentiated with IT. They want to get going.
And it's not just the technologists -- it's the business folks as well.
I describe it as making a shift from overcoming distance -- to exploiting distance.
And that's cool, if you think about it.