This Changes Everything

For those of us in the IT business, we occasionally encounter a fundamental new enabling technology that forces us to reconsider some of our long-held notions around the way things work.

I'd put server and desktop virtualization into that category, as well as the ubiquitous web.  If you're a storage person, flash has that potential as well.  If backup is your thing, the combination of dedupe and low-cost disks has changed how you think about things.

In this post, I'd like to start to introduce a technology concept that -- yes -- has the potential to change a great deal of how we think about IT at scale. 

And, yes, this is going to be a long post ...

Context

Today, Pat Gelsinger did an important event with industry analysts.  You can see his materials and webcast here.  In addition, I wanted to offer up my views on this topic as well.  I'll be using his slide deck as a reference point.

If you're looking for a big product announcement, that's not here quite yet ...

But as a precursor, it's worthwhile to talk about the big ideas you'll undoubtedly see in real products before too long. 

And if you're a regular reader of this blog, you'll see some familiar themes and ideas brought forward with a bit more clarity and specificity.

At The Beginning

A great starting point for the discussion is the private cloud model we all have been discussing for the better part of a year.  For enterprise IT types, this shows all signs of being the dominant cloud model in the industry.

What makes a cloud a cloud is three things: technology (dynamic pools of virtual resources), operations (low-touch and zero-touch service delivery) and consumption (convenient consumption models, including pay as you go). 

In my book, anything that does IT-as-a-service along these lines qualifies as "cloud", irrespective of where it lives or who owns it.

What makes a cloud "private" is that IT exerts control over the resources and associated service delivery.  After all, enterprise IT is accountable for results, and to do this, they're going to need control.

In most cases, this takes the form of fully virtualized environments in the data center, with corresponding next-gen operational and consumption models.  Initial workloads are usually non-critical and pointed at unmet needs in the enterprise.

Over time, progressively more challenging workloads are brought over as confidence is gained -- without compromising the technology, operational or consumption model that has been established.

At the same time, these same enterprises are interested in working with compatible service providers -- compatible not only in terms of technology, but respecting the enterprise IT need to control, secure, manage, audit, etc.

The recent vBlock and the supporting VCE Coalition are useful instantiations of these concepts -- but obviously not the only way to get there.

And, yes, it's working.

Why Federation Matters

If you think about it, the whole idea is to create progressively larger and larger pools of resources.  The bigger the pool, the better optimization, and the easier it is to flex capacity, etc. 

"Federation" is nothing more than a handy term to describe this sort of orchestrated cooperative pooling.

A pooled cluster is better than standalone server and storage.  A data-center scale cluster farm is better than a handful of servers and storage.  Multiple data centers pooling resources is better than a single one. 

And being able to pool (and federate) across multiple data centers that can either be owned or rented  -- while retaining control -- is best of all.

The federation discussion usually breaks down into two categories: federation of resources, and federation of control.  Resources are the usual compute, memory, storage and network stuff.  Federation of control is resource allocation, service delivery, security, compliance, etc.

And, yes, we believe that storage federation is an extremely important topic going forward.

It's one thing to federate compute and applications.  It's another thing entirely to federate terabytes and petabytes of information behind them.  Organizations tend to have strong concerns about their information, and with justifiable reasons.

EMC Virtual Storage

Into this discussion, EMC would like to introduce the extended notion of "virtual storage".   Compare and contrast that with the more tangible "physical storage".

One definition of "virtual" is the complete abstraction of the logical entity from physical device or location.  Indeed, it's getting progressively more difficult to have any sort of "cloud" discussion without a "virtual" discussion, and vice versa.

Put differently, the ideal concept of "virtual storage" would be the complete separation of information from both physical media and location -- while under the control of IT.  If you think about it, that's a far cry from earlier attempts at more traditional "storage virtualization" that we're all familiar with -- "virtual storage", as discussed here, is a greatly extended concept.

If we go back to the private cloud model, any technology enabler that separated physical from logical can be combined with newer operational and consumption models to create entirely new classes of IT solutions that many will find attractive.

If you consider the top-level attributes that we need from any next-gen enterprise cloud-like model, we end up with many of the same ideas and concepts we've seen before -- automated, integrated, on-demand, efficient, always-on, secure, etc.

And, yes, fundamentally new storage technology can play a key role in making this both more attractive and more real.

Freeing Logical From Physical

Just imagine, for a moment, that we could transcend many of the physical limitations around storage and information today. 

What new sorts of things would be possible?

You'd be able to seriously consider moving thousands of VMs over thousands of miles.  Or you could easily play the market on energy costs by moving workloads to where they're most efficient to run.

It'd be much easier to aggregate larger data centers from smaller ones -- or put information closer to users, if that was needed.  Applications could move to alternate locations on demand and we'd eliminate major sources of downtime.

And we're not talking about just large-scale enterprises here -- smaller organizations would find this sort of IT infrastructure attractive as a service, to complement their traditional investments.

The private cloud model takes a big step forward.

Parallels With Server Virtualization

As we explain concepts, we've found it useful to show the parallel between the evolution of virtual servers -- and virtual storage.

The virtual server story is pretty familiar to most of us.  We came from a world of standalone servers that weren't fully utilized, and weren't pooled. 

The first wave of technology consolidated multiple servers onto more fully utilized ones.  Efficiencies were gained simply by having to use fewer servers to get the job done.

The second wave of technology created pools of server resources.  Not only were these more efficient than their predecessors, they created entirely new usage models -- virtual servers on demand, dynamic resizing and relocation of workloads, better approaches to load balancing and high availability. 

We now are entering a world where we think about compute resources very differently than we did just a few years ago -- thanks to VMware and others.

Storage is on a similar journey.

We started with stand-alone storage pointed at specific applications or workloads: use storage array X for workload Y.  We then learned how to effectively consolidate multiple storage workloads and use cases onto a single storage array more effectively.

This sort of "storage workload consolidation" is where technologies like FAST -- fully automated storage tiering -- play a key role -- the effective consolidation and optimization of multiple, changing workloads on a single array -- much like VMware did for servers.

And now we're ready to take the next step on the storage journey -- the creation of progressively larger (and more dispersed) storage pools that create entirely new usage models -- not only more efficient than their predecessors, but entirely new approaches to storage on demand, dynamic resizing and relocation of workloads, and better approaches to load balancing and high availability.

Putting The Pieces Together

As any IT professional knows, both servers and storage are needed to get work done. 

Combine and integrate server federation with storage federation, and we've got something that's worth taking note of -- compute and information working together to create substantially new approaches to IT architectures. 

Especially as we all pursue the goal of delivering IT-as-a-service. 

Once fully considered, it's a brave new world :-)

It's Much Harder Than It Looks

For one thing, to be truly useful, storage federation must overcome many of the fundamental challenges that result from distance. 

And this is hard for us -- both technologically as well as intellectually.

Why? 

We're all familiar with issues around latency -- the time it takes information to move from one place to another at the speed of light, and back again.  We know that bandwidth is expensive -- and it directly limits what might be achievable.

And, if we've got multiple writers who are concurrently updating information, consistency issues (having a single version of the truth) become progressively more problematic as we consider the more interesting use cases.

The other side is that we've just grown so accustomed to these customary barriers. 

We are all well aware of the issues around latency, bandwidth and consistency -- and we architect our solutions around these traditional obstacles.  Indeed, it affects so much of our IT thinking that overcoming traditional perceptions will be a serious obstacle to any new enabling technology in this category.

But without overcoming distance -- and I mean more than few dozen kilometers -- none of this is particularly interesting as a strategic enabling technology.  Up to now, most of our pooling and clustering thinking has been limited to "what can I do in a data center" or "what can I do with co-located data centers". 

We propose nothing less than breaking this barrier in a fundamental and meaningful way.

Exploiting Distance

In previous posts, I offered up some thoughts as to what happens when we change our perspective from "overcoming distance" to "exploiting distance".

We may look at protection and high availability in a fundamentally different way, for example.

Less attractive are traditional "source-target" DR approaches, which give way to a geographically distributed N+1 cluster where all resources are put to work, and failover is nothing more than a node failure.

We may look at resource and cost optimization in a fundamentally different way, as well.  If it becomes easy and transparent to move stuff around (e.g. reducing friction), more stuff can and will be moved.  And, instead of optimization windows that are measured in years (e.g. building data centers), we'll think about things in progressively shorter optimization windows: months, weeks, days, hours, etc.

And, finally, let's not forget that users want to be close to their data.  Long, slow and latency-rich networks seriously degrade the user experience, and can hamper productivity.

Introducing Distributed Cache Coherency

If you're a storage person, you know how important cache is.  For decades, the enterprise IT industry has built storage devices around a simple idea -- not all data is accessed uniformly. 

Find the data that matters, put it in cache, and you can dramatically increase application performance.

Indeed, the storage industry has proven -- for many decades -- that caching is an effective way of improving the performance of traditionally slower disk drives.   The algorithms -- and their effectiveness -- are generally well understood. 

For a slightly more detailed discussion, please see this post.

Intelligent storage caching can also work at a distance -- whether it's content delivery networks, or WAN acceleration for file systems.  But none of these approaches have dealt well with changing data, or very high I/O rates.

When considering caches, it's important to point out that memory is becoming dirt cheap -- whether we're talking DRAM or flash.  We now can seriously consider very large caches at economic prices in a way we couldn't consider a few years ago.  As a recent example, the latest Cisco UCS supports 384 GB in a single blade -- and that's just for starters.

The concept of cache coherency is well understood to anyone familiar with CPU and server architectures.  When you have multiple readers and writers, it's important that every entity has "one version of the truth" when working with cache or memory. 

Indeed, there is a large body of work around designing progressively more efficient cache coherence mechanisms in the server world.

Distributed cache coherence is another animal altogether. 

Instead of fat, redundant wires measured in meters, we're talking public networks potentially measured in thousands of kilometers.  A fundamental new body of computer science research is required to take multiple, distributed caches, and make them work together well at progressively meaningful distances.

These algorithms end up being the "secret sauce" in making all of this work as promised.

Towards Global Federation

But solving this fundamental technology challenge of distributed cache coherence can enable an entirely new model for storage: global federation.  The clear potential exists for a global dynamic pool of both storage resources as well as the applications that use them.

Create a consistent global view of storage and cache state, and all of the sudden we can consider multiple writers to the same information at serious performance levels.  Do this at a low enough level of abstraction (e.g. block devices), and it becomes generically applicable to just about any enterprise use case you'd care to consider.

Yes, it's rocket science.

Use Cases For Storage Federation

So, imagine we were able to build a practical storage federation capability on top of distributed cache coherence.  How would people use it?

One obvious use case is local federation -- creating these cooperating storage pools either within a data center, or across co-located data centers.  It's easy to imagine this being delivered as both an appliance that works with most any storage, or perhaps as a feature of the storage array itself.

This is the space that traditional storage virtualization (whether array, appliance, network or server-based) has typically played, although with some well-known limitations. 

So the potential is definitely there for a better mousetrap for this well-understood use case.

The more compelling use case is multiple nodes at serious distance -- call it global federation.  That's where the mind starts to reel a bit at the entirely new potentials.

And if we could do both with the exact same technology -- well, that would be useful as well, wouldn't it?

Consider The Alternatives

Imagine, just for a second, you were handed the project to create a globally dispersed cluster of hundreds (or thousands) of applications and their many terabytes of storage.  You'd be looking at an insane level of technological and operational complexity, not to mention associated costs.

You just wouldn't consider it.  Well, now maybe you can.

Perhaps you had to build a high-volume transactional application with global user population.  You'd be looking at sophisticated application development, as well as some pretty robust infrastructure and procedures. 

Well, now you've got another alternative to consider -- putting this capability in the infrastructure, and not in the application.

Or, let's say you wanted to do something more simple, like move a large set of applications and their data from here to there -- and wanted to do it non-disruptively.  Wouldn't it be great to move the apps without having to move all the data first?

And that's just the beginning.

Serious Implications For IT Architects

As a starting point for imagining what's becoming possible, consider anything you could do today on a modest-sized compute and storage cluster -- load balancing, failover, etc.

Now stretch the nodes with arbitrarily long wires, presumably without sacrificing performance and breaking the bandwidth budget.  Maybe some of the nodes are offered by service providers, rather than owned by IT.

What could you now consider?

First, you'd have an entirely new class of resource pool to consider.  You'd look very differently at how many data centers you need, where they're located, and -- most importantly -- the roles they play.

You'd think very differently about how to provide high availability.  Or resource optimization.  Or better user experiences for demanding applications.

New fundamental enabling technology creates new insights, which create entirely new approaches to traditional architectures.

Yes, it's that cool.

You've Seen This Before -- Sort Of

Actually, this is not the first time EMC has discussed storage federation.  You've seen it in the Atmos storage platform, which uses an object and metadata model to implement geographically aware policies.  But the use cases are somewhat constrained to data with low-to-moderate change rate -- not to mention requiring applications that can bridge to object models.

A more generic block-level abstraction that can handle high I/O rates and not limit the choice of apps -- well, that's another way of looking at what we're discussing here.

Back To Virtual Storage

Stepping back a bit, the whole discussion around global storage federation is a key part of separating logical from physical.

The real impact of FAST is to separate storage workload from physical storage media.  And the real impact of global storage federation is to separate storage workload from geographical location.

And, if we step way back and reconsider a key aspect of the private cloud model -- the progressive pooling of resources inside, across and outside of data centers, you can see why we think this is a fundamental enabling technology.

Actually, there's much more to the EMC virtual storage story that we're working on -- but I think that's plenty to consider for now.

Want To Know More?

I don't blame you. 

Many of us have been captivated by the new technology's potential to -- well -- change everything :-)  And seeing is believing ...

My best suggestion? 

Start making plans to attend EMC World this year ...

Comments

[Dan Megehee]

Does it REALLY change everything?

You point out that the three historic constraints of distribution across large geographic distances are still limited by the speed of light and cost. You also point out that caching has historically been used to effectively "speed" slow devices. And, you point that maintaining a consistent cache across global distances is far from trivial - without introduction of limiting mechanisms that in the past have caused them to be far less effective.

I think you understate the dependence on new algorithms we find ourselves. Traditional approaches all (that I'm aware of anyway) seem to assume that messages messages hit the cache in the order they were sent. With increased distances between nodes, this becomes less likely to be true. Your example of vast transaction based systems points to this. It's likely that sooner or later a transaction will hit the distributed system "out of order" without some sort of synchronization mechanism, and adding the sync using technology and approaches I'm aware of would tend to show a transactional system to much for most high transaction column situations.

Yes, it's not expensive to create large high speed caches. But, we're still limited by the speed of light in syncing access to the various cache components. I'd like to see some discussion on approaches to mitigate this issue.

[Chuck Hollis]

Hi Dan

Good thoughts.

Guaranteeing order-of-delivery is a well understood issue for enterprise storage types -- it arises from DR and consistency discussions -- including complex and interrelated applications that must be recoverable at a distance, and at multiple sites (think SRDF).

And, to hear more, we'll actually have to show you the product :-)

-- Chuck

[Steve]

Chuck, great piece. You know you could use Camtasia, record this with the slides. Benefits:
1) Less typing
2) More interesting multi-media for your audience
3) Be more 3parfarley like :)

Again, thoughtful piece.

[Chuck Hollis]

Thanks for the suggestion, but -- no.

My audience seems to prefer plain, old-fashioned and boring text. My blog posts are written in Notepad. Works for me!

-- Chuck

[LAUS_eng]

Chuck-

Perhaps you should start a weekly podcast series and I can upload to my iPod... I can listen to your material on my way to work...
Thoughts?

-Dave

[Chuck Hollis]

Hi LAUS_eng

Several people tell me that they use the Odiogo functionality on this site to download MP3s -- would that do?

-- Chuck

[marc farley]

Can't wait to hear the second hand reports from EMC World and the months of clarifications to follow. Its a very ambitious goal, but EMC is the one company that appears ready to become the modern version of 1960 IBM - despite what Larry Ellison thinks.

[Martin G]

Chuck, check out Sublime Text if you are still using Notepad...It's a great text editor...

[Chuck Hollis]

Thanks, Martin, I've downloaded it and am using it now. I like it already ...

-- Chuck