The Impact Of VMware Storage Integration

Today is the day that VMware announces the availability of 4.1, and a key set of storage integration APIs (VAAI) that provide all sorts of cool speed-ups between server and storage.  EMC's announcement of day-one support is here.

Chad Sakac (our legendary Virtual Geek) does a great job of explaining what they are, how to use them, and what they can do for you.  If your passion is hands-on tech, please head on over there.

However, if you're more like me, and are always curious about what things might mean in a broader sense, this post might be more to your liking ...

Of Operating Systems And Stacks

One could argue that any operating system -- whether a traditional OS or a hypervisor -- has two primary roles.

The first primary role is to abstract and coordinate hardware resources (compute, memory, storage, network) and make them convenient to consume.  The second primary role is to provide convenient and useful services for use by applications.

If you're fan of dramatic oversimplification, you can reduce the statement to this: the goal is to make the plumbing more useful.

I think by now most people realize key aspects of VMware's game plan: over-deliver on the first role (abstraction and coordination of resources), without forcing a change of model regarding the second role (convenient and useful services for applications).

By encapsulating popular x86-based operating systems, VMware has now become the de-facto standard for server virtualization in most enterprises. 

Many believe that VMware is taking play after play from Microsoft's classic playbook.  Regardless of whether this is true or not, you most definitely can see echoes from previous Microsoft initiatives in VMware's current activities, and VAAI is no exception.

Where Does Storage Functionality Go?

Speaking as a storage vendor, there's always been a continuing debate around storage functionality -- where's the best place for it to go?

One might think that all storage vendors simply assume that the "best" place for storage functionality is always locked up in some storage device, since that ostensibly feeds our respective business models.  But that's not entirely the case ...

Snap and replica coordination, for example, requires some back-and-forth with either the operating system, the application, or both.  Attempting to do it all from the array just can't be as good.

Intelligent path management (e.g. PowerPath) has to be done from the server and nowhere else.  Encryption is another example where one could construct use cases to embed the function in either the application, the operating system, the HBA, the storage network or perhaps the storage device itself.

Historically, EMC has always seen the operating system as an attractive focal point for both storage functionality as well as storage functionality integration.  There's just so much you can do -- and do well -- if the operating system is a willing partner in delivering storage functionality.

Opening Up Is Hard To Do

But, as we look back, this has traditionally been somewhat of a vexing problem.  The classical  operating system vendors (mainframe and various UNIXes) weren't exactly motivated to invest in mechanisms to accomplish advanced storage functionality integration. 

First, they mostly didn't see storage functionality as a priority (the server mindset), and -- of course -- any potential investments made in this area tended to benefit competitive vendors (like EMC) who had rich portfolios of storage functionality.

For many years, we were at somewhat of a standoff -- the only way we could orchestrate storage functionality is to write our own code that we would introduce into the various operating system stacks.  EMC did a lot of this during the 1990s and 2000s -- APIs, CLIs, scripts, enablers, utilities, etc.

Customers liked this, and ended up using a lot of these features to integrate storage functionality with their broader environment.  But, let's face it, these capabilities weren't built it -- they were bolted on.

Microsoft Changed The Game

The picture substantially changed during the last decade: Microsoft started to makestrong investments about embedding and orchestrating storage functionality from Windows.  From strong iSCSI support to open coordination mechanisms for things like snaps and backup, to extensible management consoles -- we all saw a very compelling set of powerful and open storage capabilities to work with.

Microsoft really didn't have a tactical storage hardware business to protect like the other guys.  They believed -- and rightfully so -- that by opening up the storage stack in a way where others could play, they could create additional value and differentiation for their operating system platform.

Personally, I believe that this initiative contributed to Windows being considered for more serious workloads in the enterprise -- places where storage functionality creates more value.

And then -- at some point -- the initiative seemed to lose momentum, and we all went on to other things.  Note: interest has revived more recently with the recent spate of Powershell integrations and Hyper-V.

Enter VMware

One thing that VMware understands is the value of a strong ecosystem.  Whether it's server vendors, resellers, integrators, software vendors, or even lowly storage vendors -- they appear to fully appreciate the sum-of-parts strategy.

I think it is highly unlikely that VMware will ever anoint a "partner of choice" in any meaningful category.  Customers want strong choices -- the more the better.  And VMware is highly motivated to invest in creating as many strong choices as possible around their core value proposition.

Storage is no exception. 

At present, the two strongest VMware storage choices are EMC (#1 in Q1 external storage market share at 24.6% according to IDC) and NetApp (#3 in Q1 external storage market share at 11.1% according to IDC).  Both companies are independent storage vendors in that neither of us are in the server business, like HP and IBM.

Both companies are investing like crazy to integrate as much as possible with VMware's environment.  There's the usual back and forth around who's better, etc. -- but once you get past these two vendors, there's not too much to talk about in regards to storage integration in a VMware context.

Both vendors have rich sets of storage functionality.  Both vendors realize the value of integrating that storage functionality with operating systems and applications.  And both vendors have been itching for something like VAAI for a very long time.

What Does This Mean?

Storage functionality becomes more interesting as the environments get larger and the workloads get more challenging.  Clearly, this level of integration helps VMware better position itself to take on more of the big stuff. 

Simply put: you can't get this level of storage integration in any other operating system or hypervisor today, and it's unlikely you'll see something similar from anyone else for a very long time.  From my perspective, the only operating environment that even comes close to this level of storage functionality integration are zOS-based mainframes.

I think this move is also indicative of how VMware is thinking about its partner ecosystem.  Yes, go broad and have lots of players, but invest selectively in going very deep with a few proficient players.  VAAI is very deep integration indeed -- a big investment for them, and a very big investment for EMC, and presumably NetApp as well.

If anyone had any doubts about the independence and openness of VMware given EMC's ownership position, this announcement should go a long way to making a key point: VMware does what is in VMware's best interests -- because, ultimately, that's always in EMC's best interests as well.  EMC is a very passionate and very competitive storage company at its heart -- if we were going to somehow cross some sort of line, it'd be around this very topic.

Finally, I think this announcement is evidence in a subtle but somewhat important change in the discussion.

Virtualization with VMware is becoming less about doing what you did before a bit better.  It's now more about doing things you can't do any other way -- period. 

And, with VAAI and 4.1, storage functionality integration becomes yet another proof point.

Comments

[marc farley]

Chuck, I certainly agree with your statement "Simply put: you can't get this level of storage integration in any other operating system or hypervisor today, and it's unlikely you'll see something similar from anyone else for a very long time." Very true.

I also agree with your comment about how VMware works with its partner ecosystem - including 3PAR. Yes, we are smaller than EMC and Netapp but definitely in some very large key accounts for VMware where we work with them diligently to keep the customer satisfied. We had a very good day today with the announcement of vSphere 4.1.

[Vaughn Stewart]

Chuck,

Many times we are odds but this time, I believe you have nailed it. Well said sir.