Reflections on VMware's Cloud Infrastructure Announcement

The day before yesterday, VMware had their most-significant-ever product announcement.

The amount of technical goodness -- and what it means for the industry -- takes more than a little while to wrap your head around.

A bit of that awesomeness was overshadowed by reactions to VMware's new pricing scheme.  I watched the vocal back-and-forth intently as it unfolded through Monday and into Tuesday.  Wisely, I kept a safe distance from the fray.

But I did want to weigh in at some point.  I think some people are missing a few key perspectives, and I'd like to take a shot at broadening the conversation.  

Disclaimer: This is just plain old me talking.  VMware, as an independent company, runs their own affairs.  I am speaking strictly as an industry observer here, and not in any way communicating the official position of EMC, VMware, or anyone else for that matter :)

The VMware Technical Awesomeness

I think one of the best writeups I've seen came from our own Chad Sakac (aka The Virtual Geek).  There's at least *five* useful and detailed posts here, all worth savoring in detail.  I can't share it, but the internal briefing that Chad sent around was equally awesome.

In a nutshell, here's what we all can look forward to:

• far larger virtual machines (memory, CPU, disk space, I/O) -- enough to take on the scariest workloads
• far more robust virtual machines -- improved HA and DR capabilities -- again, enough to take on the scariest workloads
• a tightly integrated DLP (data loss prevention) capability that might take a while for some to fully appreciate
• truly unique and compelling storage integration extensions -- VAAI and VASA -- that you won't find anywhere else
• better and more integrated management tools in a few key areas
• and an interesting VSA (virtual storage appliance) that will undoubtedly find a home in smaller settings

Big news, indeed, with all this major new functionality -- especially if you've got that gleam in your eye of an environment that's 100% virtual, like so many of you do.  

If you think back a few years to Paul Maritz' use of the term "software mainframe", well, that doesn't look quite so far-fetched anymore, does it?

And then there was that new licensing scheme that was announced ...

The Basics

Few people really like structural changes in enterprise software licensing schemes.  Whether they're good changes or otherwise, an amazing amount of organizational calculus and optimization results from how vendors charge for their wares.  Change the assumptions, and a lot can be impacted elsewhere in an IT organization.

Those changes have to be assessed, analyzed, new recommendations made and communicated, procurement needs to get involved, more meetings, etc. etc.  The larger your shop, the more work is required to do all of these things.  And I think everyone should expect a certain amount of kvetching when changes are made.  

If anything, the noise level says much about the strategic importance of VMware to so many IT organizations around the world.

But software licensing schemes are powerful tools as well: they provide clear signals on how the vendor would like people consume and deploy the technology.  Change the licensing scheme, change how your software is used.  

Yes, there's always some sort of revenue angle (these are businesses, after all) but -- when the discussion gets serious -- you're turing some pretty powerful knobs indeed when you sit down in internal product planning meetings to discuss these very weighty topics.

I have every confidence that the VMware senior management team thought long and hard about what they were doing.  I, for one, am starting to appreciate the beauty of logic here; I can only hope that others will too over time.

Current State Vs. Future State

There must have been at least a dozen before-and-after comparisons that floated around on the interwebs where people shared their hardware configurations under the 4.x licensing scheme and their understanding of the 5.x scheme.  Some people came out worse, some people came out better, some people didn't see a significant change.

While all of that is interesting and educational, I think it's more useful to think in terms of future state, and less about present state.

Every software licensing scheme needs at least one "price for value" metric.  In this space, the choices are obvious: count servers, count sockets, count cores, count storage capacity, count number of VMs, etc. etc.  Or arrive at a new pricing metric entirely: vRAM.

A fascinating choice, indeed.

What I think that some people might miss is that this choice implies that -- in VMware's envisioned state --  virtualized RAM is both pooled and dynamic.  As I understand it, vRAM is the stuff that unique applications actually see and use, and does not include many forms of overhead: memory used by VMware itself, spare capacity for N+1 scenarios, pages reclaimed through sharing and/or compression schemes, etc.

Pooled, shareable and efficiently used resources are potentially *much* more valuable than the static, dedicated and overhead-burdened kind.  

If you consider the storage domain, few would argue against the benefits of pooled, shareable storage resources -- although at one time (e.g. 1995) that was a dangerous and radical thought.  In the storage world, people have become very comfortable with the idea that a pooled, dynamic resource is more valuable (and costs more!) than the static, dedicated kind.

My perception of vRAM is that it's an analagous concept -- to a degree.

But the analogy only goes so far, and there's an important difference: in VMware's near-future world, vRAM is a resource that is requested, used, and given back to the pool when no longer needed.  By comparison, in the storage world, no one ever gives anything back.  At least, not voluntarily :)

Look closely, and you'll see the underpinnings of a new world where applications (or management frameworks) can dynaimcally vary memory used over time -- expand and contract as needed.  Fully realized, that makes vRAM an incredibly useful and powerful resource indeed vs. today's world of largely static and dedicated RAM allocations.

Amidst all the noise, this is what I see: strong economic encouragement for IT architects to start thinking in terms of vRAM as a shareable and dynamic resource, which many of them aren't doing today.  Mainframe apps get dynamic control of memory; why not VMware apps?  And, not surprsingly, there's now an additional economic incentive to create new shared-services apps using a thinner, more lightweight, more modern approach.  

Not that VMware would have any ideas on what that might look like :)

And, let's not forget, those legacy apps that do require ginormous amounts of vRAM were previously running on decidedly more expensive and often less functional proprietary processor/OS combinations.  Perhaps that might make a more useful before-and-after comparison for some?

I can't speak for everyone, but if I look at EMC IT's intended use of Vsphere 5.0 to bring over final round of ginormous mission-critical apps off the monsters they're currently running on, I don't think it'll be hard to make the case :)

Final Notes

For starters, I'm fully empathetic on the disruption casued by any significant change in enterprise software licensing.  I get that, thanks.  And, yes, perhaps the VMware crew might have communicated the changes differently -- some of the sting seemed to be a direct result of "how" rather than "what".  

That being said, I'd challenge concerned IT professionals to take a moment and decide for themselves -- what is the future state environment is VMware envisioning with these changes?  

Don't assume it's all about the money -- it's really about the outcome.

Courteous comments are welcome, as always :)

Comments

[Chuck]

In the world I live in my VMs run operating systems that aren't expecting the amount of installed RAM to vary from minute to minute and it's unnecessarily complicated to spin up and shut down load-balanced VMs in response to workload (if the given application could even support such a scenario) so I allocate the VM as much as it needs to handle its peak workload and sleep easy knowing that I'm using a hypervisor that allows other VMs to take advantage of that memory when it's not being actively used. This has come to mean that more memory is allocated than is being used at any given time. I've realized some considerable cost savings from this over the past 4 years which are about to be collected in arrears.

Fortunately we just upgraded from Enterprise to Enterprise Plus for access to the additional features or we would be looking at additional license costs just to maintain compliance for our current environment. As it stands, a capacity expansion planned and budgeted for next year is going to have to go back to the drawing board because we weren't anticipating any net-new licenses being required.

Is there a shipping product today that will give vRAM back to the pool when it's not being used? Should I wait to upgrade to vSphere 5 until such a thing becomes available? If they wanted vRAM to be seen as a 'dynamic' resource maybe it should be licensed based on how much the VMs are using instead of how much is allocated. It would also be clearer if sockets were removed from the equation altogether and it was just licensed per GB. It's been a pleasure using ESX for the past 4 years as a product that essentially stayed out of the way and let me do what I was already doing better, faster, and cheaper. I'm not sure paying for the privilege of redesigning my environment to conform to their vision is going to be quite as sweet.

[Stu Fox]

Chuck

Ys, there is a shipping product that gives vRAM back to the pool when it's not being used. It's called Hyper-V Dynamic Memory, and does exactly that. It's designed exactly for the scenario you describe - you set an upper limit for peak load, and Hyper-V will allow that memory to shrink and grow as demand requires.

Cheers

Stu
Disclaimer: I work for Microsoft NZ, but this is my personal post, not that of my employer.

[Michael]

I run a couple of smallish vSphere clusters on ordinary twin-socket hosts with >100GB of RAM each. My VMs are ordinary Windows machines, so they aren't going to be dynamically changing their RAM allocation soon. I suspect I am fairly typical of the non-"ginormous" end of the market. The licensing changes make memory overcommit look very unattractive now. Which is odd, because it used to be one of VMware's better selling points.

48GB per socket is a stingy allowance now - look at the hardware. What is it going to look like in a year's time? It isn't just about the disruption, we need licensing that reflects the reality of modern hardware, is predictable, and fits customer needs.

[nate]

Party like it's 2007 - The new vRAM licensing scheme makes me think it's time to bring back 4GB dimms and 1GbE.

As a customer/user of VMware for 12 years (I still have my VMware 1.0.2 CD for linux I believe it was their first release on hard media) it makes me sad more than angry.

Unless they make a serious change I'll stick to vSphere 4.1 for a while longer then move to KVM. I suspect the backlash is becoming large enough for them to be getting cold feet now and they'll probably change direction in the coming weeks before vSphere 5 is released. I read someone else mention a similar scenario last year when VMware was pitching to service providers, there was a massive outcry and they ended up slashing the costs in half but it was apparently still really expensive.

If VMware had a more ala carte offering that could make up for a lot of it. Myself I don't care about a lot of these new fancy management bolt ons that are coming out. I care most about a hypervisor. I ran ESX and GSX before it for years without things like vMotion or DRS or HA. The only reason I would go for something like Enterprise plus is because I'm forced into it due to licensing. I like having the features but they aren't required for the stuff I do.

I didn't see one technological thing introduced in vSphere 5 that got me interested in using it -- even if it did not have this new vRAM licensing. The only thing missing from vSphere 4 IMO is to remove the limitation on CPU cores/socket which is a licensing issue, vSphere 4 itself goes to 160 cores/threads. 16-core cpus are just around the corner.

vSphere 4 by contrast I think was a much more significant technological advancement.

http://www.vmware.com/support/vsphere4/doc/vsp_40_new_feat.html

Compare that vSphere 4.0 to this vSphere 5.0
http://www.vmware.com/files/pdf/products/vsphere/vmware-what-is-new-vsphere5.pdf

I see maybe 5-6 new things in vSphere 5, a lot of "enhancements" on vSphere 4 features like

- Elimination of ESX everything ESXi now (myself I liked the thick ESX so I will miss it)
- vSphere file system - just a new version of VMFS that scales better
- Storage I/O control - already in vSphere 4.1, perhaps they made it better
- vStorage API - already in vSphere 4.1 - looks like they improved it
- Network I/O control - already in vSphere 4.1
- Distributed switch - same..
- HA - same
- vmotion - old news, F5 was showing off WAN vmotion over VERY high latency links a long time ago
http://www.vmware.com/files/pdf/products/vsphere/vmware-what-is-new-vsphere5.pdf

- ESXi firewall - it was there in ESXi v3, they removed it in v4 though only to bring it back in a new improved form
- larger VMs - I can't think of anyone in their right mind running a VM with a TB of memory, I mean come on. Run it on real hardware so your not hindered by 32 vCPU limit, since most systems that can run 1TB of ram are going to be capable of having 80+ CPU cores.

To me all the new high end scalability of vSphere is nothing more than a marketing gimmick. I believe I read somewhere vSphere 5 can do 1 million IOPS. How many customers are going to have more storage arrays than servers? Since there is no storage system out there that can come close to a million IOPS by itself.

Think back to 2008...
http://blogs.vmware.com/performance/2008/05/100000-io-opera.html

Where VMware was bragging about doing 100,000 IOPS on one system, they needed 3 storage arrays to pull it off. Even today the thought of one system needing 100,000 IOPS is just crazy.

Same goes for network throughput.

Ok I take that back - I see one good feature in vSphere 5 that is the new vCenter linux-based appliance and web management console. While I haven't tried to check I assume that is backwards compatible with vSphere 4, just like vCenter 4 could manage ESX 3 hosts.

I am not even considering building VMware hosts with less than 256GB of memory these days.

I don't think VMware understands how fragile their position is in the market. I'm sure some of their big core customers won't budge one bit, but a large portion of the market out there will(including me though I'll be kicking and screaming!)

Don't get me wrong I think VMware is a great technology and I have been an extremely loyal user/fan/customer since their inception, but this licensing stuff is worse than anything even Oracle has pulled(I like Oracle DB too btw).

[LAI Loong Fong]

The new pricing metric is vRAM per socket, not just vRAM alone. What is the point of having VM that support 1TB of vRAM when it is more expensive than a physical machine?! We have been trying to migrate all the large memory machine to VMware and setting up ESX 4 sockets hosts with 512GB RAM but it looks like we do not have enough licenses to support such an environment?! I would need 3 times the licenses to support such an environment!

We have been upgrading through various version of VMware and have been taxed along the way, i.e. ESX2.5 + vMotion -> ESX3 Enterprise Tax -> vSphere4 Enterprise+ Tax -> vSphere5 vRAM tax!! Are we going to have IOPS tax or VMFS capacity tax next??