I think many IT professionals realize that many application performance issues eventually boil down to storage and physical I/O. That was true before server virtualization, and it's certainly true now.
Storage array vendors do what they can. Operating system, hypervisor and database vendors do what they can as well.
But between the two, there's the potential for a smart layer of storage software that does what others can't.
As an example: way back when Solaris was popular, Veritas' VxVM and VxFS were almost ubiquitous. Both products offered an important value-added layer than neither the host OS nor the storage array did well. As a result, the Veritas products become almost a de-facto standard in an larger Sun environments.
While I was exploring Virsto (recently acquired by VMware), I was struck by similarities to what Veritas did years ago. It's clearly a smarter storage abstraction layer than either the hypervisor or the array can provide on their own.
The real question — now that Virsto is owned by VMware, will Virsto technology end up being a de-facto standard for many VMware environments?
What Brought This About
Like many of you, I try to follow what all the various storage startups are doing. Although Virsto was formed way back in 2007, they really didn't start to cross my radar until 2010.
Their argument was simple: highly-virtualized environments presented storage demands that weren't ideally addressed by either VMware or the array vendors. Server virtualization was causing an "IO blender" effect that could benefit from a smart storage abstraction.
While I could accept their basic premise, I was a bit skeptical that they could do anything about it, especially doing so at the software layer -- and in a way that didn't compromise either VMware or array-based functionality. I spent a good amount of time digging into the details, I came away impressed.
Although their solution doesn't improve every IO problem, it creates enough of a benefit in enough situations that it's worth seeing if it could help in your specific environment.
Virsto At A High Level
Since it's a safe bet that most of you reading this have a decent storage background, I'll get right to the meat.
Virsto software creates a new storage abstraction between VMFS and the storage array. It introduces a log-based filesystem that acts as an "assist" to VMDK volumes.
As a result, performance can see a measurable boost in many situations -- especially when you're a heavy user of snaps and clones. Virsto can also improve performance predictability and fairness across multiple VMs. And thin provisioning is now fast, efficient and also performant.
It does all of this with your choice of server and storage array. It manages well from within VMware, and all the important VMware functions "just work": vMotion, DRS, etc.
Virsto starts by aggregating external array storage into a vSpace. A vSpace uses a log-based file system to present vDisks (VMDKs) to VMware images. A vSpace pool can be tiered, if you choose.
Every server that's hosting VMs has a dedicated log device where all writes are funneled to. Ideally, this small (~10GB) buffer lands on your fastest available external storage: SSD, RAID 10, etc.
Log volumes should use shared protected external storage — if nothing
else but to enable failover across nodes. Put them on a relatively
small amount of protected SSD, and performance apparently leaps forward.
Log writes are eventually destaged back to the greater vSpace, where their log metadata helps to implement extremely efficient and performant snaps and clones, as well as thin provisioning.
The per-node logs are the beginning of the performance story: random writes are written sequentially to the log file, boosting performance from the perspective of the guest VM.
A per-node log flushing process then can leisurely destage written data back to the larger vSpace in an efficient and orderly manner, giving the array a potential assist in smoothing out random write IO bursts.
Reads can also get a significant bump here, and the reason wasn't initially obvious to me as I looked at the performance charts.
The logs are large enough to catch a significant number of re-reads after writes, and if you've used a small about of SSD for your logs, those read requests are coming from SSD, not disk — which turns out to be a very efficient take on storage caching using flash. The remaining reads are satisfied from the vSpace back-end pool, where data has been laid out carefully thanks to the log flusher.
One IO profile that doesn't seem to benefit from this approach are near-pure sequential large-block reads and writes. As IO profiles move towards more randomness and a significant write component, the approach shows its strengths. As the workloads move farther towards pure random reads or pure random writes, the approach doesn't show its strengths -- but does quite nicely with your garden-variety workloads.
However, the performance story starts getting much stronger as we look at heavy snap/clone usage.
Snaps And Clones
The snap/clone implementation appears very modern, and compares favorably to what I've seen both in VMware and many array implementations.
Since every write is logged, snap/clone implementation is straightforward. When it's time to create a snap, a "marker" indicates that previously logged data shouldn't be changed, and new writes should be written separately.
The implication is that reading and writing snap/clone VMDKs are comparatively quite performant as compared to many existing snap implementations.
Start writing to a snap, and you've created a clone -- again, new data is logged separately. Thanks to the vSpace implementation, performance of any snap or clone is very predictable, and performance holds up as more and more snaps and clones are made. Creating a new snap/clone is almost instantaneous as well — worth consideration for things like VDI environments or test/dev scenarios.
The same mechanism enables thin provisioning as well: very efficient and offering the same high performance as fully-provisioned volumes.
IO Fairness For VMs?
A team at EMC brought Virsto into the lab, and started running multiple IOmeter tests -- and found another interesting effect: better I/O fairness for each and every VM. While many of us in the storage business tend to look at performance in aggregate from the array side, it's often more important to understand what each and every VM is receiving in support of an application.
The Best Of File And Block For VMware
One of the most historically debated topics in storage for VMware is file vs. block. Without taking sides, the perceived tradeoff is convenience and simplicity vs. raw performance.
Provisioning and managing VMware storage as an NFS file system can be more straightforward as compared to a block storage model; conversely, aficianados of block storage will point to performance: low latencies, ability to tune, no file system in the data path, etc.
I'm not agreeing with either side, I'm just sharing what the debate is :)
Virsto splits the difference very nicely — it presents and manages via the NFS storage model found in VMware, but doesn't go as far as to implement a full POSIX-compliant NFS share. The back end is pure block storage — fibre channel, intelligent MPIO, and so on. Very impressive.
Where Does Virsto Fit?
Many of you reading this might be thinking -- doesn't the array (or vSphere) do many of these things for me? Yes -- and no.
Virsto seems to fit best when:
- you're starting to reach the performance limits of your current array(s), and are looking for a quick non-hardware boost.
- you have a large number of smaller random-IO intensive workloads that want to run faster
- you want to get a great bang-for-buck on a small amount of SSD
- you are looking for high-performance snaps/clones: both creation and usage
- you want a high-performance thin-provisioning solution -- creation and usage.
- you are experiencing issues with IO fairness across VMs
Conversely, Virsto doesn't offer much performance-wise if your IO profiles are dominated by extremes: pure sequential reads/writes, or pure random reads/writes.
This is not as great a limitation as it might sound -- the world is chock-full of application workloads that occasionally burst heavy writes, and that's where the base performance story plays. Add in heavy snap/clone usage, and performant thin provisioning, and that's the basic story.
VMware Integration, Of Course …
Virsto— now being sold as a standalone product by VMware — is about as tightly integrated into the VMware stack as is humanly possible: management integration, installation integration, all the cool VMware features just work, etc.
Although a bit of storage background is helpful when initially installing the product, day-to-day operations are entirely conducted within the VMware management envirornment, and by the VMware administrator.
I was. Perhaps the best detailed explanation I've found is an early draft of the Virsto Theory of Operations document. Good reading, and you'll come away with a great understanding of how Virsto works very differently than either vSphere or intelligent arrays.
The only way to find out if Virsto can help your environment is to try it out. That's relatively easy to do. Yes, there's a short reg process -- but that's mostly so someone can help walk you through the install process at the outset.
And if you're already a Virsto user — why not share what you've seen in your environment?
I'm sure a lot of folks would be interested.