My Discussions With A Cloud Builder

About a week ago, I had an interesting interaction with Randy Bias.  Randy (@randybias) has a great gig: he's CEO of a consulting firm (www.cloudscaling.com) that builds scale-out clouds much in the manner of AWS.

I found the discussion interesting because Randy's architectural philosophies are in contrast to the ones that VMware, Cisco and EMC are using to build private clouds with technologies such as a Vblock.

And anytime I meet someone who's enjoying success using a different belief system, I am very curious indeed.  Not only do I want to understand their value proposition, but I also want to see if pieces and parts of the EMC value proposition could play a role as well.

To Begin With

Randy currently has a consulting gig with a large telecommunications provider who also happens to be a very large EMC customer.  However, part of the motivation of this particular project is to do things very differently than traditional IT would do it.

The win for Randy (and his client) was if he could achieve his objectives yet stay within the EMC portfolio at least for storage.  This telco was comfortable with EMC and the relationship, so it wasn't exactly a technology free-for-all.

Yet when the EMC team usually shows up, they can come with a mindset that isn't aligned with what people like Randy are typically trying to achieve.

My goal?  Stay within his design philosophies, yet create the opportunity for additional value.

Smart Orchestration vs. Smart Devices

EMC, like most storage vendors, spends an awful lot of time and effort putting intelligence into our storage devices.  Cloud builders like Randy, however, prefer to put as much intelligence higher up the stack at the orchestration layer, reducing the dependency on things like smart storage devices.

However, the two approaches are not always incompatible.  Devices that do their magic transparently to the layers above them are not off-the-table.  And any device that exposes functions via an API can be considered as an option -- as long as the orchestration of these devices doesn't require yet-another-management-console.

How To Pool?

Anything that deserves the title of "cloud" is essentially a dynamic pool of resources, so how and where to pool becomes a matter of architectural interest.  And when it comes to storage, there's a continuum of pooling approaches.

On one hand, we've got modest storage bricks that do what they do completely isolated from its storage neighbors.  Any pooling of resources or capabilities would have to be provided higher up the stack: a distributed file system, or perhaps some sort of dynamic capacity or load balancer.

At the other end, we've now got sophisticated pooling environments like VMAX (within an array) and VPLEX (across multiple arrays, perhaps geographically distributed).

It's not hard to construct scenarios where one approach or the other would be preferred, but that misses the point: the types of clouds that Randy constructs does its pooling far above the storage layer, although there is some interesting "pooling" that can be done within the storage device, as we'll see in a moment.

Capacity, Performance, Availability ... Vs. Breadth And Flex

Propose an arbitrary combination of capacity, performance and availability, and it's relatively straightforward to architect a near-optimal solution that's price competitive.

The challenge comes with not only the breadth of the required continuum (workloads come in all sizes and shapes), but the ability to quickly flex up and down to a new point non-disruptively.

Indeed, the ideal cloud architecture would cover a very broad swath of potential capacity/performance/availability points using a common, standardized approach, as well as be able to dramatically change the delivered service level on very short notice -- the shorter the better.

Indeed, I believe this "pricing to the changes" capability will be a core issue for anyone contemplating an IaaS model, or most anything built on an IaaS model.  People will pay a premium to guard against unforseen circumstances.  And, as anyone who's done IT for any length of time certainly knows, perfect planning is an oxymoron.

What The Discussion Boiled Down To ...

Randy basically wanted a RAID brick -- nothing too large as he wanted to limit fault domains.  The brick had to support 10GbE iSCSI, and offer decent price/performance.

In the current EMC portfolio, that would likely be a CX4-120: a modest dual controller design that supports up to 120 storage devices (disk and flash) and -- due to its flexible front end design -- could potentially be populated with as many as 8 10GbE iSCSI ports, although it's hard to imagine all of that actually being required.

This part of the marketplace is extremely competitive: there are many good dual-controller designs out there from a number of vendors, so no one can get away with either charging a premium or offering tech that isn't up-to-snuff performance-wise.

So far, so good -- but what else could I put on the table to make it interesting?

Managing The Array Externally?

The new Unisphere management environment is about as open and modular a storage management stack as you could envision.  This means that anyone looking at higher-level orchestration can directly access the array at just about any level they desire: extremely low-level and very granular, or very high-level and conceptual -- or anything in between.

We've already done a lot of "plugging in" of this storage management stack with not only EMC's other management tools, but hypervisors such as VMware's vCenter.  The interfaces are pretty generic and aren't designed to be proprietary.  That sort of open management approach works well here, I would think.

Any Transparent Magic?

I did have a chance to talk about about FAST -- fully automated storage tiering.  This newer capability allows cloud builders to pool small amounts of solid state storage devices alongside more cost-effective bulk SATA drives -- and automatically move pieces of data around based on usage patterns. 

Add in FASTcache -- a large, non-volatile storage buffer that soaks up big spikes -- and you've got the ability to deliver a storage brick that's arguably both faster *and* cheaper than any traditional alternative.

The other thing that falls into this "transparent magic" category is the new "LUN compression" feature that comes with the CX4.  Simply turn it on, and it will quietly and transparently squeeze excess capacity with no one being the wiser.

Both can be used in a 'set and forget' mode.  Although it's possible to check in and monitor, reconfigure settings, etc. -- both were intended for environments without dedicated storage administrators -- such as a cloud.

Not only that, both technologies could deliver value to Randy's client -- and do so without compromising his design approach.  The customer got the ability to not only deliver a more cost-effective service, but quickly flex storage performance, and do so in such a way that didn't require complex orchestration.

Not only that, if Randy and his team decided that they didn't like Unisphere, or FAST, or LUN compression -- no problem.  It's still a competitive array without those features.

The Bottom Line

After getting Randy started, I had to turn him back over to the local sales team.  I felt a bit guilty about doing this.

Our sales teams are great when supporting traditional enterprises, but nothing in our sales training points to the importance of people like Randy, or how they prefer to be engaged with.

Hopefully, I've been able to establish enough of a relationship with him that he'll reach out to me if he isn't getting what he wants from the local EMC team.

But what I really want to track is how these modest storage devices perform in the cloud he's building -- not theory, but the real world -- and how they compare to what he's done in the past.

Comments

[Renaissance_Kay]

Nice alternative view. Let me think more...

[Chris S]

"The new Unisphere management environment is about as open and modular a storage management stack as you could envision. This means that anyone looking at higher-level orchestration can directly access the array at just about any level they desire: extremely low-level and very granular, or very high-level and conceptual -- or anything in between."

That quote makes it sound like there is some new API or web service interface introduced with Unisphere. Is that true? The closest thing I was aware of was the Navi(Sec)CLI, but that's not an ideal interface for programmatic management from some higher-level orchestration system.

[Chuck Hollis]

Hi Chris

If you drop an email, I can make sure you get the docs and whatever you might need.

hollis (underscore) chuck (at) emc (dot) come

-- Chuck

[Amiya]

Regards for the post! Here's the search engine on rapidshare ( http://filecraft.com ), which will be of good help in finding files for download