NAS Evolved

Today, EMC announced several major enhancements for the Celerra platform.  Taken individually, they're interesting.

But taken together, I think this announcement shows just how far NAS technology has evolved in the past few years.  If you haven't been paying attention to what these platforms can now do, maybe you could use a refresher?

What You Need To Know

NAS has always been about having an effective place to store your files.  Sure, we've seen block-mode emulations on NAS, and we've seen people put smaller databases and such on NAS devices, but -- far and away -- it's always been about files: lots and lots of files.

The structure of the NAS market has been the same for a while.  There are two large competitors with the majority of market share (EMC and NetApp), with EMC being ahead in total revenue for quite some time, according to IDC.

This legendary industry rivalry means that we have a convenient measuring stick to compare and contrast what EMC is doing with NAS platforms against what NetApp is doing.  Stay tuned, I'm sure they'll having something to say about this latest announcement.

One thing you'll notice when you look at both vendors is that we tend to solve the same sort of problem in different ways, usually as a result of the different engineering philosophies found at the two companies

As one example, NetApp intermixes snap data in the same volume, and EMC creates separate areas for retained snap data -- each approach has pros and cons.  EMC does native FC from the underlying CLARiiON or DMX array, NetApp emulates FC disks on top of their file system.

Again, each approach has a different set of strengths and weaknesses.

So, let's take a look at what's in this latest release, shall we?

Making Files Go Really, Really Fast

Performance is an ever-popular topic in NAS.  Sometimes it's making big files go fast, other times it's about aggregate performance, or lots of random file opens, or ...

No matter how you think of NAS performance, there are some excellent goodies in this release.

First, consider the awesome NS-G8 ("gate", get it?) -- up to 8 blades (each with dual quad-core Xeons), and the ability to support a mind-numbing 32 10GbE interfaces, running either NAS or iSCSI -- and! -- an additional 32 native FC back-end channels (4Gb FC today, 8Gb and FCoE tomorrow). 

You'll have to admit, that's a heckuva lot of crunch if you need big NAS horsepower -- probably the largest and most powerful single NAS device available in the industry today.

Second, all of the new Celerra unified platforms (as opposed to gateways) are based on the brand-spanking-new CLARiiON CX4 family - arguably the most up-to-date and highest-performing mid-tier array in the industry right now.

And, since CLARiiON supports EFDs (enterprise flash drives), we now have this very special performance sauce available now Celerra to complement what's already available on the DMX and CLARiiON. 

Much has been said about EFDs already in the storage community (and more will certainly be said!), but it's hard to argue with the simple elegance of popping in a few compatible storage devices that offers 30x the IOPS of any rotating disk drive.

If you're only looking for huge memory caches (as opposed to EFDs), consider the DMX family as back-end storage -- unlike competitive approaches, the DMX sports an enormous non-volatile cache that can be used for reads *and* writes.

Caches can only help with portions of the overall performance equation, though. These EFDs go much, much further and deliver a level of blistering performance that can't be matched by clustering approaches, or read-only caches.

We also have enhancements to EMC's MPFS (multi-protocol file system) that delivers block-protocol performance while still retaining a filesystem view.   

If you'd like to have the ability to deliver spectacular performance in your bag of NAS tricks, you should really take a look at what the newest Celerras can do. 

Faster is always better, right?

Making Files Use Less Space

OK, NetApp was early to market with a feature initially called A-SIS which allowed users to deduplicate selected volumes of primary storage.  (Minor note: NEC was first with HydraStore, but you don't hear of them much).

Turned out to be a pretty popular feature, by all accounts.

Not to be outdone, EMC invested considerable effort in coming up with a primary dedupe scheme that's we think is a better approach – much easier to use, and we think much more efficient across a wider set of use cases.

At a high level, EMC's approach generally allows larger file systems to be tackled, provides selective filters around what's to be deduped and what's to be left alone, and -- most importantly -- adds a new compression capability that does well in situations where traditional dedupe doesn't do as well.

Digging a bit deeper, you'll also find CPU throttling for the dedupe process, as well as a new technique that allows restores of deduplicated full filesystems without having to pre-allocate full-sized volumes, as has to be done in the NetApp approach.

And, of course, all of this sits behind a slick GUI, so there's no need for a periodic running of a command-line task.

When you dig into the details, you'll notice that we're explicitly not going after attempting to dedupe "hot" data that's usually found in iSCSI and FC LUNs and certain parts of the file system. 

That's for a reason -- dedupe processing can get in the way of performance, and we thought there was plenty of room for efficiencies with "cooler" data sets, not to mention the potential futility of trying to find redundancy in data that's continually changing.

Regardless of how you feel about one approach or another, there's now some clear choices for customers around primary data dedupe in NAS environments, and that's a good thing.

Making File Storage More Efficient

Dedupe has now been cleverly integrated into a nice, integrated suite of features that make overall space provisioning more efficient, including virtual (or thin) provisioning for file systems, Automatic Volume Management that will keep an eye on your NAS storage growth for you, and a new Celerra Provisioning Wizard that makes it all easy to use.

It all just works together, which is nice.

It should be noted that the deduplication, SnapSure, virtual provisioning and the Automatic Volume Manager features are all included with every Celerra at no extra charge.  Given the competitiveness of the NAS market, I think you'll see other vendors following suit before too long.

Automating File Management

And then there's the challenge of ensuring certain files stay around and are unmodified per compliance policy, as well as the inverse of making sure files are deleted when they're supposed to be.

EMC continues to do well with Centera in tackling this problem at enterprise scale, but not everyone needs a full-scale, dedicated solution to this particular challenge.

So, you'll find some nifty features in Celerra to do the compliance-and-retention thing as a simple extension to the NAS environment, if customers want.  There's the ability to selectively choose files to be retained based on metadata, or selectively deleted when they're no longer wanted. 

Again, more choice is a good thing.

Better Integration With VMware -- Of Course!

The Celerra is the first of EMC’s platforms to get a killer new capability – automated, integrated VMware Site Recovery Manager failback.   Lots of vendors can failover, for now, only EMC can do automated failback.

BTW, EMC appears to lead the VMware SRM storage platform market by a longshot – I've seen data that more than 40% of the SRM customers are using EMC, with the next closest vendor 3x less.  

That sort of market penetration means we tend to hear a lot about what customers want.   SRM is a game-changer, but the number one feature requested is automated failback.   

With this release, we made a Virtual Center plug-in that takes a complex manual process and makes it as automated as the failover by integrating with the Site Recovery Manager database and Celerra Replicator.  I think it's the first of its kind -- anywhere.

The plan is to quickly expand it to include all the EMC replication products and platforms.  EMC Celerra customers get this for FREE starting today – and everyone we’ve shown it to is blown away.  So, if you're doing VMware SRM, be sure to ask for automated failback!

The EMC team also built a Virtual Center plug-in designed to make very rapid scale out of the Virtual Desktop use case easy – blending array-based snapshots and VMware View.   All of that gets added to the long, long list of other places where the Celerra directly integrates with VMware.

There's More ...

... but I'm getting tired, and this post is getting pretty long, even by my standards.  You could spend many hours poring through all the features and capabilities of the EMC Celerra.  More local snap functionality.  More remote replication functionality.  Integration with policy and quota applications.  Etc. etc. etc.

[update: as I collect various links, I want to share them: here, here, here, here and here.  More bloggers weigh in here, here, here and this very useful one here.]

NAS has come a very long way indeed.

Strong competition spurs vendors to bring their best game, and that's what I think you're seeing with this latest Celerra release from EMC.

Let the games begin!

Comments

[KPC]

Hi Chuck,
Nice to see EMC evolve its NAS a bit.
But the major issue is with Datamover limitations. Celerra can't scale a filesystem beyond one datamover. No point in wasting time and effort in increasing speeds and Feeds. Some killer feature development will make Celerra line better. MPFS, AFM, Snapsure all are been there for years since DART 4.x.

The top NAS dog is still NetApp and now Isilon, BlueArc are gearing up.

Some Ideas,
(1) Integrate Rainfinity inside Celerra for customers looking at Global Namespace and migrations
(2) Integrate Celerra Replicator with Recoverpoint replications to make it Unified Replication for all
(3) Clustered NAS like NetApp GX or Isilon

Good Luck

[Chuck Hollis]

Hi KPC -- thanks for sharing your opinions.

Faster speeds and feeds means that we can make individual filesystems run faster -- some people appreciate that without having to get into the whole clustering thing. More CPU can also help with making dedupe more performant.

We're all still waiting for NetApp's Spinnaker-based clustered solution ... maybe this year? Maybe next? I've lost track of how long it's been.

I guess "top dog" means your personal favorite, no? IDC market share points to EMC as #1, although that's not the only valid criteria out there.

Love your product ideas -- I think most of them have been discussed at one point or another. No one gets to rest on their laurels in this part of the market, no?

Thanks again!

-- Chuck

[Vaughn Stewart]

Chuck,

Thanks for the update on the Celerra. As I read the post I had two questions which I would ask you clarify for the readers.

1. "NAS has come along way..." In your article you mention SRM. As presently SRM only supports LUNs, I am confused. Do you position Celerras LUNs as NAS?

2. On "Primary Dedupe..." Does Celerra Dedupe support Deduplicating production Virtual Machines (or VMDK files)? Your statement isn't clear as to what types of data are supported, and wether it is file level or block level deduplicaiton.

Can I ask you to shed more light in these two arras?

Thanks,
Vaughn

[Alex McDonald]

Chuck, congrats on beginning to see the NAS light. Just a few points of clarification for you to address, in no particular order;

1. SRM v1.0.1 does not provide a means to automatically fail back, so I presume the plug-in you refer to reverses the replication relationships of the Celerra? There's more to SRM than just breaking mirrors and presenting storage; it also allows you to control the order of VM startup and change/customize VMs as needed for the site being failed to.

2. SRM is block based, so inclduing it as NAS appears to be a bit of a stretch? Or are you referring to the iSCSI from the Celerra as pposed to using the seperate and different iSCSI on the CX4 backend?

3. "EMC does native FC from the underlying CLARiiON or DMX array, NetApp emulates FC disks on top of their file system." As you're talking NAS and not SAN here, can you explain what you mean by this rather odd statement?

4. "restores of deduplicated full filesystems without having to pre-allocate full-sized volumes, as has to be done in the NetApp approach." Why do you think this is necessary with a NetApp solution if the source is a snapshot? How does the Celerra avoid this problem with NDMP backups?

5. "BTW, EMC appears to lead the VMware SRM storage platform market by a longshot – I've seen data that more than 40% of the SRM customers are using EMC, with the next closest vendor 3x less." Can you tell me the source of this data please?

Thanks in advance for clarifying.

[Chuck Hollis]

What happened, did the NetApp blogging crew get some sensitivity training? My, you're polite these days!

Alex:

1. Yes, you're right. Chad's demoing it at VMworld right now, it's a pretty slick integration. More details to follow.

2. I'll let Chad handle the first part, but -- as we all know -- Celerra and NetApp both can present as block devices as needed.

3. Not going to fall for that one again.

4. We're talking full restores here. With A-SIS, you're backing up (and restoring) re-hydrated data, no? In the event that a full restore is needed, you'll need the extra capacity to recover to, and then you can re-dedupe. I'll get one of the guys to post on how we got around that particular problem.

5. I don't know where the data came from, or if it's publicly availble. I get so much stuff in front of me, ya know?

Vaughn:

1. I think we're arguing semantics here -- it's not a productive discussion. Would you like to comment?

2. There are more detailed posts coming -- watch for them!

-- Chuck

[Alex McDonald]

Politeness is my middle name.

1. I await Chad's latest smoke and mirrors with bated breath.

2. So the SRM stuff isn't NAS. Now you have 2 iSCSI methods, one of which is emulating a "LUN-as-a-file" (Celerra iSCSI) on a "real FC LUN" (CX4 backend).

3. Good, you're on a looser with that statement given "LUN-as-a-file" (Celerra iSCSI) on a "real FC LUN" (CX4 backend). So why'd you say it? I'd give up the real vs emulated FC deal if I was you, it's a shot point.

4. I've just seen StorageGorilla'z VBB post. I get it; it's an extension to NDMP. Needs a checkpoint or a read-only filesystem to back up though. No, we don't need to re-hydrate a volume. Just switches pointers. No data movement/re-hydration at all.

5. You made the data up? That's not cool.

[Chuck Hollis]

My, Alex, the politeness wore off rather quickly, didn't it?

1. I think Chad's reputation in the communinty speaks for itself.

2. Huh?

3. The word is "loser", rather than "looser" I think.

4. I think you're going to need more of an explanation than that!

5. I didn't make the data up. I can, however, track down where I saw it, and share that with you if you'd like.

BTW, check out the funny NetApp competitive YouTube video here: http://www.youtube.com/watch?v=QSsNs6LPqT0

Keep that competitive blogging up, Alex! We're just warming up here!

Best regards,

Chuck

[Alex McDonald]

My, you're touchy. I'll use smilies.

1. Who's knocking Chad? The man's a magician, of that there's no doubt. :-)

2. Double huh? Aren't you aware of how a Celerra does iSCSI? :-o

3. Thanks for the spell check. :-\

4. Yes please.

5. Yes please.

:-D

[Chad Sakac]

Ok, trying desperately to stay above the fray:

1) Celerra Dedupe can reduce the amount of storage consumed for a broad set of use cases. It is primarily designed for the biggest consumer of NAS data, and the place where we felt like we could have the biggest impact, which is unstructure general purpose filesystems.

Being an engineer about this....

Alex - that hurt man - smoke and mirrors.... geez. There's been MANY times where I've bit my tongue because I don't want to play the game you guys play - ahem - SRM 1.0.1 HA issue, NFS.DisableLock, I intentionally didn't say anything because that's a below the belt punch, and are the things that bedevil us all - solutions integration is HARD WORK. I'm glad I don't have "competitive" in my job function

Ok, back to the topic... it eliminates files that are the same, and compresses the remainder. It avoids open active files. That means that today, for the NFS VMDK use case, there would some, but only moderate dedupe as a result (couldn't compress while the VMDK was active). Now, note that the PR talked about View Composer/Celerra Dedupe integrated solutions (along with a Virtual Center plugin to that effect) in the VDI use case.

Honestly, while View Composer isn't perfect (I did a post on that here http://virtualgeek.typepad.com/virtual_geek/2008/12/vmware-view-composerlinked-clones---they-are-not-a-panacea.html when we posted the joint VMware/EMC reference architecture on storage for View), the key is that for the sophisticated image management tasks, VMware is in a better architectural position than the array (as we have - at least today - no idea what objects we're dealing with, they are just files or blocks)

Also, I will say this, **I** think think you guys were right on the growing importance of primary storage dedupe. We are also right/first in many cases - there is no tyranny on good ideas, only on control on intellectual property -specific implementation details.

We're pretty confident in this mechanisms dedupe capabilities in the general purpose NAS use case, and Netapp and EMC will clearly make arguments over total system efficiency (all the factors, including RAW->Used->Deduped Used).

BTW - one thing that wasn't clear in the PR. This dedupe functionality is in the mainstream release codebase. If you bought a Celerra in the last 2-3 years, merry christmas :-)

We're also happy to compete at any customer for TOTAL efficiency of the solution (production, backup, archive - on performance, capacity, and power vectors) - for ANY workload and use case. It's a CANARD to position dedupe rates (I apply that equally to all of us) without thinking in terms of total system efficiency, no?

Also, just to be transparent - this is only the first foray of an ongoing effort - we think in this area (as Chuck said) - you guys led the way - primary storage dedupe (for a broad set of use cases) will become a "everyone needs to have it" feature.

As much as I hate to say it (cringe) based on my kool-aid drinkin', VMware-lovin' nature, VMDKs aren't the only storage consumer in town, and for an array - the name of the game is the total efficiency (on all the vectors) availability, flexibility, and use case integration at the end of the day.

On to the SRM points.

Vaughn - as you know, NFS datastore SRM support is currently targeted for SRM u2, but that is changing based on the upcoming VMware release schedule. We're ready in the Beta (I've got a bunch of customers lined up and ready to go), and I'm sure you guys are too. I would fully expect us to be neck and neck on NFS SRM support, but we depend on the SRM APIs there. So, what are we talking about? iSCSI on the Celerra of course.

SRM Automated Failback - Alex, you're absolutely right. There is a LOT more to SRM failback than just reversing the LUN replication, and that's what we're talking about - it's why I personally hate talking about what others do, and try to avoid it. It's so easy to be wrong.

So, one thing we did early in all our SRAs is automatically do a personality swap after a failover, but this takes it to a whole other level.

We heard the pounding for this feature, and took a protoype to VMware about 6 months ago as a proof case. In fact, for the greater good, we've been discussing how to pull in the general purpose SRM failback code base. YES - literally working to accelerate the mainstream function to enable our competitors - for the good of the market. WE CAN, WILL, AND DO COMPETE (and win the customer choice) on those grounds - happy to compete with our replication portfolio vs. anyone (not denigrating others, but rather measured confidence in our differentiated capabilities - how about SRM if a customer needs Sync? Heterogenous? Continuous? Consistency Groups across datastore groups?). Every approach has pros/cons - a pragmatist looks at it that way.

what it does (and I will be demoing this tomorrow, and it's available for download on powerlink for free this week) is that it looks at failed over replication states, correlates it with the VMs on the datastores, reads into the SRM DB for certain recovery plan policies. Being transparent, there's some stuff it doesn't do today (re-IP for example), but count on us to make that very, very soon. At the same time, we continue to work to try to accelerate the mainstream SRM APIs development in this area. What the VMware team has planned in the long term takes this to a whole other level, but that's a LONG ways out. EMC customers get this NOW.

Furthermore, we will be rapidly expanding this to cover all EMC's SRAs - from the smallest of the small, to the largest of the large, all products, all functions.

BTW - the source of the SRM stats are the VMware.com SRA statistics (excluding the internal use case stuff). I'm sure my NetApp peers get a similar report that I do every month from VMware - SRM downloads, what SRA a customer uses. VMware holds to their commit of independence - they won't tell me who is number two or any of there stats, and only EVER give me the data on all the EMC SRA downloads, but will express it as a fraction, and knowing how good you guys are (and you're good), I'd bet you're number two.

The new Celerra Virtual Storage Appliance with this functionality will be posted on virtualgeek as soon as I get back from VMworld (Vaughn, miss you out here!), so you guys can do with it as you will (and the market too).

Oh, and that was only 1 of 3 Virtual Center Plugins immediately available for EMC customers. The other two are doozies :-)

There's more coming this week of course!

[Chuck Hollis]

Thanks, Chad -- nice to know you're covering my back when the NetApp Deathmatch Competitive Blogging crowd starts weighing in!

Can't they take a little competition?

Sheesh!

-- Chuck

[Alex McDonald]

@chad

I've been compared to a clown on crystal meth by Storagezilla, so (as he said to me when I suggested he was out of order); settle down. By comparison, being called a magician is a compliment of the highest order. There is a distinct lack of reciprocity in EMC's blogging etiquette.

Smoke and mirrors. In a virtual world, what else is there?

On to business. The reply is appreciated, and I'm sure Vaughn Stewart will make more of it than I can. As to my other points, it would be nice to at least get a response on the question of attributing Chuck's statement

"BTW, EMC appears to lead the VMware SRM storage platform market by a longshot – I've seen data that more than 40% of the SRM customers are using EMC, with the next closest vendor 3x less."

to a reliable source.

Thanks.

[Chuck Hollis]

Alex, I empathize with your role as a competitive analyst for NetApp. Trust me, I know how difficult and unpleasant your job can be.

And right now, your job is especially challenging, what with all those smaller vendors eating into your market share, never mind EMC doing you one better at dedupe, flash and a bunch of other topics.

We can understand your need to lash out and rant occasionally. Don't worry, we forgive you.

You're not alone, though ...

Val now has to spend his time defending an ancient user interface, promising (once again) good things will come to those who wait, and wait, and wait -- that is, when he's not exploring his latest conspiracy theory.

None of us would be surprised if the next one included Elvis and UFOs.

Kostadis appears to have lost his way again, and now has been reduced to rambling on about storage virtualization -- maybe he can take lessons from Hu and the HDS team?

I mean, the whole virtualization thing has worked out so darn well for HDS, hasn't it?

And none of you NetAppians have a word to say as to the complete disappearance of Spinnaker. Now THAT'S a conspiracy!

We were thinking of sending out a search party ...

But you have to admit -- all in all -- some pretty amusing stuff comes out of you and your NetApp compatriots in your collective never-ending quest to prove the world wrong and yourself right.

We all have our personal favorites.

As far as your latest demand, I think Chad answered that in his comment reply -- the data I saw was a report from VMware itself which is not publicly shared.

That's their choice, not ours.

We know that VMware provides the same stats to all its vendor partners, Vaughn probably has access -- what is NetApp's figure?

C'mon, share with us, won't you? :-)

-- Chuck

[Alex McDonald]

Chuck; thanks for pointing out the source of the SRM data. I missed it on first reading, as I couldn't match it up with your original statement.

I would ask you to consider removing the statement from your blog on a few grounds;

1. You said; "EMC appears to lead the VMware SRM storage platform market by a longshot - I've seen data that more than 40% of the SRM customers are using EMC".

Chad's statement says that the data is from an internal VMware report to EMC, and is a count of "SRM downloads, [and] what SRA a customer uses". The two are not synonymous, and to conflate downloads with market share is disingenuous to say the least.

2. Both you and I have access to internal reports from various sources. Most are quite clear on how they may be used. I don't have access to our version of the report, but you clearly say that your report was for EMC use only, not to be publically shared. Manipulation and public use of data, when it is clear that the data is for your internal use only, is unethical. Maybe even a breech of contract.

3. The statement is not verifiable by a member of the public, and some of your readers will be misled into thinking that this is indicative of EMC market share.

I don't care that EMC own a sizable chunk of VMware. If I had done the same and published a statement like yours on my blog, VMware and EMC would have rightly been all over me like a rash.

The rest of you reply is patronising, and all the more unremarkable for that.

[Chuck Hollis]

Alex -- relax, take a deep breath, and enjoy the journey!

I know I am!

-- Chuck

[Stephen Foskett]

Let me just say, for the record, that I'm really glad I'm not on the receiving end of some of these comments. Ouch! Can't we all disagree without being disagreeable, as a wise man said?

As for the Celerra update, in the words of my favorite film, "he's a suitor!" The model range has a nice wide span with three distinct personalities, and the feature list is impressive. I welcome EMC to the primary-dedupe party, though perhaps the wide range of options might be a little off-putting. Yes, the customer can pick what works, but do you really need four different compression schemes that can work together, so there's, what, 10 different options?

I'm definitely watching what happens on the Iomega side, too, since so many folks need a much smaller $$ NAS unit, and the LifeLine products are so good (if ugly).

[Snig]

So your dedupe as stated by Chad sounds more like single instance storage with compression of files than true dedupe to me. Am I missing something? It sounds a lot like Avamar running in the Celerra to me.

Are you "deduping" files or blocks? If blocks, can they be variable length or fixed only?

[Chuck Hollis]

Hi Stephen -- always good to hear from you!

Yes, it can get rather bitter, especially with the competitive bloggers, but -- hey -- everyone has their own agenda, right?

Yes, I'd agree, the Celerra family has moved up a bit in the world with this latest round of announcements.

Even though there are a variety of dedupe/compression options, the user interface couldn't be simpler: a simple yes/no as to whether you want it or not for your file systems, and some nice stats on how well it's working for you.

All the complexity is hidden from the user, which I think will be appreciated by many :-)

I agree with you on the older Iomega products. I took one home, and I guess it didn't pass the WAF (wife approval factor) test, so it was banished to an obscure corner. Some of the newer stuff is looking better, IMHO.

Thanks for writing!

-- Chuck

[Chuck Hollis]

Hi Snig

I think I know the answer here, but I don't want to get it wrong, hopefully I can get one of the other EMCers to answer this.

Back to you in a bit ...

-- Chuck

[Snig]

I found the answer over on Zilla's blog.

http://storagezilla.typepad.com/storagezilla/2009/02/unified-storage-file-system-deduplication.html

[kostadis roussos]

Chuck,

I

> lost my way

?

If only. If only.

kostadis