« Really Long Wires | Main | Back To Flash (Again) »

November 10, 2008

TrackBack

TrackBack URL for this entry:
http://www.typepad.com/services/trackback/6a00d83451be8f69e2010535e1cdbf970b

Listed below are links to weblogs that reference EMC Atmos (Maui) Is Here:

Comments

Reuven Cohen, CTO Enomaly Inc

A while back I came up with a term I described as the " Content Delivery Cloud" I think the approach of EMC's cloud optimized storage fits into this concept very nice.

Here is an overview of CDC
A Content Delivery Cloud is a system of computers networked together across the internet that are orchestrated transparently to deliver content to end users, most often for the purposes of improving performance, scalability and cost efficiency. Extending the model of a traditional Content Delivery Network, a Content Delivery Cloud may utilize the resources of end-user computers ("the cloud") to assist in the delivery of content.

More details here > http://www.contentdeliverycloud.com

Stephen Foskett

Thanks for that quick answer to my central question, Chuck!

It seems that any time EMC releases anything, observers want to position it as earth-shattering. But that's just not the case, at least in terms of customer scope. EMC has a wide range of products, from LifeLine to CX to DMX to Centera to Atmos, and while the whole pile covers the market, none is applicable across the board. It looks like Atmos will have a few very large customers, for now at least, and it looks like that's fine by you.

It does seem a bit out of place to call it "a new era for IT" on the product page, though. Cloud computing is a new era, but Atmos isn't, at least for IT in general.

Stephen

Chuck Hollis

Hi Steve -- agreed -- "a new era for IT" is a bit over the top! It's cool technology and all, but it doesn't solve world hunger, does it?

Thanks for commenting!

Richard Brannigan

Where is the software to let me run this in my VMware guest? The only thing I could download on your site was a PDF with a lot of expensive looking racks. I would suggest the following edit to whomever ghost-wrote this post, "it works fine as a VMware guest, but we won't let you do that."

Joseph Martins

While this is certainly different, the individual elements aren't new...at least none appear to be new to me and I've been in the information management game for more than a decade.

The news is in the integration. Intelligent geographical content distribution and caching combined with elements of content management, personalization, security etc...we saw some of the influence in the earlier days of ECM, but I'm certain no single company ever delivered on it.

We'll certainly continue to watch this closely.

Chuck Hollis

Hi Richard

Sorry if I disappointed you. I said "runs well in a virtual machine". I didn't say "and available for download to anyone who wants to".

Umm, there are no ghostwriters here.

Unfortunately, I have to write each and every word that appears here -- warts and all.

Chuck Hollis

Hi Joseph -- perhaps you're right, perhaps we've seen elements of all of this before.

I suppose we could say the same of most new or popular technologies in the marketplace today.

And, if you ever share this thought in front of an IBM employee, you'll learn that they invented it all ... :-)

-- Chuck

Calvin Zito

Hey Chuck,

As you probably guessed, we had some thoughts from what we knew before hand and what was announced yesterday. Those are in two posts: http://www.communities.hp.com/online/blogs/datastorage/archive/2008/11/10/pie-in-the-sky-with-atmos.aspx and http://www.communities.hp.com/online/blogs/datastorage/archive/2008/11/11/further-quot-day-1-quot-thoughts-on-atmos-maui.aspx.

The thing I'd most like to hear you address is the proprietary appoach you are taking with Maui that will effectively mean "vendor lock-in". The second blog entry is the one that talks about that.

Also noticably absent from your efforts yesterday is a single customer that has been using the "GA release shipping since June". I can't believe that if you've been shipping GA code since June that you don't have a customer who can talk about Maui. I could point to a couple of customers who've told us that Maui isn't ready for enterprise customers but I'm guessing they've signed some legal documents that prevents them from saying anything in public.

Congrats on a nice marketing job yesterday,

Calvin

Chuck Hollis

Calvin

I can't believe you're trying to compare HP's traditional clustered NAS offering with something like Atmos. Goes to my earlier theme about servers guys and storage ...

Different use cases. Different technologies entirely. Now, if you'd like to compare and contrast with something like, say, EMC's Celerra, we could have a decent dialogue.

Appreciate the "closed, proprietary" labelling -- tells me that you folks really don't understand what this product is all about. That's great, in my book. Took you a while to figure out Centera, if I remember right.

Atmos exposes APIs that anyone can use. It's pretty open as far as its ability to incorporate external policy logic, encryption engines, codecs -- you name it. Pretty open, as far as such things go.

Certainly more open than, say, an EVA?

As to the rest of your nit-picking and "I've talked to customers" nonsense, have a good go at it, please. As far as responses from our competitors generally go, that's pretty weak stuff.

We really enjoyed the two posts you made. The first one (yours) missed the point entirely, which caused a considerable amount of amusement here at EMC. The second one (from your NAS guy) is a bit more thoughtful, so we give him a bit more cred. Still doesn't get it, though.

But, let's face it, until HP has some sort of offering for cloud-optimized storage (policy engine, etc.) you're not really in the game -- all you can really do is hurl rocks and petulant taunts.

Same goes for flash, FCoE ... -- well, we're back to our old dialogue, aren't we?

Cheers!

Mike Dutch

Hi Chuck, I tend to think that Cloud does solve problems in today's traditional data center. COS opens up some interesting ways to simplify BURA/DR and data migration (think "stretch VMotion"). I like how www.active-circle.com can include tape as a tier and though it's NAS/CIFS/FTP-based, I wouldn't be surprised if a little engineering couldn't add an object-interface. As you say, it's all about the use case you're trying to address.

Dave Tauzell

Is this similar to Google's GFS or Amazon's S3? It seems to have some of the pieces and then some more (the ability to move data to where it is being heavily accessed, for example).

Robert Ingram

Chuck, congratulations with the general availability of your product.

With great interest I have read this new post. However you haven’t convinced me on your third bullet in “what it’s not” just yet. Perhaps you can help us there a bit more.

Retention control of information wasn’t our driver. We selected Centera because of its substantial lower cost in managing large volumes of “active content” which makes the statement that Atmos’ use cases and reasons being totally different no longer hold. I am confident that we are not unique in why we own Centera.

EMC did introduce a new storage paradigm with Centera combining object addressing, RAIN architecture, self configuring and self-healing. Rightfully so earned a place as CAS beside NAS and SAN. I can’t say from the reading so far that Atmos does the same. Atmos seems to build on the success that the Centera product has with applying these techniques. This begs the question why EMC decided not to leverage the successful and proven Centera to add placement policies and network a bunch of Centera’s. One almost thinks that EMC it self does not understand it’s own Centera product that well to recognize the opportunities it has.

Chuck Hollis

Hi Robert -- you bring up some great points, so thanks.

First, I can see your point, which is that -- at scale -- many of the challenges that Centera addresses are the same ones that Atmos is intended to address.

I think we might be in a "glass half full or glass half empty?" situation regarding this topic.

On one hand, it might be persuasively argued that EMC should have simply extended Centera's functionality to include the advanced geographic distribution policies, or related capabilities.

On the other hand, it's also fair to say "different use case, different market" and justify Atmos based on the optimization in the product for the specific tasks at hand.

One important influencer in all of this is "will customers need both at the same time?", and the initial assumption was "highly unlikely".

So far, we haven't found a single customer (yet) that needed both.

Thanks for your thoughts ...

Paul Warren

Hi Chuck,

Great blog. Very inmformative.

I notice there is quite a bit of positioning, by Microsoft, of Azure against Atmos and Vmware. They seem to be playing down atmos and positioning azure as a platform as opposed to vmware as infrastructure only.

Will you be commenting on this? Love to hear you thoughts.

_Paul

Chuck Hollis

I don't see any overlap whatsoever between what Atmos does and what Amazon's EC3/S3 and Microsoft's Azure might be doing.

The latter two are primarily application environments. They target software developers and very small companies who need a somewhat scalable application delivery platform and usually can't consider traditional IT alternatives. Great, there's a need for that in the world today.

Atmos is quite different: it helps manage mega-content on a mega-scale. Its target is (typically) large organizations who want to be in the business of capturing large content from around the globe, and distributing it around the globe -- in a cost-effective, automated and secure manner.

Neither Amazon's offering nor Microsoft's offering does anything like this. Maybe some day, but certainly not anytime soon.

VMware is another story -- it takes existing applications and wraps them in a virtual container, making them easier to run better in the data center, or in the hands of a service provider (sometimes called cloud, but not in the most accurate sense of that word).

With Amazon's and Microsoft's offering, you're developing a new application using their tools. With VMware's offering, you can run existing apps as well.

There are more differences, but I think that's a key one.

Charlie McSweeny

This would be a good solution for internal delivery of the high density video training we are currently trying to squeeze through a 400kbps pipe to the COEs ... many of our engineers would rejoice to have this solution implemented immediately. What's the chance we could do that?

Chuck Wegrzyn

I know you are self-promoting Atmost as the end all, but check out twistedstorage.sourceforge.net - it was out in the open source community over 3+ years before you guys even thought about the idea. I ought to know since I met with the biz-dev people and Fred Olivera to discuss storage. They were trying to find out what I knew!

I started talking about "COS" long long before you guys even considered it. I talked to every major VC in the area about where storage was going to move, and this was late 2005/early 2006.

Today my little open source project is in V4 and deployed at sites around the world, and as of today have two very important institutions evaluating it for wide use in their organization!

Chuck Hollis

Congratulations on a successful open source project! I'm not entirely surprised that the ideas behind Atmos and COS have been rattling around for a while.

Then again, you'd probably appreciate that not every organization is comfortable using open source directly -- our friends at RedHat and Novell/Suse can attest to that.

Thanks for sharing the background!

-- Chuck

Sundar

Are large players like Akamai planning to use Atmos Cloud for edge content delivery and aggregation?

Andy Sparkes

Hi,

I've been trying to follow Atmos for a while now and haven't seen much in the press. I'm genuinely interested in the adoption of these cloud storage technologies. The issues that I run into with customers is "lock-in", thats not to be confused with a particular hardware platform but rather the lockin that occurs through capacity and bandwidth - as you store more and more it becomes more and more difficult to move to another cloud source -we all know about special offers to entice new customers but ultimately its the current customer base that subsidises that - with your cellphone you can just move, but with a few 100TB of data stored its actually pretty expense to move it out and then into another provider. Ultimately doesn't this hurt the economic arguement around cloud?

Chuck Hollis

Hi Sundar

Akamai's strength (as with all CDN approaches) is pushing content to millions of consumers around the globe. While Atmos *could* be used for that purpose, I think its strengths lie elsewhere.

For example, a CDN approach would be a poor answer to models where content is generated everywhere, and consumed everywhere. Likewise, not a good fit when you want a long-lasting repository of content, rather than today's hot viral video.

Again, both approaches have their strengths and weaknesses.

-- Chuck

Chuck Hollis

Hi Andy

You ask an insightful question, since -- undoubtedly -- it's very possible that capacity exceeds bandwidth in many situations.

I think it won't be as bad as you think for two reasons.

First, we're already seeing a few "federated clouds" where people are considering using an Atmos model that is comprised of several infrastructure providers, all under a single domain of control and security. This has the dual effect of (a) spreading the risk around, and (b) keeping everyone honest.

The second thing to consider is that 100TB can be moved over a network, it just takes a long time. So, if you're patient, the move shouldn't be that expensive.

Thanks for question -- it made me think a bit ...

-- Chuck

Andy Sparkes

Thanks for responding.

If you look at the historical trends of bandwidth verse capacity there is a 1.5/1 differential which leads to an exponential separation between the two. This means the problem is getting worse rather than better and that 100TB will grow to something that probably can't be moved over a network. The federated approach may help but you still have that control point, in your case Atmos and I think that most users hate control points in their data paths. It maybe that I'm currently working in a space where 100TB's a day being created is not unusual but not representative of most use cases but its probably a forerunner of whats to come. I am trying to overcome this issue of cloud as its seen as a requirement. I don't have a solution for this problem yet but it maybe that perhaps removable storage (Tape, RDX etc) with a logisitics company might be part of the answer but intuitively it feels like a backwards step and its making me seriously question the hosting of large data sets in the cloud.

Chuck Hollis

Hi Andy

Good points. I accept it as an article of faith that data repository size will continue to outstrip available bandwidth. That was true in 1985, and likely to be true in 2015.

So, the strategies are (as we both have pointed out):

- slow dribbles
- have the information live in multiple locations
- ship physical media around

The new bandwidth of several large arrays in a truck needs to be calculated! 100TB is less than half a rack of storage these days ...

-- Chuck

The comments to this entry are closed.

Chuck Hollis


  • Chuck Hollis
    Chief Strategist, VMware Storage and Availability Business Unit
    @chuckhollis

    Chuck works for VMware, and is deeply embroiled in all things software-defined storage these days.

    Previously, he was with EMC for 18 years, most of them great.

    He enjoys speaking to customer and industry audiences about a variety of technology topics, and -- of course -- enjoys blogging.

    Chuck lives in Holliston, MA with his wife, three kids and four dogs when he's not traveling. In his spare time, Chuck is working on his second career as an aging rock musician.

    Warning: do not buy him a drink when there is a piano nearby.
Enter your Email:
Preview | Powered by FeedBlitz

General Housekeeping

  • Frequency of Updates
    I try and write something new 1-2 times per week; less if I'm travelling, more if I'm in the office. Hopefully you'll find the frequency about right!
  • Comments and Feedback
    All courteous comments welcome. TypePad occasionally puts comments into the spam folder, but I'll fish them out. Thanks!