Chuck's Blog

The Shape Of The Storage Market: Q4 2011 IDC Results

I know many of you follow the storage market for various reasons: whether you're a vendor, a customer or perhaps an interested analyst.

The best independent data out there comes from IDC -- the de-facto gold standard in this category.  

Every quarter, they publish their scorecard on vendor and category shares.  Since getting a detailed analysis requires a subscription to their service, I occasionally pore through the spreadsheets, and try to find a few summarized nuggets that others might find interesting.

As before -- there are a handful of interesting takeaways from the latest batch of data.  And I promise I will make every effort to keep chest-thumping to an absolute minimum ...

The Inherent Difficulty Of Managing Data Growth

One of the most difficult questions I get asked by customers is painfully straightforward: how should we manage the explosive data growth going on in our organizations?

I wish I had a simple recipe.  Do these three ridiculously easy things, and you're set -- for example.  

No such luck.

You'd probably think that EMC would have some sort of semi-official position paper with deep insights on this thorny topic, but -- alas -- that's not the case either.  Truth be told, we tend to be somewhat divided amongst ourselves on the topic.

So, in an effort to be helpful, let me share with you how I'm answering the question when asked.

Lightning, Thunder ... And Reverberations

Well, a relatively interesting week here -- at least from my narrow and biased storage vendor perspective.

EMC launches Project Lightning (now VFcache) with immediate availability.  

We preview a near-term follow-on technology -- Project Thunder.  

The web then reverberates with reactions from extremely positive to somewhat negative to occasionally "meh".

It was a pretty interesting show if you watch this part of the tech industry closely.

Now it's Friday.  

What have we learned?

The Other Side Of Big Data

Yes, I admit it.  I am caught hook, line and sinker on the transformative potential of big data analytics to not only change business, but change the world we live in.  It grows more heady and more intoxicating with every passing week.

Red pill, blue pill ...

But there's another important aspect to the big data discussion that shouldn't be ignored -- and that's the plethora of big data models that aren't entirely based on advanced analytics, but are still strategically important big data applications in their own right.

Our cohorts at Isilon have done a masterful job at accelerating value for our customers in this space, which occasionally tends to get overshadowed by the more flashy topics.

So, in the interests of inclusiveness, I'd thought I'd share with you the other side of the big data story.

From Lightning To Thunder

We all know that thunder closely follows a big lightning strike, and with today's unveiling of Project Lightning (now VFCache), there's thunder as well, as in Project Thunder.

To get a real appreciation for what this is all about, I'd strongly encourage you to read this background post first, and then my overview of VFCache.

Now, with all of that out of the way, let's dive into this interesting technology preview.

VFCache Means Very Fast Cache Indeed

Big news today for IT infrastructure professionals. 

EMC has taken the wraps off of the much-talked-about Project Lightning, and there's much more there than most people expected :)

In my previous post, I did my best to set context as to what was going on, why this was important, and so on.  Unless you're a storage technologist, you really should start with that post.

In this one, we're going to get into the details: what it is, what it does and how it's different than what's come before.

And in my next post, I'm going to preview its sibling -- Project Thunder -- which, as you'll see, is a natural extension of what Project Lightning accomplishes.

The Next Generation Of Enterprise Flash Storage

Big news today for IT infrastructure architects -- and the businesses that depend on them.

Today's "Lightning Strikes" announcement from EMC represents a new milestone in giving our customers a new-found ability to deploy advanced infrastructure that's more efficient, more optimized -- and a helluva lot faster :)

It's a big deal.  There's a lot to cover, so I'm breaking this into three posts.

In this post, I'm going to set context -- what the problem is, what server-side flash brings to the party, what challenges exist with existing approaches, and so on.

In the next post, I'm going to dive into the details of VFCache (formerly known as Project Lightning): what it does, how it's different, what's there now, what's down the road, etc.

And, in my final post, I'm going to introduce you to Project Thunder -- which *hasn't* been as widely talked about -- but, as you'll see, is a natural extension of what Project Lightning (now VFCache) is targeting -- only at scale :)

If you're into IT infrastructure at any level, go get a cup of coffee, and let's dig in together.  Yes, there's a lot to digest, but you'll probably end up just as impressed as we all have been internally for a while now.

HDFS: Coming Soon To An Array Near You

As enterprises around the globe are beginning to exploit the value in their unstructured data, Hadoop is starting to find its way into more enterprise settings.

Enterprise IT buyers are justifiably a demanding lot, which creates plenty of opportunity for IT vendors such as EMC to create products that meet their newer needs.

Such is the case with the new native HDFS (Hadoop Distributed File System) capabilities from EMC's Isilon group.  

As you'll see, it makes exploiting the power of Hadoop and large unstructured data sets incredibly more efficient and powerful.

And before too long, I suspect other storage vendors will scramble to offer something very similar :)

My Continuing Infatuation With Storage

During my many decades as a technology guy, I've been exposed to all sorts of interesting things: compilers, AI, networks, security, processors, databases, virtualization, apps, mobile, etc. etc.  

But no technology has held my interest over the many years as storage.  I do have my flings with other disciplines from time to time, but I keep coming back.  

Good thing I work for EMC, eh?

I occasionally get asked about this fascination with the topic: polite inquiries as to what is it about storage that endlessly fascinates me -- almost as if they were inquiring into a dark association with Dungeons and Dragons, or maybe I'm into retro computing?

So, rather than have to answer the question repeatedly, I thought I'd share my personal reasons as to why storage continues to enchant me as a technology, and show no sign of slowing down.

Maybe it will end up seducing you, too -- if it hasn't already :)

EMC Atmos And The Challenge Of Doing Things Differently

There are two ways to bring a new technology to market.

One way is to give people a better product to do exactly what they're doing today: no process change required.

Another way is to give people a better way to do things: substantial process change required.

Meaningful breakthroughs in technology almost always involve process change.  The downside is that any substantive process change takes significant time and effort.  

There's no shortcut that I'm aware of.

In our world, EMC Atmos is an excellent example of innovative technology that demands both a mindset and process change.  The lessons we've learned about how to go about introducing new technology (and new process!) to large audiences is instructive -- not only to other technology companies like us, but to any IT organization looking to introduce new ways of doing things into their ranks.

As technologists, we obviously have to learn to get good at getting people to actually embrace what we come up with.  Otherwise, the stuff just sits on the shelf :)