Chuck's Blog

The Other Side Of Big Data

Yes, I admit it.  I am caught hook, line and sinker on the transformative potential of big data analytics to not only change business, but change the world we live in.  It grows more heady and more intoxicating with every passing week.

Red pill, blue pill ...

But there's another important aspect to the big data discussion that shouldn't be ignored -- and that's the plethora of big data models that aren't entirely based on advanced analytics, but are still strategically important big data applications in their own right.

Our cohorts at Isilon have done a masterful job at accelerating value for our customers in this space, which occasionally tends to get overshadowed by the more flashy topics.

So, in the interests of inclusiveness, I'd thought I'd share with you the other side of the big data story.

From Lightning To Thunder

We all know that thunder closely follows a big lightning strike, and with today's unveiling of Project Lightning (now VFCache), there's thunder as well, as in Project Thunder.

To get a real appreciation for what this is all about, I'd strongly encourage you to read this background post first, and then my overview of VFCache.

Now, with all of that out of the way, let's dive into this interesting technology preview.

VFCache Means Very Fast Cache Indeed

Big news today for IT infrastructure professionals. 

EMC has taken the wraps off of the much-talked-about Project Lightning, and there's much more there than most people expected :)

In my previous post, I did my best to set context as to what was going on, why this was important, and so on.  Unless you're a storage technologist, you really should start with that post.

In this one, we're going to get into the details: what it is, what it does and how it's different than what's come before.

And in my next post, I'm going to preview its sibling -- Project Thunder -- which, as you'll see, is a natural extension of what Project Lightning accomplishes.

The Next Generation Of Enterprise Flash Storage

Big news today for IT infrastructure architects -- and the businesses that depend on them.

Today's "Lightning Strikes" announcement from EMC represents a new milestone in giving our customers a new-found ability to deploy advanced infrastructure that's more efficient, more optimized -- and a helluva lot faster :)

It's a big deal.  There's a lot to cover, so I'm breaking this into three posts.

In this post, I'm going to set context -- what the problem is, what server-side flash brings to the party, what challenges exist with existing approaches, and so on.

In the next post, I'm going to dive into the details of VFCache (formerly known as Project Lightning): what it does, how it's different, what's there now, what's down the road, etc.

And, in my final post, I'm going to introduce you to Project Thunder -- which *hasn't* been as widely talked about -- but, as you'll see, is a natural extension of what Project Lightning (now VFCache) is targeting -- only at scale :)

If you're into IT infrastructure at any level, go get a cup of coffee, and let's dig in together.  Yes, there's a lot to digest, but you'll probably end up just as impressed as we all have been internally for a while now.

HDFS: Coming Soon To An Array Near You

As enterprises around the globe are beginning to exploit the value in their unstructured data, Hadoop is starting to find its way into more enterprise settings.

Enterprise IT buyers are justifiably a demanding lot, which creates plenty of opportunity for IT vendors such as EMC to create products that meet their newer needs.

Such is the case with the new native HDFS (Hadoop Distributed File System) capabilities from EMC's Isilon group.  

As you'll see, it makes exploiting the power of Hadoop and large unstructured data sets incredibly more efficient and powerful.

And before too long, I suspect other storage vendors will scramble to offer something very similar :)

My Continuing Infatuation With Storage

During my many decades as a technology guy, I've been exposed to all sorts of interesting things: compilers, AI, networks, security, processors, databases, virtualization, apps, mobile, etc. etc.  

But no technology has held my interest over the many years as storage.  I do have my flings with other disciplines from time to time, but I keep coming back.  

Good thing I work for EMC, eh?

I occasionally get asked about this fascination with the topic: polite inquiries as to what is it about storage that endlessly fascinates me -- almost as if they were inquiring into a dark association with Dungeons and Dragons, or maybe I'm into retro computing?

So, rather than have to answer the question repeatedly, I thought I'd share my personal reasons as to why storage continues to enchant me as a technology, and show no sign of slowing down.

Maybe it will end up seducing you, too -- if it hasn't already :)

EMC Atmos And The Challenge Of Doing Things Differently

There are two ways to bring a new technology to market.

One way is to give people a better product to do exactly what they're doing today: no process change required.

Another way is to give people a better way to do things: substantial process change required.

Meaningful breakthroughs in technology almost always involve process change.  The downside is that any substantive process change takes significant time and effort.  

There's no shortcut that I'm aware of.

In our world, EMC Atmos is an excellent example of innovative technology that demands both a mindset and process change.  The lessons we've learned about how to go about introducing new technology (and new process!) to large audiences is instructive -- not only to other technology companies like us, but to any IT organization looking to introduce new ways of doing things into their ranks.

As technologists, we obviously have to learn to get good at getting people to actually embrace what we come up with.  Otherwise, the stuff just sits on the shelf :)

The Squabble Over Single File Systems

Many of you are endlessly entertained by the back-and-forth bickering between us storage vendors over things like benchmarks. 

Sometimes the disagreement is over how the test was conducted, or the use of "lab queen" configurations that would never be found in a customer environment.

And, occasionally, there's very strong disagreement around comparing two very unlike things using a common standard.   That's what this post is about.

Why should you care? 

In a world of exploding data growth, massive scale and limited resources, how you do things may end up being more important than what you do. 

I believe many IT architects will want to take note of this particular debate, because you'll be seeing ever-more variations of this same theme in the near future.

Catching Up With VNX

At the beginning of this year, EMC announced over 40 new storage products and enhancements.  Among these, a great deal of attention was focused on the new VNX and its smaller sibling, the VNXe. 

The VNX brought together the traditional CLARiiON and Celerra storage arrays into a single, unified platform: more powerful and far easier to use than its predecessors.  The VNXe was a strategic foray into what we saw as an under-served market: storage for the IT generalist: the VNXe takes simplicity and cost-effectiveness to new levels in this important market segment. 

That was then, what about now? 

On the tail of EMC’s positive Q3 earnings results, I wanted to dig a bit deeper into what was going on in this important storage segment: how were the existing products doing, and what was new in this space? 

And there ended being a lot to talk about …. 

The Shape Of The New Storage Market

Part of my role here at EMC is to spend time with what we call the "industry influencers" -- the press, analysts, and the consultants who ostensibly shape public perceptions.

When it comes to EMC's storage business, they collectively seem to be fascinated by one minor market development or another -- mostly it's about a specific technology and specific vendors.  Only a few seem to have an appreciation for the big, secular trends that are driving the transformation of storage technology -- and the vendors that play.

Although EMC doesn't have an official talk track on "the changing shape of the storage industry", I do have a personal one.

And, if you're interested, I'd like to share the back story of how I put all of the day-to-day discussion into a broader framework.