Say Hello To EMC SourceOne

I love major product introductions at EMC. 

Fortunately, we get to do this on a relatively frequent basis :-)

Sure, it's great to go through all the new features and functions, but what I like to do is attempt to reflect on some of the thinking behind any new product -- what trends does it try and anticipate?

Such is the case with EMC's new SourceOne platform, publicly announced today.

And you can look at the product one of two ways.

Is it just an email archiving platform with extremely advanced features? 

Or is it a new generation of ILM software that just happened to target emails first?

Email Archiving?  Really?

Sure, email archiving is a growing marketplace these days -- we're all generating tons of the stuff, and in more and more organizations there's a growing interest in managing it effectively.

I'd argue that we're in the second phase of the email archiving marketplace -- the first-gen solutions had their place, but were quickly overwhelmed in terms of functionality and scalability.  So, yes, the world probably could use an email archiving environment designed to anticipate the next ten years, rather than react to the last ten years.

However, the motivations these days are far more complex than they were a few years ago.

The IT folks are looking at storage costs associated with email (not to mention servers!) and are highly motivated to prune through the avalanche and delete what's no longer needed -- or at least move it to storage that's more cost-effective (e.g. deduped and/or spun down).

Scale matters -- a modern organization processes a real-time river of incoming and outgoing messages these days.  If the email archiving system can't keep up, bad things happen.  Trust me on this.

The policies associated with targeting user emails for archiving have gotten amazingly complex -- no longer is it a simple "emails older than 30 days are archived" world -- many organizations have evolved very sophisticated policies based on user roles, email type, etc. -- and these are changing all the time.

Users have learned to value their email archives as well -- they don't want to see valuable messages disappear into some corporate repository, never to be seen again.  They want flexible, searchable access to all their old emails -- even while they're not connected to the corporate network.

To make matters more interesting, it's hard to have any serious email archiving discussion these days without having an eDiscovery discussion as well.  Sure, the legal department cares about compliant handling of email -- and being able to prove "chain of custody" in certain circumstances -- but the real concern is responding to lawsuits and discovery actions quickly and effectively.

If you're a corporate lawyer, you don't want a "surprise" email surfacing during proceedings.  You'd like to be able to find what you need, whittle things down to a manageable working set, and integrate your findings with the overall workflow for the case.

So, where do the interests of the IT people looking to be more efficient end, and the needs of the legal department to search and process archival emails begin?

Well, these needs aren't separate -- and that's the point.

ILM in Action?

Several years ago, EMC introduced ILM concepts -- information lifecycle management -- into the industry discussion with great success. 

We made a simple point -- that certain kinds of information had to managed from creation to deletion.

And -- during its lifecycle -- there would be an interesting tradeoff between controlling costs, maximizing value and reducing risks that wasn't going to be a straightforward discussion.

As an example, emails mean different things to different people -- and they all have different priorities.  Indeed, I hear that determining email policies can be one of the more contentious discussions in IT land -- an example of information governance at work.

Maybe we were a bit early with the concept at the time, but one can take a look at SourceOne and conclude that -- yes -- this product is an enterprise ILM platform for email.

It helps users get the most value out of their email archives.

It helps IT control costs for storage and archiving.

And it helps the legal people get their job done -- by not only ensuring compliant handling of selected emails, but supporting the entire eDiscovery process -- from search to case management. 

A nice end-to-end value chain for important information, no?

But ILM Can Be A Tough Sell

So, you've got an advanced software platform with something for everyone.  Who do you sell it to?

Users look at the powerful features for managing and leveraging their personal email archives, and they're excited.  I know I am -- we've been using the product internally for a while.

But who listens to users these days?  Frankly speaking, they don't have a lot of say in things because they don't act collectively.  When it comes to things like this, we're all sheep ...

You could target the legal department, and point to all the great things that the package does to reduce their costs and efforts, identify and control certain risks, and generally improve the outcomes of legal proceedings.

But -- face it -- how many IT meetings have you been in that had representatives from the legal department in attendance?  Maybe they're involved in IT decisions, maybe not.

So the core of the value proposition has to be aimed at the IT audience, and -- for them -- it gets down to very practical things like being able to slash storage costs, or being able to easily handle ever-increasing volumes, or making it easy to set up and monitor the results of new email archiving policies, or providing a single dashboard that provides end-to-end status of the environment.

Sure, all of that is there in SourceOne -- in abundance.  But I think it's fair to say that many of its features won't get full value simply because they appeal to people outside of IT.

From Email To Content?

If you have any time to spend drilling into the SourceOne architecture, you'll notice something rather interesting about its design approach: it isn't architecturally limited to emails.

Squint your eyes at the roadmap, and you could easily see it doing the same ILM things for user files, for example.  Or content repositories.  Or those dumping grounds for reports and analytics at the back end of ERP or a DW/BI environment.

Or just about anything that doesn't live in structured database, for that matter.

And, for all the content types, the same general goodness could apply:

Helping users get the most value out of their content.

Helping IT control costs for storage and archiving.

And helping the legal people get their job done.

Because, in one sense, there's nothing really that magically special about emails -- they're just another form of content, when you think about it.

Platforms Vs. Products

And that's why I keep thinking of SourceOne not as an email archiving product, but as the first release of an enterprise ILM platform.

I wonder how many IT organizations will spot this -- and be interested in investing in infrastructure that helps them manage information like money?  All kinds -- and not just email?

Because that's really informationist thinking ...

Comments

[Dave]

Hmmm...struggling with this one Chuck. Maybe I'm missing it but seems like there's a long way to go here.

Would like to see more meat around auto-categorization and automated policy management and more discussion about getting rid of stuff. And would like to see some examples at scale (I know references this early are difficult). But these are the challenges clients face with scaling.

I like the vision but don't see the pieces yet. If I'm wrong I'll be happy to admit it so I'll keep an open mind:
http://wikibon.org/blog/hello-source-one-good-bye-emailxtender/

Dave from Wikibon

[John F.]

"I wonder how many IT organizations will spot this -- and be interested in investing in infrastructure that helps them manage information like money? All kinds -- and not just email?"

Hmmm.

I guess those would be the ones that invested in Symantec and Mimosa.

Don't you thing you're just a few years late and a couple major features short in this game?

John

[soikki]

Is this the same souce-one as was announced last autumn? Or is something now different?

[Chuck Hollis]

Very different -- thanks!

[Chuck Hollis]

Hi John

I would hope that you had better things to do with your time at NetApp than peppering my blog posts with cheap shots.

-- Chuck

[Chuck Hollis]

Hi, someone on some other blog said they had a problem leaving a comment here, and may be implying that I'm censoring it.

Sorry, whoever you might be, I have no record of you ever leaving a comment. Unless you remember to enter the "captcha" characters, it assumes you're a spambot and deletes your comment.

When I saw your post (thanks to Google Alerts!) I tried to leave a comment on your post, but I was denied access as I did not have an account on your site, and you had no mechanism for anyone to register for an account.

You might want to fix this.

I thought about sending you an email or twitter, but there was no name or email visible anywhere that I could see.

You might want to fix this as well.

In the spirit of fairness, here's a rather lengthy reply from our thoughtful but mysterious commenter.

http://www.datamobilitygroup.com/saltworks/archives/22

-- Chuck

[joseph martins]

Good afternoon Chuck,

Captcha was presented to me and your blog responded as if my comment had been successfully submitted. I'm not sure why it failed, but I do apologize for the misunderstanding, and I appreciate the heads up. Note that we do not permit comments on the DMG blog for reasons explained in the section titled Setting Expectations.

Here is the body of the original comment which has been removed from our blog since it need not exist on both sites:

I'd like to offer a few comments about your post if I may.

The concept of information management (IM) pre-dates EMC. It was rebadged as ILM by the storage vendor community several years ago, but cradle to grave information management has been around for a very long time. The "Lifecycle" in ILM was always implied...the rest of us simply referred to it as information management. Storage vendors felt the need to squeeze an L in there.

Having written that, I will share with you what I believe to be the storage industry's single greatest early contribution to information management - a hard dollar ROI. The IM community had been floundering around for years in search of a meaningful way to quantify ROI. Frankly, most practitioners in the IM community are not familiar with the ins and outs of storage technology and management, and increased knowledge worker productivity is a tough sell. Hard dollar storage-related costs and cost-saving concepts such as HSM weren't even on our radar.

Regarding the tough sell, I'll forward a relevant presentation to you. You can also find it on my LinkedIn page.

In the section of your post titled "From Email to Content" you demonstrate an understanding of what IM folks have known all along - email is just another type of content. With that understanding you can now see that ERPs, CRMs, SRMs, SCMs, WCMs, ECMs, RMs etc, are all specialized forms of information management. At their core they accomplish the same IM goals. Hypothetically speaking one could develop a core repository and common IM features upon which vertical solutions such as resource management, code management, sales management etc could be layered. Some companies have been working on that problem for years, but they have all lacked one critical piece of the puzzle - control of the real estate. Ultimately, he who controls storage controls the information assets.

Most of today's IM applications are sold to customers as solutions that "manage the lifecycle of information from cradle to grave". That claim is inaccurate and misleading. Storage applications play an important role in cradle-to-grave IM too. But, a lack of communication between storage and information management applications means that storage applications can replicate, migrate, delete and manipulate data without authority and without warning, completely outside the purview of most information management applications. Imagine the following scenario:

A file is deleted from an IM application. Business users think the file is gone, deleted, purged from the system, but it's living in a dozen backups or an archive just waiting to be restored, discarded or discovered. The IM application does not communciate with nor control the storage environment. And storage applications do not check with IM apps to confirm that their actions [against specific files or data] are permissible. In fact, both environments typically rely upon their own separately managed policies and policy engines. It's an alarming disconnect between IM and IT that exists in nearly every application out there today, from corporate governance apps and ECMs, to ERPs, SRMs and SCMs. Come discovery time, customers will always be caught with their pants down.

Ultimately, I am hoping that efforts such as SourceOne will eventually eliminate the disconnect between backups, archives, production repositories and the many applications that use them. I'd like to see storage vendors come together and take ownership of the repositories - a common set of data models and APIs would be wonderful though I won't hold my breath (CMIS is a start). And, frankly, IM vendors should focus on building vertical solutions on top of the repositories.

That, to me, has always been the future of IM: common repositories managed in the storage fabric accessed and shared by purpose-built vertical solutions. It was what I began preaching in 2002 to anyone who would listen, and I haven't stopped. We have a long long way to go.