When Big Data Met Security: Is The New Era Beginning?

Two powerful IT mega-trends appear to be combining in an interesting and powerful way.

One highly visible trend is the rise of APTs: advanced persistent threats.  These newer attacks have routinely defeated traditional approaches to information security, and show no sign of abatement.

The other power trend is the amazing power of analytics-at-scale in the hands of data scientists.  I continue to be fascinated by the dazzling potential here.

Could the stunning power of big data be part of the answer to the new class of advanced threats?

The Background

If you're not at least somewhat familiar with APTs -- and the threat they represent -- you owe yourself 30 minutes on the topic.  This is not the usual scare-em tech marketing stuff; it's real and it's serious.  APTs are but just one highly-visible example of what's more broadly referred to as "advanced threats" in the security domain.

The core of the most effective response appears to be a new breed of security analytics that help quickly detect anomolous patterns -- basically power tools in the hands of a new and important sub-category of data scientists: the security analytics expert.

In Search Of An Analogy

There's some colorful language being used to describe the new role, borrowed heavily from combat fighter pilots.  You'll hear talk of "situational awareness" and "lose sight, lose fight".

For the security analytics expert, this means as much fresh and relevant data from as many sources as possible -- internal and external -- looking for immediate signs of the unusual, followed by a speedy deep-dive into the relevant context.

It's interesting to point out that when you chat with a proficient data scientist, that's exactly how they describe what they do :)

Thinking Big

The fighter pilot analogy isn't that far off -- all signs point to this being an escalating arms race between the good guys and the bad guys.  If you're serious about combating the threat, be prepared to not only take out your checkbook, but do some heavy organizational lifting as well.

For starters, you'll need a core of fighter pilots -- a small cadre of very intelligent security people who are continually honing their combined skill set through the use of security analytics vs. traditional measures.  

One or two probably won't be enough for a meaningful response.

Next, you'll have to give them the organizational muscle to cut across the multiple technology silos so frequently found in larger IT settings: the network team, the desktop team, the database and application team, the infrastructure team, and so on.  

Relevant log and traffic data is everywhere, and responses tend to be highly coordinated.  

I am reminded of the historical organizational conflicts between traditional ground-based armed forces and the newer aviators from the early 1900s.

And finally, you'll need to arm them with meaningful technology.  Lots of storage for all those log files.  Powerful computational resources for crunching data in real-time.  And specialized software tools for the task at hand.  

In a nutshull, it's a familiar big data environment.

The EMC Angle?

We see helping our customers respond to this new threat as very important.  We have direct and relevant experience as to just how disruptive an APT attack can be, and we've learned a lot in the process.  Helping to combat this threat is a big deal for us.

If you look across our portfolio, you can see many of the necessary components: big data storage farms for all sorts of log files (remember, scale and performance matters here).  The ability to stand up cloud-like compute farms for the variable and inherently bursty workloads involved.  A healthy and growing ecosystem of consultants and partners who can help.  

And an investment in assets that represents this new style of security thinking: NetWitness and Network Intelligence's enVision.

Today's RSA product announcement is a good starting point on the sorts of capabilities we'll undoubtedly need going forward.  The new NetWitness Panorama module integrates multiple internal and external data sources, and helps create that "situational awareness" we think will be needed going forward -- the power tools that a new breed of security analysts will undoubtedly need.

The Inevitable Two Paths

Unfortunately, there's no 100% answer to the new class of advanced threats.  Although vendors like EMC and RSA can help, the real challenge will be in driving the organizational investment pattern to detect and react to the new class of attacks.

For me, there's an interesting historical (and personal) IT precedent here.  You might not know this, but EMC's SRDF was developed at the request of our customers whose data centers were impacted during the first World Tower attack in 1993.  They didn't have a good way to get their data synchronized to a second site.  

We thought we could solve that problem, and the rest is history.

More importantly, as we've talked with people over the years about disaster recovery and business continuity (another important risk-mitigation concern), we've learned that there are two classes of IT organizations out there: those that take disaster recover and business continuity very seriously and have decided to be very good at it, and those that have decided to take a less demanding approach.  

No amount of external exhortation or argumentation will change the second group's approach to the topic.  It's something the business decides to do or not do -- period.

I am quite sure we will see the same sort of bifurcated response to the new class of advanced threats.  

There will be those that decide to invest in being quite good at detecting and responding.  

And there will inevitably be those that take a decidedly less demanding approach to the topic.

Which one do you think your IT organization will end up being?

Comments

[@mjbrender]

There are a few fascinating points you focus in on:

1) The language.
We all comprehend complexity through analogy. Technology is filled with them: code bugs, computer viruses and trojan horse spyware (which is a two-for-one). I can't think of a more appropriate comparison for APTs than air combat.

The complexity of cyber threats these days is beyond the mono-y-mono of "fishing" attacks; we are now surrounded, flying high above the ground with no time to stop and look around - we have to keep going forward. If that scares the hell out of you, then you're thinking about it the right way.

2) The focus.
Companies are making headlines in some tough ways these days. As you mentioned, we felt that first hand. It is out of this adversity that strong products like NetWitness Panorama and others within the RSA portfolio developed. They no longer be an afterthought to Data Center architecture. Security must be first and foremost for all valuable data.

Thanks as always for your insight Chuck.