There's an important new class of information that's being captured more frequently -- and that's physical security information.
All those CCTV devices and other sensors you see everywhere are silently gathering more and more information about us and our movements.
And, despite the obvious benefit of providing more security for all of us, it's raising interesting questions at several levels.
This post was triggered by an interesting press release today, announcing further progress on EMC's physical security solutions.
If you're a regular reader of this blog, you know I like to dig down behind these announcements, and share the story behind the story; hopefully to expose an interesting thought or two.
And this one is no exception.
The Background
EMc first started working on physical security solutions in 2004, with our first announcements in 2005. We work with a robust ecosystem of partners to deliver end-to-end solutions in a variety of environments: public sector, retail, entertainment and so on.
Our initial thinking was at basically two levels: yes, of coursee there's going to be a lot of storage involved, but -- more importantly -- this was going to end up being important information that had to be managed, protected, and so on.
I think we were right.
As of now, we've got several hundred deployments. Not surprisingly, interest and adoption in information-based physical security solutions is steadily rising.
There's A Lot Of Information Involved
A good-sized deployment can generate a terabyte a day; and that's after compression, etc. Depending on the use case, there might be a good reason to keep this information around for quite a while, so the information footprint continues to grow.
Lest you have a mental picture that customers prefer the absolute cheapest stuff to store this information on, it's turning out to be quite the opposite. When you need this information for some reason, you really need it, and usually in a big hurry. It's the norm to think in terms of high availability, redundancy, backup -- even disaster recovery in some cases.
Occasionally, some of this information can be involved in a legal proceeding. That means chain-of-custody and other compliance concepts can apply. As anyone who's watched the TV series "Cops" knows, video evidence can be pretty compelling stuff.
It's Not Just Catching Bad Guys Anymore
There are some interesting uses for this information popping up. Retailers and others use these systems to quickly find lost children in crowded environments. Building designers can look at traffic flows and come up with layouts that are more comfortable and more engaging to the people who use them. Retailers can look at tconsumer patterns to figure out what people are interested in, and what they're not.
New kinds of killer apps are being designed that can recognize individuals or behaviors from camera to camera. Lots of interesting applications, but some of them make us feel a bit uneasy.
Cultural Differences Towards Phsyical Security Information
The EMC people who work with these customers have remarked on several occasions that there's a wide range of cultural norms and legal expectations associated with how this information can be used.
Some societies (most notably the UK) are mostly comfortable with cameras capturing their every move in public, opening up a wide range of potential uses. I, for one, feel a bit creeped out when I'm walking around over there.
Other countries have very strict limitations on how long this information can be kept, with timeframes as short as 24 hours.
Still other countries (most notably the US) are leaning towards auditing and controlling who can use this information, and for what purpose. It's a very fuzzy area in our society.
New Services
As part of the press release, EMC has announced new services. Some are around very practical IT concerns, like using ILM techniques to cost-effectively manage an information store that's growing by a terabyte a day.
Others hybridize with our eDiscovery offerings and help customers use video recordings as evidence in legal matters.
Still other services hybridize with our RSA technology to secure these recordings, and provide audit trails around how they were used, and by who.
The trends are pretty clear to me, though.
There's going to be a lot more of this kind of information in the future.
It's going to be increasingly valuable, costly -- and perhaps risky -- to gather and use this kind of information.
And all of us who are casting off an increasingly large digital shadow will have to figure out just how we feel about all of this, won't we?

Chuck,
Sounds good. Isn't this personal info crying out for a policy-driven store similar to one suitable for compliance with record access and retention/deletion policies attuned to each geography? That would be a VAR thing I guess.
Chris.
Posted by: Chris Mellor | March 27, 2008 at 12:19 PM
Hi Chris -- lots of different ways this could go, including the one you suggest.
Or maybe a cloud of some sort ;-)
Hey, BTW, thanks for the link to the new site. Very good!
Posted by: Chuck Hollis | March 27, 2008 at 12:30 PM
Speaking as a Brit, I don't think for one moment that Brits are comfortable with CCTV on just about every street corner. I just think that they have been used to solve a growing problem - or the perception is that they are solving a growing problem - of pety theft, traffic offences.
In Britain, the Data Protection Act is meant to be a protection against misuse of data, although the interpretation of the Act as well as its implementation leaves quite a bit to be desired.
We've seen the sorts of problems that one can run into with the use of this sort of data (Google in Canada displaying identified people on the streets, Google in USA identifying government sensitive sites, people suing for being identified to spouses at traffic light infractions, etc.)
In the USA, we're going to need some very strong and binding legislation to avoid abuse of this information. I think that people have an expectation of being filmed in certain environments (lobbies of government buildings, casinos) but there should be an expectation that in a free society that one is free to travel and not be tracked without cause.
Posted by: mgbrit | March 27, 2008 at 04:52 PM
amen!
Posted by: Chuck Hollis | March 27, 2008 at 05:03 PM
I think there's going to be VERY strong growth in this CCTV area, even in the USA, in the coming years... which of course drives storage demand (and bandwidth).
It seems that both Fortune 1000's and government agencies view CCTV as a way to greatly increase the reach and productivity of police departments.
In the area of the USA where I live (South Florida), there's increasing pressure on law enforcement agencies to provide more services despite decreased tax dollars and budget crunches.
Police departments and municipalities are really taking notice of just how powerful and inexpensive the video surveillance systems have become in recent years.
Yes, privacy advocates do make some valid Orwell-style points. However there's an awful lot of public sentiment and political emphasis on safety.
Posted by: Joshua Feinberg | March 30, 2008 at 08:33 PM
If this is the trend then I see a huge need for guidance on privacy rights. Who will provide this guidance? Today ILM guides companies on what types of data to store for how long and in what capacity, will it extend to an individual's privacy rights?
Posted by: Bhadra Gordon | April 06, 2008 at 11:43 AM