If you're a rock musician, you know you've "made it" when you get inducted to the Rock and Roll Hall of Fame.
If you're a societal mega-trend, you know you've hit the big time when you've become the main show at the World Economic Forum being held in Davos.
I suppose this is just another indication that big data analytics -- and its breathtaking transformative potential -- has officially hit the major leagues.
Big data is now officially "big".
The profound societal implications of big data analytics has now overflowed the neat category of the tech world, and is quickly spilling into pointed discussions around such weighty topics as global health care, economic development, education, governance, agriculture, privacy, national security and more.
One useful observation is that information is becoming the "new currency". We're seeing the early days of how aggregated information is becoming the new source of innovation and improvement, power and wealth.
And what better place to discuss this new "information economy" than at the World Economic Forum?
The Quick Overview
The New York Times does a great job providing a synopsis for the casual reader.
If you dig a little deeper, you'll come across this useful backgrounder "Big Data, Big Impact: New Possibilities for International Development" which exposes policy makers and policy influencers to both the amazing potential as well as the potential challenges and risks. This document joins the "McKinsey Manifesto" as a call to action in the private sector.
After reading the latest, I had to sit back and sort through a few things that were bothering me. Fortunately, airplanes can be great places to think quietly. A few minor light bulbs went off as a result, which I'd like to share.
Before long, I think more of us will have to do the same.
The Role Of The Individual In A Big Data World
In our current financial economy, we are all (mostly) producers and consumers. We work, we get paid, we turn around and spend money on other things. The magic elixir in modern economics is the notion of productivity: economies, organizations and individuals tend to do well when the value of their output greatly exceeds the costs of all the inputs.
One of the reasons that I've always been drawn to IT is that it is an evergreen productivity-enhancer in a classical microeconomic sense. When applied well, it both reduces input costs and maximizes output value. We, as individuals and organizations, appear to be getting continually and collectively more productive thanks to information technology.
And the dawning era of big data analytics will certainly be a new chapter in this future history we're all now creating.
But, if you think about it, we're largely in control of our economic participation. We can choose where to work, what kind of work we prefer, and so on. We also have great choice as to how the results get allocated -- unless we're married, that is :)
Other than work rules and tax codes, government doesn't usually play all that heavy a hand in how we go about participating in the global economy -- or shouldn't.
Now, let's reframe that familiar discussion in the new information economy.
Once again, we are all producers and consumers of digital value. Our thoughts and actions result in a data exhaust that's the raw fueld for an infinitely-useful and ever-growing number of predictive analytical models.
Who "owns" this information isn't entirely clear: the producer or the aggregator?
But this discussion isn't about mere commerce and business models -- it's far, far broader. It's easy to see there's clear evidence that our "data participation" in many of these same analytical models can directly benefit us -- and others -- as well.
Leaving the world of commerce, there are no shortage of powerful societal examples if you go looking. Health care is being transformed, thanks to big data. Education. Environmental Science. Crime and policing. Energy production. Urban planning. Physics and cosmology. Bioscience.
And that's just for starters.
Just about every societal issue I personally care about will be very likely unrecognizably transformed for the better during the next decade as a result. If I'm correctly, that would result in a historically unprecedented rate of positive societal change.
As a consumer and an official card-carrying member of the human race, there's no way I could be anything other than absolutely delighted by the potential that lies before us.
Although, as a producer of big data, I'm a bit more timid. For starters, seeing what people can do with my personal data can often creep me out. How do I know that my "data exhaust" won't be used for something I don't like or want?
Back to the financial world, I pay a lot in taxes, as many of you do. Like you, I frequently despair at the boneheaded use of my tax money by people who appear to be barely qualified to dress themselves in the morning, let alone make sophisticated policy decisions.
When I consider information in the same light, it's my data, shouldn't I have some say in how it's going to be used? Well, we sort of lost some of that debate when it comes to government and money.
I am not optimistic about this, though. For example, the SOPA hearings in Congress were very, very discouraging to follow -- clearly revealing that many policy makers don't have even a dim understanding of how information technology is quickly transforming our society for the better.
An Interesting Discussion With My Doctor
I went to the doctor for a routine problem, and he made some recommendations as to what I should do. Since I tend to be curious -- especially when it comes to my health -- I started to ask questions as to why he was making that specific recommendation.
To my surprise, he started to talk about some of the new predictive models that he was starting to use. He had to sheepishly admit that science didn't 100% understand why they worked the way they did, only that they tended to make demonstrably better recommendations than before.
I quickly realized I was faced with an interesting choice.
I could either accept a treatment where the underlying mechanisms were relatively well understood, or I could opt for a different course of treatment based on the predictive power of a new analytical model whose underpinnings weren't exactly clear.
Trust the black box, or not.
Of course, I went with the doctor's advice. But it was a rather unsettling experience.
I suppose there will be more of that in the future.
The Role Of Government In A World Of Big Data
Getting back to Davos and the World Economic Forum, that's what this discussion is really about.
There's a brand new era of economic and societal potential dawning. Like all revolutions, it doesn't come without some interesting and uncomfortable policy discussions that essentially boil down to balancing the needs of society against the rights of the individual.
The government has a clear role in providing policies that accelerate and maximize the benefits. But at the same time, most would agree that our individual rights and needs shouldn't be trampled in the process.
As with most things, where you stand depends on where you sit. At one extreme, the government of Kenya is investing in making as many data sets accessible as possible. And at the other, the EU is currently discussing more stringent privacy protections. Both approaches have their merits -- in context.
Mash up any small number of "sanitized" data sets, and it's often trivially easy to trace your personal data back to you. Policy makers who claim that removing or masking PII (personally identifiable information) will somehow protect your privacy are either ignorant or lying
Shocking, I know :)
We All Have Choices To Make -- Or Do We?
Where you personally come down on these issues will inevitably vary. All I can say is that my own views have shifted dramatically in favor of policies liberalizing and promoting increased data use.
Yes, I'm very uncomfortable with some of the implications.
But the potential is so overwhelming I sort of feel it's my societal duty to share my useful data so that others can potentially benefit. For example, I pay substantial property taxes that are primarily used to fund schools that my kids don't attend. It's a bit painful, but I do get the "greater good" benefits.
I wonder how many others will feel the need to be "charitable" with their personal data, much as they do with their other resources?
For example, would you donate your genome and entire life history to the public domain if it could benefit others?
Does the government have a right to my data -- even if I protest -- if it will greatly benefit others? The government insists on taxes for the greater good. During times of war, governments will force many of their citizens into military duty -- often against their wishes.
Think about the concept of "eminent domain" for a moment. Your head may start to hurt as a result.
In aggregate, the needs of society are very great indeed. Big data has already shown enormous promise as a once-in-a-generation breakthrough that can redefine our society. Governments around the globe will begin to struggle with balancing the data needs of society against the needs of individuals. It won't be perfect, and it won't be pretty.
I believe we can't afford to ignore -- or even slow -- big data's amazing potential. Others -- including those in positions of power -- will inevitably agree. It's not a question of "if", more a question of "when".
As a result, individually and collectively, I'm predicting we're all going to have to get a lot more comfortable with the unsettling notion of more freely sharing our personal information.
Food for thought.