Our poor, struggling brains.
Evolved for survival on the serengeti, we're now attempting to thrive in a digital world that we weren't programmed for.
Sure, we can adapt -- over time. Through thoughtful reinforcement, we can learn to overcome many aspects of our evolutionary programming.
But being aware of the problem is half the challenge.
I've noticed that there's more and more conclusive evidence that we -- as a species -- weren't wired to work effectively with data. It's something we learn to do, often despite ourselves.
And in the coming era of big data analytics, we're going to have to collectively get better at it.
Much, much better indeed.
Have You Been Sucked Into The Big Data Analytics Vortex?
My personal realization is that we're living at a rather special time in history.
For the first time in our society, data about everything is relatively plentiful, with more coming every minute. The technology resources needed to move, amass and process said data is relatively plentiful. The skills, sophisticated math and machine learning algorithms are also becoming more widely accessible.
And there is no shortage of evidence that -- when combined -- the result is amazing predictive insight about the world around us. It doesn't really matter all that much what topic you're interested in: business, research, social good, technology, crime, politics, science, etc.
Gather enormous amounts of uncorrelated data, invest the time to correlate it and extract insights, and mind-bending things are now routinely happening. I'm not amazed by the stories any more.
At one level, it's right up there with the discovery of fire.
I'm hooked. Many of my co-workers are hooked. Quentin Hardy at the New York Times appears to be hooked. McKinsey is hooked. The Davos crowd is hooked. To many of us, it looks like humankind is poised to take this great, historical leap forward as a society.
But there's just one problem. It's us, or -- more specifically -- our brain's ability to reason and act intelligently in the face of analytical data.
It's often true at an individual level. It's largely true at a company and organizational level. It's most definitely true at a political and societal level. We don't handle data well, and we make poor decisions -- or fail to act appropriately -- as a result.
Examples Abound
I came across this excellent and thoughtful article by James Guszcza and John Lucker in the Deloitte Review which is definitely worth reading in its entirety.
They had me at the opening quote:
“The most difficult subjects can be explained to the most slow-witted man if he has not formed any idea of them already; but the simplest thing cannot be made clear to the most intelligent man if he is firmly persuaded that he knows already, without a shadow of doubt, what is laid before him.” — Leo Tolstoy
My spin? Our intelligence and perspectives can become intellectual prisons from which escape can be difficult, even in the face of compelling data.
The article also recounts one of the many psychology experiments where subjects make poor decisions, especially in the face of strong evidence to the contrary. We collectively tend to reject data and conclusions that don't fit with our world model and personal experiences.
Why This Is A Real-World Problem
As part of my role, I get exposed to many organizations that are bringing big data analytics into their organizations, including here at EMC. For the first time, amazing predictive models are bubbling up, with mind-bending implications that fly in the face of current practices and current organizational wisdom.
The result can be spectacular conflicts between perceptions of how the world works, and how it can be proven to really work. And sparks can fly.
I joke that data science denial goes through three phases.
1 -- Your data sucks.
2 -- Your model sucks.
3 -- You suck.
Funny? Yes. Until you see it unfold exactly that way a dozen different times.
New Leaders Required?
All of the sudden, the role of the strong business leader becomes oh-so-critically imporant. This individual has to steer the organization towards the new model, the new process and leveraging the new insights -- and do so without blowing things up along the way.
Most insightfully, the Deloiite team offers practical advice on this key aspect of organizational change management in a data driven world.
Are We Making Things Worse For Ourselves?
Way back when, I took a lot of coursework in statistics and probability. I found the subject absolutely fascinating in college, but couldn't -- at the time -- figure out how one might go about making a living at it. Besides, useful data was hard to get to, computing resources were expensive, the tools were a pain to use.
I think I was 30 years too early. If it was today, I'd probably end up chasing a data science career.
As part of the recent Data Science Summit at EMC World 2012, there was a most excellent rant on this subject from Michael Chui, Senior Fellow at the McKinsey Global Institute. Check this out, and see if you agree:
“We’ve got to stop teaching so much calculus,” he said. “I think we should teach more stats. Who does an integral anymore in business? There are a few people in engineering. But who needs to understand conditional probability, and who needs to understand selection bias, and all those things that a data scientist just wakes up with and understands. One of the things we need to do in addition to solving all those technical problems is somehow effect the education — I don’t just mean formal education, but the way that people are thinking broadly throughout our organizations.”
Put differently, are we teaching our students the skills they needed for the last economy, or to thrive in the next one?
Creating The Analytically Enabled Organization
One of my personal heros on this topic has been Tom Davenport, who's written several excellent books on what it means to be an analytically enabled organization -- and how they tend to perform better than those that don't. I've read his books. They're spot on. And I think he was perhaps 10 years too early in his thinking.
That's understandable, H.G. Wells was a century early:
“The time may not be very remote when it will be understood that for complete initiation as an efficient citizen of one of the new great complex world wide states that are now developing, it is as necessary to be able to compute, to think in averages and maxima and minima, as it is now to be able to read and write.” H. G. Wells, Mankind in the Making (1904). Wells is commonly paraphrased as having written “statistical thinking.”
It's On Us As Leaders
The picture is clear -- if we're as leaders are going to help our organizations and those around us grow and thrive in this new era, we're going to have to do our part.
Investing in creating big data analytics is going to turn out to be the easy part; getting people to engage and change their behaviors around the new insights provided is going to be where the really heavy lifting will need to be done.
And here's the hard part: the more you think you know about something, the harder it will be to initiate meaningful and substantive change in the face of data.
Tomorrow's business champions may end up being better described as "data champions" -- strong, insightful individuals that seek out compelling insights, and are willing to buckle down to the unenviable task of proving to people that they don't really know what they think they know -- and getting them to do something about it.
And simply putting up the platform to do this work will end up the easy bit :)
Chuck,
If you haven't read "Thinking Fast and Slow" by Daniel Kahneman, it's a refreshing perspective on decision making and the underlying fundamentals that drive this constant process. He identifies two different "systems" for decision making that are running in the brain. System 1 always gets to go first. Yeap, it's the one that kept us from being eaten (fast thinking) but isn't capable of statitics-based decisions (slow thinking).
System 2 is kind of lazy and is capable of handling statistical decisions, although usually not very well since it's strongly influenced by what System 1 has concluded.
So, we may have the computing means to crunch a lot of data and find patterns that are insightful, but acting on that (making decisions) is about being human and that's another game entirely with altogether different rules. Daniel provides some clues about how to improve human decision making, but knowing you shouldn't make decisions with the first system doesn't help you stop using it as its instinctual and first in line.
When we move into social policy and very complex systems, well then its millions of people who have to become comfortable in making decisions differently, not just a few executives. That's a lot of interita.
If system 1 is defined by genetic processes and evolutionary change, then I humble submit the time required for structural changes is on the order of 5 to 10 millenia for these adaptations to make a difference.
If System 1 is not entirely genetic, but more like a cultural habit, then changing the habit may only take a half a millenium or so. (I think 500 years is a good metric for deep social changes to become dominant. Recall the history of "constitutional" rights starting with the Magna Carta, and how long it has taken for that cultural idea to spread widely.)
I suspect that system 1 won't change quickly.
Onward ...
Posted by: Brook Reams | June 08, 2012 at 10:31 AM
Brook
Thanks for the comment. More than a few people have recommended the book to me, so I guess I'll go read it now.
Me? I don't want to wait 500 years for deep social changes :)
Thanks again
-- Chuck
Posted by: Chuck Hollis | June 08, 2012 at 01:18 PM
Chuck,
Some good quotes. Here's one of my favorites that I think applies here as well:
“ Faced with the choice between changing one’s mind and proving that there is no need to do so, almost everybody gets busy on the proof.”
John Kenneth Galbraith
Posted by: Dick Sullivan | June 19, 2012 at 03:59 PM