The Age of Big Data
Big Data, the application of a new breed of analytical tools to the vast caches of data being produced by computers and other forms of technology, is on the brink of becoming a household word – thanks to new books, conferences and articles that are slated to appear over the next few months.
But as impressive as these Big Data stories will be – and many of them, like mapping the insides of F5 tornados, tracking every patient heartbeat over a lifetime and predicting the behavior of consumers from millions of store purchases – the real story is unlikely to be told.
That is, Big Data has the potential to utterly transform the relationship that individuals have with institutions, customers with companies, patients with the healthcare system, students with universities, and voters with government. And that means once it has fully penetrated society and industry, the Big Data revolution may very well prove a turning point in our economic – and ultimately, cultural – history as great as the electronics revolution. . . perhaps even as great as the first and second Industrial Revolutions.
Why? Because once the relationship of individuals to institutions transforms, the benefits to the individual consumer, citizen, patient and student will be profound.
To explain why this will be so, let’s look more closely at the impact of Big Data on the commercial world, where this revolution is likely to occur first.
The story of business over the last two centuries can be seen as the evolution of the relationship between sellers and buyers. The first Industrial revolution, at the end of the 18th century, brought mass production of standardized goods – and thus greater volume and lower cost – to the masses, supplanted the high-cost, customized goods that had characterized commerce for millennia. The second Industrial Revolution, beginning in the middle of the 19th century, brought that mass production to the creation of interchangeable components – meaning that those parts could be swapped out and pieced together into finished goods almost everywhere. . . meaning even more availability and lower prices to everyday consumers.
The digital/electronics revolution, by putting intelligence into the production process – even into the finished goods and services – made possible for the first time mass customization. That is, manufacturers and providers could create products and servers in high volume and reduced cost, and those creations could then use their on-board intelligence to adapt to the buyer’s unique needs. Think laptop computers, cellphones, the Internet, user customizable automobiles, etc. The extraordinary richness of modern life – especially as it has reached out to include three billion of the world’s people – can be largely credited to the mass customization revolution.
But now, Big Data – and the billions of high-speed, low-cost processors, computers, sensors and networks that make it possible – promises to take this relationship to the next level: mass personalization. That is, it uses all of this computing power, and the mountains of data now being produced by each person each day, and uses it to continuously and uniquely modify products and services to every customer.
And that’s only the beginning: because in this pursuit of the perfect 1-to-1 relationship with each customer, the very nature of that relationship begins to change – not least because it becomes part of the duty of the supplier to keep that relationship continuously optimal. And to do that, the manufacturer or service supplier can no longer (as with mass customization) ask for a one-way flow of information from the customer. Now for the first time it must send valuable information the other way. They have to share with customers what they know in order to get further information on what those customers want – the information they need to make their Big Data analytics even more accurate.
A pioneering example of this is Facebook. The only way that it can make money and continue to grow (and reward those frustrated new shareholders) is to personalize their offering so completely to each of its near-billion users that those users trust it so much that they give up ever more layers of personal information. . . which can then be analyzed and sold to industry. For example, again thanks to Big Data, every Web ad you will soon see on Facebook and across the Web will have been bid upon in real time by advertisers who will pay based upon your perceived value as a potential customer.
As we’ve seen with some of Facebook’s failed attempts in the past to do this, it is not an easy task. But the reality is that in our technology- and media-saturated world, they have no alternative – nor do most companies, whether they know it now or not. The new world of mass personalization requires relationships built upon mutual trust. And more than at any time in history, companies – and soon, big government, big education, big everything – are going to have to work hard to earn that trust from each and every one of us. The world now has to come to you and me as consumers – not the other way around.
That’s why Big Data is so important. It changes the social contract. And the more influential it becomes, the more its tools will be in demand. Ready or not, we are entering the Age of Big Data.
Originally posted on Forbes.com.